Crawl errors

curl https://api.stophy.dev/v1/crawl/{id}/errors \
  -H "Authorization: Bearer $STOPHY_API_KEY"

const response = await fetch("https://api.stophy.dev/v1/crawl/{id}/errors", {
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
  },
});
console.log(await response.json());

import os

import requests

response = requests.get(
    "https://api.stophy.dev/v1/crawl/{id}/errors",
    headers={"Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}"},
)
print(response.json())

{
  "success": true,
  "status": "scraping",
  "failed": 0,
  "errors": [
    {
      "retryable": true,
      "url": "<string>",
      "code": "<string>",
      "message": "<string>",
      "at": "<string>"
    }
  ],
  "robotsBlocked": [
    "<string>"
  ],
  "error": "<string>",
  "isTruncated": true
}

{
  "success": false,
  "error": "<string>",
  "code": "apiKeyRequired",
  "retryAfterSeconds": 123,
  "signupUrl": "https://stophy.dev/signup",
  "signup": "POST https://api.stophy.dev/v1/signup"
}

{
  "success": false,
  "error": "<string>",
  "code": "apiKeyRequired",
  "retryAfterSeconds": 123,
  "signupUrl": "https://stophy.dev/signup",
  "signup": "POST https://api.stophy.dev/v1/signup"
}

GET

crawl

{id}

errors

curl https://api.stophy.dev/v1/crawl/{id}/errors \
  -H "Authorization: Bearer $STOPHY_API_KEY"

const response = await fetch("https://api.stophy.dev/v1/crawl/{id}/errors", {
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
  },
});
console.log(await response.json());

import os

import requests

response = requests.get(
    "https://api.stophy.dev/v1/crawl/{id}/errors",
    headers={"Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}"},
)
print(response.json())

{
  "success": true,
  "status": "scraping",
  "failed": 0,
  "errors": [
    {
      "retryable": true,
      "url": "<string>",
      "code": "<string>",
      "message": "<string>",
      "at": "<string>"
    }
  ],
  "robotsBlocked": [
    "<string>"
  ],
  "error": "<string>",
  "isTruncated": true
}

{
  "success": false,
  "error": "<string>",
  "code": "apiKeyRequired",
  "retryAfterSeconds": 123,
  "signupUrl": "https://stophy.dev/signup",
  "signup": "POST https://api.stophy.dev/v1/signup"
}

{
  "success": false,
  "error": "<string>",
  "code": "apiKeyRequired",
  "retryAfterSeconds": 123,
  "signupUrl": "https://stophy.dev/signup",
  "signup": "POST https://api.stophy.dev/v1/signup"
}

Free.

Failed pages

Lists every page the job could not read, up to 1,000, each with url, code, message, retryable and at. isTruncated is true when there are more. Retry a page when retryable is true.

No example is recorded. The schema below lists every field.

curl https://api.stophy.dev/v1/crawl/{id}/errors \
  -H "Authorization: Bearer $STOPHY_API_KEY"

const response = await fetch("https://api.stophy.dev/v1/crawl/{id}/errors", {
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
  },
});
console.log(await response.json());

import os

import requests

response = requests.get(
    "https://api.stophy.dev/v1/crawl/{id}/errors",
    headers={"Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}"},
)
print(response.json())

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

string

required

Query Parameters

maxTextLength

integer

Cut every text value in the response to this many characters. A value that was cut ends with …, and a page whose text was cut has isTruncated set to true. A screenshot is never cut. Leave it out to get the full text.

Required range: x >= 1

Response

The data, and the credits this call used.

success

boolean

required

status

enum<string>

required

Available options:

scraping,

completed,

failed,

cancelled

failed

integer

required

Required range: -9007199254740991 <= x <= 9007199254740991

errors

object[]

required

Show child attributes

robotsBlocked

string[]

required

error

string

isTruncated

boolean

Using the API

Read pages

Crawl a site

Batch scrape

Extract

Browser tasks

Catalog

Account

Failed pages

Authorizations

Path Parameters

Query Parameters

Response

​Failed pages

Authorizations

Path Parameters

Query Parameters

Response

Failed pages