Skip to main content
A batch and its pages are kept for 24 hours. Check the batch until its status is final, then read its pages, its errors, or both. All of these calls are free.

Check status

Get an API key in the Quickstart. GET /v1/batch/scrape/{id} returns the counts and the finished pages. Poll it every few seconds while status is scraping.
This real batch read one page and could not read the other, because its domain does not exist. The page text is cut.
A page that answered 404 or 410 is not an error. It counts as completed, its metadata.error says what happened, and it costs nothing.

Read the results in parts

GET /v1/batch/scrape/{id}/results returns the same pages in a shape made for paging. Send limit and cursor.
This real answer used limit 2. The page text is cut. The cURL tab reads one part, and the TypeScript and Python tabs read every part, 100 results at a time.
limit is 20 by default and at most 100. Send cursor from the answer, with the same limit, to read the next part. A part can hold fewer results than limit, so keep going until there is no cursor. Each result has its own url, creditsUsed and costUsd. Pages that failed show up in the errors of the part where they landed. This real second part of a two-page batch had no results and one error.

Read the errors

GET /v1/batch/scrape/{id}/errors lists only the pages that could not be read, with the reason for each.
code is one of the error codes. retryable: true means a new batch with that URL can work. robotsBlocked lists pages the site’s robots rules kept Stophy from reading. At most 1,000 errors are listed, and isTruncated is true when there were more. See Limits and errors.

Delete the results

DELETE /v1/batch/scrape/{id}/results removes the stored pages at once. It works once the batch is completed, failed or cancelled. The batch keeps its counts.
After the delete, reading the results answers with the counts, an empty results, and message set to The stored results were deleted. Deleting again returns deletedPages of 0. A batch that is still running returns 400 invalidRequest: This batch scrape has not finished, so its results cannot be deleted yet. Wait until its status is completed, failed or cancelled, then send the call again. To stop a batch first, see Limits and errors.

List your batches

GET /v1/batch/scrape lists your batches, newest first, and is free.
A page can hold fewer batches than limit, so keep sending cursor until it is missing. A batch that stopped early has its reason in message. Run this to confirm that no batch you started is still reading.

Options

Limits and errors

  • A batch, its pages and its errors are kept for 24 hours after it starts. After that, reading it returns 404 notFound: No batch scrape with that id. Results are kept for 24 hours. Copy what you need before then.
  • A status read returns at most 100 pages or 2 MB in one answer.
  • An id that is not yours returns 404 notFound too.