> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Submit a batch

> Send up to 10,000 URLs with the formats and options every page uses, add a webhook, add URLs to a running batch, and retry a submit without a duplicate.

`POST /v1/batch/scrape` takes a list of `urls` and the options every page shares, and answers with an `id`. This page covers what you can send. [Results](https://docs.stophy.dev/batches/results.md) covers how to read the pages.

## Send a list

Get an API key in the [Quickstart](https://docs.stophy.dev/quickstart). The options are the ones [Scrape](https://docs.stophy.dev/scrape/overview) takes, sent at the top level next to `urls`. This request reads two pages as `markdown` and `links`, keeping only the main content.

```bash cURL theme={null}
curl -X POST https://api.stophy.dev/v1/batch/scrape \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://bun.com/docs/runtime/env",
      "https://bun.com/docs/runtime/bunfig"
    ],
    "formats": ["markdown", "links"],
    "onlyMainContent": true,
    "maxAge": 0
  }'
```

```typescript TypeScript theme={null}
const response = await fetch("https://api.stophy.dev/v1/batch/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    urls: [
      "https://bun.com/docs/runtime/env",
      "https://bun.com/docs/runtime/bunfig",
    ],
    formats: ["markdown", "links"],
    onlyMainContent: true,
    maxAge: 0,
  }),
});
const { id } = await response.json();
console.log(id);
```

```python Python theme={null}
import os

import requests

response = requests.post(
    "https://api.stophy.dev/v1/batch/scrape",
    headers={"Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}"},
    json={
        "urls": [
            "https://bun.com/docs/runtime/env",
            "https://bun.com/docs/runtime/bunfig",
        ],
        "formats": ["markdown", "links"],
        "onlyMainContent": True,
        "maxAge": 0,
    },
)
print(response.json()["id"])
```

The answer is the one in the [overview](https://docs.stophy.dev/batches/overview#read-the-response): `id`, `url`, `invalidURLs` and `costUsd`. A batch that waits for a free place also has `status` set to `scraping` and a `queuePosition`.

## Options

| Field | Type | Default | Effect |
| - | - | - | - |
| `urls` | `array of string` | none | The pages or PDFs to read, 1 to 10,000, each up to 2,048 characters. An address sent twice is read once. Required. |
| `formats` | `array` | `markdown` | What to return for every page. See [Scrape](https://docs.stophy.dev/scrape/overview). |
| `ignoreInvalidURLs` | `boolean` | `true` | Skip addresses that are not public website URLs and list them in `invalidURLs`. `false` refuses the whole batch instead. |
| `maxConcurrency` | `number` | none | The most pages read at once, 1 to 100. Reads run at most 4 at once, so a larger number gives the same speed. |
| `webhook` | `string` or `object` | none | Where to be told about the batch. See below. |
| `appendToId` | `string` | none | The id of a batch that is still running: add `urls` to it instead of starting a new batch. |

Every scrape option works on a batch, and applies to every page. Those include `onlyMainContent`, `includeTags`, `excludeTags`, `maxAge`, `waitFor`, `mobile`, `timeout`, `actions`, `location`, `profile`, `maxChars`, `query`, `checkPromptInjection`, `redactPII` and `zeroDataRetention`. `timeout` is how long each page may take, in milliseconds. Browse them in [Scrape](https://docs.stophy.dev/scrape/overview). A batch takes `urls` in place of the single `url`, and has no `offset`. See the [API reference](https://docs.stophy.dev/api-reference/endpoint/batch-scrape.md) for the full list.

## Get told when pages finish

Send `webhook` as an address, or as an object, and Stophy sends a JSON `POST` when the batch starts, for each finished page and when it ends.

```json theme={null}
{
  "urls": ["https://bun.com/docs/runtime/env"],
  "webhook": {
    "url": "https://app.example.com/hooks/stophy",
    "headers": { "X-Source": "batch-import" },
    "metadata": { "import": "2026-11" },
    "events": ["page", "completed"]
  }
}
```

| Field | Type | Effect |
| - | - | - |
| `url` | `string` | A public address that answers with a `2xx`. Required. |
| `headers` | `object` | Sent with every call. Up to 10, with values up to 1,000 characters. |
| `metadata` | `object` | Comes back in every call. Up to 4,000 characters. |
| `events` | `array of string` | Which of `started`, `page`, `completed`, `failed` and `cancelled` to send. All by default. |

Each call is signed, and a call that gets no `2xx` is sent again, up to 3 times. [Webhooks](https://docs.stophy.dev/webhooks.md) explains how to verify the signature. If your address answers slowly, a `page` event can carry only the page's `url`. Read that page from the batch. An address that is not a public URL returns `400 invalidRequest`.

## Add URLs to a running batch

Send `appendToId` with the new `urls` while the batch is still running. The batch keeps its own options and webhook, and the new URLs are read with them. A URL the batch already has is skipped.

```json theme={null}
{
  "appendToId": "job_rmBUA_bFeF_ah47HLpgftQ",
  "urls": ["https://bun.com/docs/runtime/templating"]
}
```

The answer has the same `id`. This real batch started with 4 URLs, and its status then showed `total` 5 and all five pages in `data`. When the batch is over, the call returns `409 jobFinished`: `This batch scrape already finished, so pages cannot be added. Start a new batch, or read the pages with GET /v1/batch/scrape/{id}.` The most pages in one batch is still 10,000, new ones included.

## Send the same batch twice safely

Add an `Idempotency-Key` header, any text you choose. If the same key and the same body arrive again within 24 hours, Stophy returns the first answer instead of starting a second batch.

```bash cURL theme={null}
curl -X POST https://api.stophy.dev/v1/batch/scrape \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: site-import-2026-11-17" \
  -d '{ "urls": ["https://example.com"], "formats": ["markdown"] }'
```

```typescript TypeScript theme={null}
const response = await fetch("https://api.stophy.dev/v1/batch/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
    "Content-Type": "application/json",
    "Idempotency-Key": "site-import-2026-11-17",
  },
  body: JSON.stringify({ urls: ["https://example.com"], formats: ["markdown"] }),
});
console.log(response.headers.get("idempotent-replayed"), await response.json());
```

```python Python theme={null}
import os

import requests

response = requests.post(
    "https://api.stophy.dev/v1/batch/scrape",
    headers={
        "Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}",
        "Idempotency-Key": "site-import-2026-11-17",
    },
    json={"urls": ["https://example.com"], "formats": ["markdown"]},
)
print(response.headers.get("idempotent-replayed"), response.json())
```

Sent twice, both answers had the same `id`, and the second had the header `idempotent-replayed: true`. Only one batch ran. A different body with the same key starts a new batch, so keep the key and the body together when you retry. Use it for every submit you might send again after a timeout.

## Price

A page costs what a scrape costs: $1 per 1,000 pages, plus the extras of the options you send. `checkPromptInjection` adds $2.50 per 1,000 pages when the scan runs. `json` with a schema or a prompt, `summary`, `highlights`, `question`, `product`, `menu` and `changeTracking` in `json` mode add $3 per 1,000 pages each when they return something. A PDF adds $0.50 per 1,000 pages of text from its text layer and \$2.50 per 1,000 pages of text read from its picture, that come back. Browser actions and CSS rules add nothing. A page that comes back with a 404 or 410 status, or cannot be read, costs nothing. Each page is charged when it finishes, and starting the job is free. See [Pricing](https://docs.stophy.dev/account/pricing.md).

## Limits and errors

* A batch holds at most 10,000 URLs. More returns `400 invalidRequest`: `A batch holds at most 10000 pages. Send fewer.`
* An empty `urls` returns `400 invalidRequest`: `urls: Send at least 1 item.` A list with no public website URL returns `400 invalidRequest`: `urls: None of these is a public website URL.`
* With `ignoreInvalidURLs: false`, one bad address returns `400 invalidRequest`: `urls: Expected public website URLs. Not valid: ftp://example.com/file.txt`.
* Too many jobs returns `429 rateLimited`. See [Limits and errors](https://docs.stophy.dev/batches/limits-and-errors.md).
* Your balance is checked as pages are read, not when you submit. A batch that runs out of balance ends as `failed` with `Your balance ran out. The pages already read are kept. Add funds, then start a new job for the pages left.` Adding URLs with `appendToId` returns `402 insufficientCredits` when your balance cannot cover them.

## Related

* [Batches](https://docs.stophy.dev/batches/overview) explains what a batch is and what it costs.
* [Results](https://docs.stophy.dev/batches/results.md) reads the pages, the errors and the list of batches.
* [Limits and errors](https://docs.stophy.dev/batches/limits-and-errors.md) covers cancelling and jobs at once.
* [Scrape](https://docs.stophy.dev/scrape/overview) has every format and option.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.