> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Describe a crawl

> Turn a plain-language request such as only the blog posts into the options a crawl takes, without starting one. Check the options, then send them.

`POST /v1/crawl/params-preview` turns what you want into crawl options. Send a site and a sentence, read the options it picks, and pass them to [Crawl](https://docs.stophy.dev/crawl/overview) yourself. Nothing is crawled and no page is read.

## Make a request

Get an API key in the [Quickstart](https://docs.stophy.dev/quickstart). This request asks for the blog posts of a site, two links deep.

```bash cURL theme={null}
curl -X POST https://api.stophy.dev/v1/crawl/params-preview \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://bun.com",
    "prompt": "only the blog posts, two links deep"
  }'
```

```typescript TypeScript theme={null}
const response = await fetch("https://api.stophy.dev/v1/crawl/params-preview", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://bun.com",
    prompt: "only the blog posts, two links deep",
  }),
});
const { data } = await response.json();
console.log(data);
```

```python Python theme={null}
import os

import requests

response = requests.post(
    "https://api.stophy.dev/v1/crawl/params-preview",
    headers={"Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}"},
    json={
        "url": "https://bun.com",
        "prompt": "only the blog posts, two links deep",
    },
)
print(response.json()["data"])
```

## Read the response

`data` holds the crawl options the request maps to. A field the sentence does not mention is left out.

```json theme={null}
{
  "success": true,
  "data": {
    "url": "https://bun.com/",
    "includePaths": ["blog/.*"],
    "maxDiscoveryDepth": 2
  },
  "costUsd": 0.003,
  "creditsUsed": 2,
  "requestId": "2d1687ce-601b-4ad6-9757-79d9030b22da"
}
```

Check the options, change what you want, and send them to `POST /v1/crawl`:

```bash theme={null}
curl -X POST https://api.stophy.dev/v1/crawl \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://bun.com/",
    "includePaths": ["blog/.*"],
    "maxDiscoveryDepth": 2,
    "limit": 20
  }'
```

## Options

| Field | Type | Default | Effect |
| - | - | - | - |
| `url` | `string` | none | The site to crawl, such as `https://bun.com`. Required. |
| `prompt` | `string` | none | What you want from the crawl, such as `only the blog posts`, 1 to 1,000 characters. Required. |

The answer can hold these fields, each described in [Scope and limits](https://docs.stophy.dev/crawl/scope.md): `url`, `includePaths`, `excludePaths`, `maxDiscoveryDepth`, `crawlEntireDomain`, `allowSubdomains`, `allowExternalLinks`, `sitemap`, `ignoreQueryParameters` and `limit`. It never sets `scrapeOptions`, so choose the page [formats and filters](https://docs.stophy.dev/crawl/page-content.md) yourself.

## Limits and errors

* The options are a suggestion made from your sentence. Read them before you start a crawl.
* Each pattern in `includePaths` and `excludePaths` is checked as a valid regular expression, and a pattern that is not valid is dropped. At most 20 patterns of each are returned.
* `maxDiscoveryDepth` is at most 10, and `limit` is at most 10,000.
* A request that finds nothing to set returns only `url` and costs nothing.
* The call needs an API key. Without one, it returns `401 apiKeyRequired`.

## Price

\$3 per 1,000 calls. A call that finds nothing to set is free. See [Pricing](https://docs.stophy.dev/account/pricing.md).

## Related

* [Crawl overview](https://docs.stophy.dev/crawl/overview) starts a crawl with these options.
* [Scope and limits](https://docs.stophy.dev/crawl/scope.md) describes every field in the answer.
* [Page content](https://docs.stophy.dev/crawl/page-content.md) chooses what each crawled page returns.
* [Map](https://docs.stophy.dev/map/overview) lists a site's pages before you crawl.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.