> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape

> Read any public web page as Markdown, HTML, links, a screenshot or JSON in one request. Pick formats, shape the visit, and pay $1 per 1,000 pages.

Scrape reads one URL and returns the outputs you list in `formats`. Send several formats in one request. Without `formats`, you get `markdown`.

## Make a request

Get an API key in the [Quickstart](https://docs.stophy.dev/quickstart). This request returns the main content of a docs page as Markdown, plus every link on it.

```bash cURL theme={null}
curl -X POST https://api.stophy.dev/v1/scrape \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://bun.com/docs/runtime/bunfig",
    "formats": ["markdown", "links"],
    "onlyMainContent": true
  }'
```

```typescript TypeScript theme={null}
const response = await fetch("https://api.stophy.dev/v1/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.STOPHY_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://bun.com/docs/runtime/bunfig",
    formats: ["markdown", "links"],
    onlyMainContent: true,
  }),
});
const { data } = await response.json();
console.log(data.metadata.title);
console.log(data.markdown);
```

```python Python theme={null}
import os

import requests

response = requests.post(
    "https://api.stophy.dev/v1/scrape",
    headers={"Authorization": f"Bearer {os.environ['STOPHY_API_KEY']}"},
    json={
        "url": "https://bun.com/docs/runtime/bunfig",
        "formats": ["markdown", "links"],
        "onlyMainContent": True,
    },
)
data = response.json()["data"]
print(data["metadata"]["title"])
print(data["markdown"])
```

You can try Scrape without an API key, up to 10 calls a minute and 50 calls a day per IP address. See [Rate limits](https://docs.stophy.dev/production/rate-limits.md).

## Read the response

Each format you asked for is a field of `data`. This real response is cut to a few lines. The full Markdown was 48,826 characters, and `links` held 105 URLs.

```json theme={null}
{
  "success": true,
  "data": {
    "markdown": "# bunfig.toml\n\nConfigure Bun's behavior using its configuration file bunfig.toml\n\n…",
    "links": [
      "https://bun.com/docs/runtime/bunfig",
      "https://bun.com/",
      "https://bun.com/docs"
    ],
    "metadata": {
      "title": "bunfig.toml | Bun Docs",
      "description": "Configure Bun's behavior using its configuration file bunfig.toml",
      "language": "en",
      "sourceURL": "https://bun.com/docs/runtime/bunfig",
      "url": "https://bun.com/docs/runtime/bunfig",
      "statusCode": 200,
      "contentType": "text/html",
      "cacheState": "hit",
      "scrapeId": "1d83b735-d666-4ca5-84cc-ae7efedc7385",
      "costUsd": 0.001
    }
  },
  "id": "1d83b735-d666-4ca5-84cc-ae7efedc7385",
  "costUsd": 0.001,
  "requestId": "1d83b735-d666-4ca5-84cc-ae7efedc7385"
}
```

`metadata` describes the page whatever formats you ask for: its title, description, language, status code and social tags. See [Page metadata](https://docs.stophy.dev/scrape/page-metadata.md). `costUsd` is what the call cost. To read the same result again within 24 hours, send `GET /v1/scrape/{id}` with the `id`. See [Freshness and caching](https://docs.stophy.dev/scrape/freshness-and-caching.md).

## Formats

| Format | What you get | Guide |
| - | - | - |
| `markdown` | The page text as markdown | [Markdown](https://docs.stophy.dev/scrape/markdown.md) |
| `html`, `rawHtml` | The cleaned page, or the page as sent | [HTML](https://docs.stophy.dev/scrape/html.md) |
| `screenshot` | A link to a JPEG of the page, kept for 7 days | [Screenshot](https://docs.stophy.dev/scrape/screenshot.md) |
| `links` | Every link on the page | [Links](https://docs.stophy.dev/scrape/links.md) |
| `images` | Every image URL on the page | [Images](https://docs.stophy.dev/scrape/images.md) |
| `rawBase64` | The bytes the site sent, Base64 encoded | [Bytes](https://docs.stophy.dev/scrape/bytes.md) |
| `json` with `parser` | A site's own data, like a video, a listing or a profile | [Pre-built parsers](https://docs.stophy.dev/scrape/prebuilt-parsers) |
| `summary` | A short summary of the page | [Summary](https://docs.stophy.dev/scrape/summary.md) |
| `highlights` | The passages that best match a query | [Highlights](https://docs.stophy.dev/scrape/highlights.md) |
| `question` | The answer to one question. Left out, with a warning, when the page does not state one | [Question](https://docs.stophy.dev/scrape/question.md) |
| `json` with `schema` or `prompt` | An object in the shape you describe | [JSON](https://docs.stophy.dev/scrape/json.md) |
| `product`, `menu` | A product with its variants, or a menu with prices | [Product](https://docs.stophy.dev/scrape/product.md) |
| `branding` | The page's colors, fonts, logo and styles | [Branding](https://docs.stophy.dev/scrape/branding.md) |
| `changeTracking` | What changed since you last read the page | [Change tracking](https://docs.stophy.dev/scrape/change-tracking.md) |
| `network` | The JSON the page loaded while it rendered | [Network](https://docs.stophy.dev/scrape/network.md) |

## Shape the page visit

Run [browser actions](https://docs.stophy.dev/scrape/browser-actions.md) before the page is read, or keep working on it in a [browser session](https://docs.stophy.dev/scrape/interact.md). Set [waiting and rendering](https://docs.stophy.dev/scrape/waiting-and-rendering.md), keep only the parts you want with [content filtering](https://docs.stophy.dev/scrape/content-filtering.md), and pick a country or send headers with [location and headers](https://docs.stophy.dev/scrape/location-and-headers.md). Read [PDFs and documents](https://docs.stophy.dev/scrape/pdfs-and-documents.md) the same way as pages. Choose how old a stored copy may be with [freshness and caching](https://docs.stophy.dev/scrape/freshness-and-caching.md), scan for prompt injection or redact personal data with [safety](https://docs.stophy.dev/scrape/safety.md), and handle failures with [timeouts and errors](https://docs.stophy.dev/scrape/timeouts-and-errors.md). The full field list is in the [API reference](https://docs.stophy.dev/api-reference/endpoint/scrape.md).

## Price

$1 per 1,000 pages. A page that comes back with a 404 or 410 status, or cannot be read, costs nothing. `json` with a schema or a prompt, `summary`, `highlights`, `question`, `product`, `menu` and `changeTracking` in `json` mode each add $3 per 1,000 pages when they return something. Browser actions and CSS rules add nothing. See [Pricing](https://docs.stophy.dev/account/pricing.md).

## Choose another API

* [Map](https://docs.stophy.dev/map/overview) lists the pages of a site without reading them.
* [Crawl](https://docs.stophy.dev/crawl/overview) reads every page of a site in the background.
* [Batches](https://docs.stophy.dev/batches/overview) read a list of URLs in the background.
* [Search](https://docs.stophy.dev/search/overview) finds pages from a query and can read each result.
* [Extract](https://docs.stophy.dev/extract/overview) pulls the fields you name from one or more pages, each checked against the page.
* [Parse documents](https://docs.stophy.dev/parse/overview) turns a file you upload into Markdown.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.