Read pages
Scrape
Scrape any public web page as markdown, HTML, links, a screenshot or JSON. Send one request and get JSON back. Examples in cURL, TypeScript and Python.
POST
Best for reading a page as markdown, HTML, links or a screenshot. The page runs its JavaScript before it is read. Without formats you get markdown. Send formats as names, [“markdown”,“links”], or as objects,
[{"type":"screenshot","fullPage":true}], or mix them. For a site’s own structured data (video, social, places, shopping, jobs, homes) send formats [{"type":"json","parser":"<name>"}]; GET /v1/parsers lists the parsers. A YouTube video link comes back as its title, channel, date, length, views, likes and description.
Works without an API key, up to 10 calls a minute and 50 a day per IP. See Rate limits.
Price
1 credit for any page, however it is reached. json with a schema or a prompt, summary and highlights add 4 credits each when they return something. CSS rules and attributes add nothing. A PDF adds 1 credit for each page read. Each perform action adds 2 credits. An answer from a stored copy within maxAge is free. A page that comes back with a 404 or 410 status is charged like any page. A page that cannot be read costs nothing. Each page of a urls call is priced on its own. creditsUsed shows what the call cost. See Pricing for what credits cost.Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Query Parameters
Cut every text value in the response to this many characters. A value that was cut ends with …, and a page whose text was cut has isTruncated set to true. A screenshot is never cut. Leave it out to get the full text.
Required range:
x >= 1Body
application/json
The page or PDF to read, such as https://bun.com/docs.
What to return: markdown (clean page text), summary (a short summary of the page), html (the cleaned page), rawHtml (the page as sent), links (every link on the page), images (every image address), screenshot (a link to a JPEG of the visible page, kept for 7 days), changeTracking (what changed in the markdown since your key last read this page with the same options, kept 30 days) network (the JSON the page loaded while it rendered, up to 30 responses) and branding (the page's colors, fonts, typography, spacing, button and input styles, logo and brand images, read from how the page looks, with a field left out when the page does not show it). Send { "type": "screenshot", "fullPage": true } for the whole page. Send { "type": "highlights", "query": "refund policy" } for the passages that best match a query. Send { "type": "json", "schema": { "type": "object", "properties": { "price": { "type": "number" } } } } or { "type": "json", "prompt": "the price and the stock" } for fields filled from the page, each checked against it. Send { "type": "json", "parser": "amazon-product" } to get a site's own data as json instead of reading the page; GET /v1/parsers lists the parsers. Send { "type": "json", "rules": { "title": "h1", "links": { "selector": "a", "type": "list", "output": "@href" } } } to read the page with CSS rules and no model, at the page's own price. Send { "type": "attributes", "selectors": [{ "selector": "a", "attribute": "href" }] } for attribute values, returned in attributes. Without a parser or rules, json needs a schema or a prompt and is filled from the page. json with a schema or a prompt, summary and highlights cost 4 credits more each when they return something. json with rules or a parser, and attributes, cost nothing more. A screenshot, network or branding read costs 1 credit, the same as any page. A PDF comes back as markdown only, and costs 1 credit more for each page read.
Required array length:
1 - 20 elementsAvailable options:
markdown, summary, html, rawHtml, links, images, network, branding, screenshot, changeTracking Keep only the page's main content, without menus, headers, footers and ads.
Keep only the parts of the page that match these CSS selectors, such as article or .price. When set, onlyMainContent is not applied.
Minimum array length:
1Minimum string length:
1Remove the parts of the page that match these CSS selectors, such as .comments.
Minimum array length:
1Minimum string length:
1Accept a stored answer up to this many milliseconds old, at most 30 days. Without it, 2 days, or less for a site whose pages change often, and a copy up to twice that age (at most 7 days) comes back at once as stale while a fresh read runs. 0 always reads the page live. metadata.cacheState says which one you got.
Required range:
0 <= x <= 9007199254740991Answer only from a stored copy that is at least this many milliseconds old, and never read the page live. 0 accepts any stored copy. A page with no such copy is notFound.
Required range:
0 <= x <= 9007199254740991Extra milliseconds to wait before reading, up to 60000, for pages that load their content late. To wait for an element, add a wait action with a selector. A read that waits costs 1 credit, the same as any page.
Required range:
0 <= x <= 60000How long to keep trying, in milliseconds, between 10 seconds and 2 minutes. Pages behind a bot check can need 10 to 30 seconds the first time.
Required range:
1000 <= x <= 300000Only ["pdf"] is supported: PDFs are read as markdown.
Minimum array length:
1Allowed value:
"pdf"Up to 50 steps to run on the page, in order, before reading it: { "type": "wait", "milliseconds": 500 } (up to 30 seconds), { "type": "wait", "selector": ".results" }, { "type": "click", "selector": "button.more" }, { "type": "write", "text": "tides" }, { "type": "press", "key": "Enter" }, { "type": "scroll", "direction": "down" } (also up, left or right; amount is pixels, "viewport" or "max"; maxScrolls 1 to 50; selector scrolls an inner container; "max" keeps scrolling until the page stops growing, for feeds that load as you go), { "type": "screenshot", "fullPage": true } (links come back in actions.screenshots), { "type": "scrape" } (the page at that point, in actions.scrapes), { "type": "executeJavascript", "script": "document.title" } (results in actions.javascriptReturns), { "type": "pdf", "format": "A4" } (links in actions.pdfs) or { "type": "perform", "action": "Click the Load more button" } (a step in plain words, up to 5 browser steps on this page, never leaving the site; at most 5 per call). A read with actions costs 1 credit, the same as any page, and each perform action costs 2 credits more. If an action fails, you still get the page as it stood, with a warning that names the action and says why.
Required array length:
1 - 50 elements- Option 1
- Option 2
- Option 3
- Option 4
- Option 5
- Option 6
- Option 7
- Option 8
- Option 9
- Option 10
- Option 11
Where to read the page from. The default is the United States.
Allowed value:
"auto"Send false to neither read nor keep a stored copy of this page.
Read the page with the cookies and local storage that earlier calls saved under this name, kept for your account for 30 days after the last use. A call with a profile always loads the page in a browser, costs 1 credit, the same as any page, and is never answered from a stored copy. Only one call at a time can save to a profile.
Send true to keep nothing from this call: no stored copy is read or made, and nothing from it is logged. Costs 1 credit more per page. Not with a screenshot.
How links appear in the markdown. inline keeps text [blocked]. numbered writes Pricing⟨3⟩ and lists each address once at the end. none keeps only the link text, which makes link-heavy pages 10 to 70% shorter; ask for the links format to get the addresses.
Available options:
inline, numbered, none Cut the markdown at this many characters; isTruncated says when that happened.
Required range:
500 <= x <= 1000000