Skip to main content
Add scrapeOptions to a search to read each web and news result, or set highlights to get the passages that match your query. A result stays in the answer even when its page cannot be read.

Read each result

Get an API key in the Quickstart. Reading result pages needs an API key. This request reads two results as markdown.

Read the response

Each web row gets markdown and metadata, the same as a scrape of that page. metadata.costUsd is what reading that page cost. This real response is cut to one row. The Markdown was 38,988 characters long.
costUsd is $0.003 for two results: $0.001 for the search and $0.001 for each page. A format other than markdown, such as summary, html or links, appears as its own field on each row. See Scrape for every format. With scrapeOptions.formats, the description of a web row is filled from the page.

Get highlights

Send highlights as true to get the passages of each page that best match query. They replace the description, in Markdown, up to about 2,000 characters. A result whose page Stophy has already read gets passages. The others keep their usual description. Highlights add nothing to the price.
This real response is cut to two of the three rows. The first row has no description to show. In the second, the … between passages marks where text was skipped.
To get the passages as their own list, with your own query and count, add a highlights format with a query to scrapeOptions.formats. See Highlights.

Read each outcome

A page that cannot be read does not fail the search. The row keeps its url, title and description, has no markdown, and its metadata.error says why. A row with an error is not charged for the page. This real call asked for a site that Stophy does not read as a page.
Check metadata.error before you use markdown. The request was {"query": "site:linkedin.com bun runtime", "limit": 2, "scrapeOptions": {"formats": ["markdown"]}}, and the response is cut to one of its two rows. These are the other messages you can meet:

Options

These fields live inside scrapeOptions. It takes the options of Scrape, so every Scrape page applies here.

Limits and errors

  • Only web and news results are read. scrapeOptions.formats with images in sources returns 400 invalidRequest. The error says Page text is only added to web and news results. Drop images.
  • Without an API key, scrapeOptions returns 400 invalidRequest with scrapeOptions: Reading result pages needs an API key. Get one at https://stophy.dev/signup, or leave scrapeOptions out.
  • Up to 10 pages are read at once. The search timeout is 60,000 ms by default when pages are read.

Price

$1 per 1,000 results whose page is read, on top of the search price. `summary` and `json` with a schema or a prompt add $3 per 1,000 more each when they return something. A result with a metadata.error is not charged for its page. See Pricing.