Make a request
Get an API key in the Quickstart. This request lists the first 5 pages of a site.Read the response
links is a list of pages. Each has a url, and a title and description when Stophy knows them. A page with none has only its url.
Options
To narrow the list by topic, sitemap, subdomain or query string, see Filter and search.
Use the results
Pick the URLs you need and read them:- Scrape reads one URL.
- Batches read a list of URLs in the background.
- Crawl follows links from one URL on its own, without a list.
Limits and errors
- A site that cannot be reached returns
503 sourceUnavailable. sitemap: "only"on a site with no sitemap returns404 notFound. Sendsitemap: "include"to list its pages instead.- A site Stophy does not read returns
403 siteNotRead. - A URL on a private network returns
400 invalidRequest. includeSubdomains: truewithout an API key returns401 apiKeyRequired. Without an API key, the list covers one host.- A map lists what was found in the time allowed. It may not find every page of the site. A site that exposes few pages returns a short list.