Web web Map API

Scrape the list of pages on a website

curl --request POST \
  --url https://api.stophy.dev/v1/web/map \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "url": "https://bun.com",
  "search": "sqlite"
}
'

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({url: 'https://bun.com', search: 'sqlite'})
};

fetch('https://api.stophy.dev/v1/web/map', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://api.stophy.dev/v1/web/map"

payload = {
    "url": "https://bun.com",
    "search": "sqlite"
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "success": true,
  "data": {
    "results": [
      {
        "url": "<string>",
        "title": "<string>"
      }
    ],
    "page": 0,
    "totalPages": 0
  },
  "creditsUsed": 123,
  "requestId": "<string>"
}

{
  "success": false,
  "error": {
    "code": "cliSessionCompleted",
    "message": "<string>",
    "retryable": true,
    "requestId": "<string>",
    "retryAfterSeconds": 123,
    "requiredCredits": 123,
    "availableCredits": 123,
    "billingUrl": "https://stophy.dev/billing"
  }
}

{
  "success": false,
  "error": {
    "code": "cliSessionCompleted",
    "message": "<string>",
    "retryable": true,
    "requestId": "<string>",
    "retryAfterSeconds": 123,
    "requiredCredits": 123,
    "availableCredits": 123,
    "billingUrl": "https://stophy.dev/billing"
  }
}

POST

web

map

Scrape the list of pages on a website

curl --request POST \
  --url https://api.stophy.dev/v1/web/map \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "url": "https://bun.com",
  "search": "sqlite"
}
'

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({url: 'https://bun.com', search: 'sqlite'})
};

fetch('https://api.stophy.dev/v1/web/map', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://api.stophy.dev/v1/web/map"

payload = {
    "url": "https://bun.com",
    "search": "sqlite"
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "success": true,
  "data": {
    "results": [
      {
        "url": "<string>",
        "title": "<string>"
      }
    ],
    "page": 0,
    "totalPages": 0
  },
  "creditsUsed": 123,
  "requestId": "<string>"
}

{
  "success": false,
  "error": {
    "code": "cliSessionCompleted",
    "message": "<string>",
    "retryable": true,
    "requestId": "<string>",
    "retryAfterSeconds": 123,
    "requiredCredits": 123,
    "availableCredits": 123,
    "billingUrl": "https://stophy.dev/billing"
  }
}

{
  "success": false,
  "error": {
    "code": "cliSessionCompleted",
    "message": "<string>",
    "retryable": true,
    "requestId": "<string>",
    "retryAfterSeconds": 123,
    "requiredCredits": 123,
    "availableCredits": 123,
    "billingUrl": "https://stophy.dev/billing"
  }
}

Scrape the list of pages on a website.

Request

url is required.

This endpoint can be called without an API key, subject to per-address limits. Pagination requires a key.

import { Stophy } from "stophy";

const stophy = new Stophy({ apiKey: process.env.STOPHY_API_KEY });
const result = await stophy.web.map({ url: "https://bun.com", search: "sqlite" });
console.log(result.data.results);

from stophy import Stophy

stophy = Stophy()  # reads STOPHY_API_KEY
result = stophy.web.map(url="https://bun.com", search="sqlite")
print(result["data"]["results"])

curl -X POST https://api.stophy.dev/v1/web/map \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "content-type: application/json" \
  -d '{"url":"https://bun.com","search":"sqlite"}'

stophy web map https://bun.com --search sqlite

Response

If successful, the response body contains the standard envelope. data contains results, page and totalPages. The following example is a real response, truncated: each list shows one item and long strings are cut.

{
  "success": true,
  "data": {
    "results": [
      {
        "url": "https://bun.com/docs/runtime/sqlite",
        "title": "SQLite | Bun Docs"
      }
    ],
    "page": 1,
    "totalPages": 1
  },
  "creditsUsed": 1,
  "requestId": "2fe9dbad-fc48-4890-9751-0b24b596035b"
}

Field	Type	Description
`page`	integer	The page number of this response.
`totalPages`	integer	The number of pages that match the request.

For every field, see the response schema on this page.

Add search to rank pages about one topic first.

Optional parameters

Parameter	Type	Description
`search`	string	Keep only addresses that mention these words, best matches first, such as pricing.
`includeSubdomains`	boolean	Also list pages on subdomains, such as docs.bun.com for bun.com. If unset, defaults to `false`.
`page`	integer	The page to return. Pages are numbered from 1. Must be between 1 and 100. If unset, defaults to `1`.

Billing

Each successful request consumes 1 credit, regardless of the number of items returned. Requests that fail, or that return no items, are not billed.

Pagination

Results are paginated. To retrieve the next page, increment page. An empty data.results indicates the last page. The maximum value of page is 100.

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Query Parameters

maxTextLength

integer

Cut every text value in the response to this many characters. A value that was cut ends with …, and a page whose text was cut has isTruncated set to true. A screenshot is never cut. Leave it out to get the full text.

Required range: x >= 1

Body

application/json

url

string

required

Any page of the site to map, such as https://bun.com.

string

Keep only addresses that mention these words, best matches first, such as pricing.

Required string length: 1 - 200

includeSubdomains

boolean

default:false

Also list pages on subdomains, such as docs.bun.com for bun.com.

page

integer

default:1

Page number, starting at 1.

Required range: 1 <= x <= 100

Response

The data, and the credits this call used.

success

any

required

data

object

required

Show child attributes

creditsUsed

integer

required

requestId

string

required

Using the API

Catalog

Account

Search and AI answers

Video

Social

Places and travel

Jobs

Shopping

Apps

Real estate

Ads

Web web Map API

Request

Response

Optional parameters

Billing

Authorizations

Query Parameters

Body

Response

​Request

​Response

​Optional parameters

​Billing

​Pagination

Authorizations

Query Parameters

Body

Response

Request

Response

Optional parameters

Billing

Pagination