targets says what a monitor watches. It holds 1 to 50 targets, and each has a type of scrape, crawl or search. One monitor can mix types, and every check reads every target.
Make a request
Get an API key in the Quickstart. This request watches one page and judges each change against a goal. The page returns a new identifier on every read, so a check always finds a change.Read the response
This real response is cut to the fields this page is about. Stophy added anid to the target and turned the judge on because you sent a goal.
Scrape targets
A scrape target lists the pages to read on every check. An address sent twice is read once.Scrape options
scrapeOptions takes the same fields as Scrape, except url. A monitor always reads the page live, so it ignores maxAge and minAge. zeroDataRetention is refused, because a monitor keeps a snapshot of each page to compare with the next.
formats says what to compare. The first of markdown, html, rawHtml and links is compared, and markdown is the default. A page counts as changed when that text differs from the last read.
To compare fields instead of text, add a changeTracking format or a json format with a schema or a prompt, as in Change tracking and JSON. The modes of changeTracking are git-diff, which returns the text diff, and json, which returns each field before and after. With only json, a page whose text changed but whose fields did not is same. json adds $3 per 1,000 pages when it returns fields.
Crawl targets
A crawl target starts from one page. Every check finds the pages under it again and reads each one, so a page that appears later shows up as anew change.
PATCH /v1/monitor/{id}, was stored with every default filled in:
A crawl target takes
scrapeOptions too. If a check finds no pages, it fails and says so in error.
Search targets
A search target runs your queries on every check and reports results it has not seen before. It reads no page, so it has no baseline: a meaningful result is reported the first time it is found.excludeDomains set to [] and an id. A check with it returned three results and cost $0.004: $0.001 for the search and $0.001 for each result the judge read.
A search target needs a
goal, or judgeEnabled: false. Without either, the call returns 400 invalidRequest with goal: a search target needs a goal to judge results against, or judgeEnabled set to false. A result the judge finds meaningful is reported as new. A result it finds not meaningful is marked ignored and is not looked at again. A result you have already been told about is already_seen. See Checks and changes.
Goal and judge
goal is what matters to you, in plain words, up to 2,000 characters. With a goal, a model judges every changed page against it. judgeEnabled: false turns the judge off while keeping the goal.
The judgment is on the page of the check and in the webhook payload.
confidence is high, medium or low. A change the judge calls not meaningful is still recorded and still sent, with meaningful: false, so you decide what to do with it. If the judge cannot decide, judgment is null and the page is not charged for it. Judging adds $1 per 1,000 pages for each changed page it reads.
Update targets
PATCH /v1/monitor/{id} replaces targets as a whole, so send every target you want to keep. You can also change goal and judgeEnabled in the same call.
- A page you keep stays compared with its last snapshot.
- A page you add to a scrape target is read for the first time on the next check, which only records it as
newand sends no event. - A page you remove is no longer read or billed.
Limits and errors
- A monitor takes at most 50 targets and 100 pages in one scrape target. Your account takes 25 monitors and 1,000 pages in all. See Limits and errors.
- An address that is not a public website returns
400 invalidRequestwithtargets.0.urls.0: Expected a public website URL. zeroDataRetentioninsidescrapeOptionsreturns400 invalidRequestwithzeroDataRetention is not available for monitors, which keep a snapshot to compare.- Send
targetsandscheduletogether. The older fieldsurl,urls,interval,formatsandchangeTrackingstill work on their own:urlorurlsfor the pages,intervalfor seconds or text such as30m,6hor1d, andchangeTrackingfor the compare modes. Sending them together withtargetsorschedulereturns400 invalidRequest.
Related
- Schedules sets how often a monitor checks.
- Checks and changes reads what a check found.
- Webhooks tells you when a page changes.
- Change tracking compares one page with your last read, without a monitor.