Crawlers

curl --request GET \
  --url 'https://truestat.io/api/v1/sites/{siteId}/ai/crawlers' \
  --header 'Authorization: Bearer <token>'

{
  "data": [
    {
      "views": 450,
      "purpose": "ai_training",
      "operator": "gpt",
      "spoofed_views": 3,
      "verified_views": 447
    },
    {
      "views": 470,
      "purpose": "ai_training",
      "operator": "claude",
      "spoofed_views": 0,
      "verified_views": 467
    }
  ],
  "meta": {
    "page": {
      "limit": 100,
      "offset": 0,
      "has_more": false
    },
    "range": {
      "to": "2026-09-16T13:40:00.000Z",
      "tz": "UTC",
      "from": "2026-09-09T13:40:00.000Z",
      "clamped": false
    },
    "site_id": "6fed8980-a119-4a9c-a4d4-53e448ef9f9e"
  }
}

Every AI crawler that fetched a page, grouped by who it was and why it came.

A user-agent is a claim, not a fact

Anything can call itself GPTBot. Setting a header takes one line, and a scraper that wants to look legitimate sets it.

So every view here is checked: TrueStat takes the address the request came from, asks DNS what name it belongs to, and asks that name what address it resolves to. A crawler that is who it says it is survives the round trip.

That check sorts your traffic into three groups, and the third is the one worth understanding:

That third group has no field. Subtract the two from views to get it.

Why the unverifiable group is not reported as a problem

It would be easy to show a number called "unverified" in red and let you draw a conclusion from it. It would also be misleading.

Some operators publish their address ranges and support reverse DNS. Others publish nothing at all. A crawler in the second group is not behaving worse than one in the first — it simply cannot be checked, by any tool. Counting those as suspicious would blame a crawler for its operator's documentation, and counting them as genuine would be a guess made on your behalf.

We report what was proved, what was disproved, and leave the remainder plainly unclaimed.

One operator, several rows

purpose is why the crawler came, and the same operator crawls for different reasons:

• ai_training — collecting pages to train on later

• search — building an index

• ai_assistant — fetching a page to answer somebody's question right now

• other — a recognised crawler with an unrecognised purpose

Your dashboard labels that third one AI answers, which reads better on a screen. The API returns the stored value, and it is the one to filter on.

So gpt appears once under ai_training and again under search. Sum the rows for an operator's total, or call traffic split, which does that grouping and adds people as a row.

The distinction is the reason this endpoint exists. A fetch that answers a question now is traffic in the ordinary sense — somebody wanted to know something and your page was the answer. A fetch that harvests pages for a training run is not: nobody is reading, and nobody will arrive from it. Reporting both as "bot traffic" collapses the one difference that matters.

Crawler traffic is always counted here

bots=excluded is the default on the rest of the API, because a report about people that quietly counts machines is wrong.

On the /ai/* endpoints the machines are the report, so the parameter is ignored. Passing bots=excluded here does not empty the response.

The same numbers, on a screen

These are the figures the AI panel draws. If you want to see them before you write any code, which crawlers walks through the same list in the dashboard.

Authorizations

Path parameters

Query parameters

Response