Read this before you quote a number from the AI panel to anyone.
There are two separate reliability questions, and they have different answers. The first is about coverage — how much crawler traffic reaches us at all. The second is about identity — whether a crawler that did reach us is who it says it is.
1. Coverage: the panel undercounts today
Most AI crawlers request raw HTML and never run JavaScript. A browser tracking script cannot see them.
That is not a design choice or a bug. It is what a JavaScript tracker is. GPTBot fetches your page's HTML, reads it, and leaves. Our script — which only runs when a browser executes it — is never executed, so no event is ever sent.
What this means in practice
| Seen | Not seen | |
| Crawlers that execute JavaScript | ✅ | |
| Anything reaching our collector with a crawler user agent | ✅ | |
| GPTBot, ClaudeBot, Bytespider and most training crawlers fetching raw HTML | ❌ |
So the panel's number is a floor, not a total. Every crawler view it shows is real. There are more that it does not show, and we cannot tell you how many.
How much is missing depends on your site
This is the part that determines whether the current number is useful to you.
If your pages are server-rendered and your server also serves our script, the gap is smaller — a crawler fetching a rendered page still makes a request that can be recorded.
If your site is a static marketing site or a client-rendered app, the gap is larger. A crawler fetching your static HTML from a CDN produces nothing that reaches us at all.
We are not going to put a percentage on this, because we would be making it up.
What is being built
A server-side reporting path: a small hook you can add on your own server that reports crawler requests directly, including the ones that skip JavaScript. It is the highest-priority addition to this panel.
Two things about it worth stating now:
It will be optional. The zero-install browser path stays the default. The server-side path is for people who want complete coverage, not a prerequisite for using the panel at all.
When it ships, your numbers will jump. That will be the missing traffic becoming visible, not a change in your crawler traffic. Do not read the step change as growth.
Until then, the honest way to read this panel is relatively rather than absolutely: which operators are reading you, which of your pages they prefer, and whether that is trending up or down. Those comparisons hold even under a constant undercount. The absolute count does not.
2. Identity: a crawler is identified by what it claims
A crawler is classified from its user-agent string — a header the client chooses. Anything can send any user agent. A scraper can call itself GPTBot and we will initially record it as GPTBot.
That is true of every product that reports crawler traffic, including ours. Here is what we do about it.
The verification tiers
Every crawler view carries one of these:
| Tier | Meaning |
| IP verified | The request's source address falls inside the operator's own published address ranges. |
| Unverified | User agent only. It says it is GPTBot; nothing confirms it. |
| IP mismatch | It claimed a crawler user agent from an address outside the operator's published ranges. A strong spoofing signal. |
IP mismatch is surfaced, never hidden. A crawler impersonating OpenAI on your site is something you should be able to see.
Which operators can be verified at all
| Can be verified | Cannot be verified, ever |
| OpenAI, Anthropic, Google, Microsoft Bing, Apple, Common Crawl | Perplexity, Meta, ByteDance |
The right-hand column is not a limitation of ours. Meta and ByteDance publish no address ranges, and Perplexity's list has been stale since early 2025 with its traffic resolving to generic cloud addresses.
For those three, "unverified" is the honest ceiling. A Perplexity hit can be reported and can never be confirmed — and equally, the absence of a confirmed Perplexity hit is not evidence that Perplexity did not visit.
Right now, IP verification is not running
Today, the "Verified crawlers" tile reads 0% on every site, because the address-range checking is not yet switched on. Every crawler view is recorded as unverified.
That is the honest state rather than a fake confidence tier: claiming "IP verified" against an address list we have not yet loaded would be a fabricated number on the one panel this product is sold on.
When verification goes live, that tile starts reflecting reality and your historical rows stay as they are — they cannot be verified after the fact, because the address they came from was never stored.
What this panel is good for today
Which operators are reading you, relatively.
Which of your pages they prefer — see AI traffic by page.
The assistant-versus-training split, which is a different and more useful question than "how many bots".
Trends over time, which survive a constant undercount.
What it is not good for today
An absolute count of AI crawler visits. It is a floor.
Proving a crawler did not visit. Absence here is not evidence of absence, especially for Perplexity, Meta and ByteDance.
Confirming a crawler was genuine. Not until IP verification is live.
If any of that changes, this page changes with it. A claim on this page that stopped matching the product would be a fault, not a documentation backlog item.