Back to Lastfind

AI bots: which AIs read your site.

The AI crawlers that visit your site, the pages they read, the errors they hit and the visits AI answers send back, from your Cloudflare.

AI bots reads your site’s own Cloudflare analytics: no tag, no script and nothing installed on the site. It is available on every Lastfind plan. Your site must be served through Cloudflare (its DNS proxied, the orange cloud).

Connect Cloudflare#

  1. In Cloudflare, open My Profile > API Tokens, choose Create Token and then Create Custom Token. Name it, for example, "Lastfind, read only".
  2. Under Permissions, add these three, all Read: Account > Account Analytics > Read; Zone > Analytics > Read; Zone > Zone > Read.
  3. Under Account Resources, include your account. Under Zone Resources, include the specific zone of your site.
  4. Create the token and copy it. In Lastfind, open AI bots, choose Connect Cloudflare, paste the token and choose your site’s zone.

The token only reads analytics: it cannot change anything in your Cloudflare account. Lastfind checks it with Cloudflare before saving it, stores it encrypted and never shows it again. Workspace admins replace it or disconnect under Manage connection; disconnecting deletes the token at once and keeps the days already collected. Revoking the token in Cloudflare also stops the collection, and the screen asks to reconnect.

What the screen shows#

On connecting, Lastfind reads the last 30 days, then one more day every day, so the history grows past the 31 days Cloudflare keeps. The header’s period applies; the AI, topic, tag and brand filters do not. The screen shows the AI bot visits and their share of all visits, the pages they read, the distinct bots and their companies, the error rate (404, 403 and 5xx), the visits by purpose (training, search, user request) per day or week, every bot with its visits, pages, errors and last visit, the most read pages and whether a monitored answer cited them, and what your site answered the bots.

On paid Cloudflare plans (Pro and above), it also shows the visits that arrive from links in AI answers (ChatGPT, Perplexity, Claude, Gemini, Copilot and others) and, for each company, how many times its bots read your site for each visit its answers sent. On the Free plan Cloudflare does not report where visits came from, so those parts say so instead of showing zero.

Limits#

Bots are recognized by the name they declare (their user agent), which anyone can fake. The figures are estimates from Cloudflare’s sampled analytics. A day keeps up to 10,000 combinations of bot, page and status, the most visited first. Today’s figures are partial until the next day’s collection.