AI Crawler Log Analyzer
The robots.txt generator tells crawlers what they may do; your access log tells you what they actually did. Upload or paste a server access log to see which AI bots — GPTBot, ClaudeBot, PerplexityBot, CCBot and more — visited, how often, and whether you were blocking them. Everything is parsed in your browser.
Where to get your log
Most hosts expose an access log: Nginx/Apache (access.log), or export from Cloudflare, Vercel, Netlify or your CDN. The common “combined” format works out of the box.
Why it matters
If GPTBot or ClaudeBot never appears, AI systems may not know your content exists. If they appear but get 403, a robots rule, firewall or WAF is quietly making you invisible to AI answers. Decide crawler access on purpose with the AI robots.txt Generator.
Related
Frequently asked questions
▸ ▾ Where do I get my access log?
Most hosts expose an access log (Nginx/Apache access.log), or you can export request logs from Cloudflare, Vercel, Netlify or your CDN. The common 'combined' format works out of the box.
▸ ▾ Is my log uploaded anywhere?
No — the log is parsed entirely in your browser and never sent to a server.