Back to all gems
Gem
69-bot allowlist
Liverobots.txt explicit Allow for Tier-1 + Tier-2 + Tier-3 AI crawlers. GPTBot · OAI-SearchBot · ClaudeBot · Claude-SearchBot · PerplexityBot · Googlebot · Bingbot · etc. (Google-Extended and Applebot-Extended are content-use policy tokens, listed as policy · not traffic.)
Status on your site
- Deployed and verified live
- Auto-monitored · alerts on any regression
- Last verified · 2 hours ago
Receipts
Validator output, schema dumps, screenshots, and deploy logs all sit in your vault. Download anytime.