Check which AI crawlers can access your site and detect llms.txt.
Enter a URL to check which AI crawlers can access your site.
Check which AI crawlers can access your site and detect llms.txt.
Check which AI crawlers can access your website based on the rules in your robots.txt file. Covers 16 major AI crawlers including GPTBot, ClaudeBot, Google-Extended, PerplexityBot, Applebot-Extended, Bytespider, and more. Shows whether each bot is explicitly allowed, blocked, or falling back to your wildcard User-agent rules. Also detects whether you have an llms.txt file — an emerging standard for AI-readable site content.
Yes, completely free with no account required. All tools on SEOKit are available without registration or usage limits.
Yes, you can analyze any publicly accessible URL. Some tools also work with lists of URLs for bulk analysis.