AI SEO crawler check
Checks whether a website allows AI bots (e.g., GPTBot, Google-Extended, PerplexityBot) to crawl and use its content. It analyzes the website's robots.txt file and returns the access status for all major AI bots along with suggestions to enable AI crawling if blocked.
GET /seo/aiseochecker
Parameters
website
Required
query
string
enter the website url you want to check in
X-API-KEY
Required
header
string
Your vebapi api key
Example Request
curl -X GET "https://vebapi.com/api/seo/aiseochecker?website=codeconia.com" \
-H "X-API-KEY: YOUR_API_KEY" \
-H "Content-Type: application/json"
Response
{
"url": "codeconia.com",
"robots_found": true,
"ai_access": {
"GPTBot": true,
"ChatGPT-User": true,
"Google-Extended": true,
"AnthropicBot": true,
"ClaudeBot": true,
"PerplexityBot": true,
"CCBot": true,
"Amazonbot": true,
"Bytespider": true,
"facebookexternalhit": true,
"cohere-ai": true,
"YouBot": true,
"NeevaBot": true,
"ai-crawler": true,
"Applebot": true,
"Baiduspider": true,
"Sogou": true,
"YandexBot": true,
"PhindBot": true,
"DuckDuckBot": true,
"Yeti": true,
"360Spider": true,
"ias-va": true
},
"ai_bots_allowed": true,
"suggestions": [
"Your site is currently open for AI bots. You're AI-friendly!",
"You can still improve by providing structured data (schema.org) for better AI comprehension.",
"Add clear content usage terms if you want to allow or limit AI training use."
]
}