Free live tool · reviewed July 2026

Is OAI-SearchBot blocked on your website?

Test the exact public robots.txt rule that applies to OpenAI’s search crawler. The report keeps ChatGPT search discovery separate from GPTBot training and user-requested access.

OpenAI search discovery · live robots.txt

Check OAI-SearchBot access.

Enter a public homepage. The focused result appears first, followed by the other current crawler-policy tokens for context.

This tests robots.txt policy for the homepage. Firewalls, CDNs, and bot challenges are separate.

Copy-ready starting point

Allow discovery. Restrict training.

This conservative example separates automated search and user retrieval from training controls. Merge it with existing rules; do not overwrite the full file.

robots.txt starterReview against your publishing policy
# Search and answer discovery
User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

# User-requested retrieval
User-agent: Claude-User
Allow: /

# Model training and Gemini grounding control
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

OAI-SearchBot and GPTBot do different jobs.

OAI-SearchBotOpenAI documents this token for search features. Allowing it makes public content eligible to appear in ChatGPT search answers, but does not guarantee crawling, inclusion, ranking, citation, or traffic.
GPTBotThis is the separate training and model-improvement crawler. You can block GPTBot while allowing OAI-SearchBot.
ChatGPT-UserOpenAI uses this token for user-initiated visits. OpenAI states that robots.txt may not apply in the same way because the request is made by a user.

A balanced OpenAI robots.txt policy

If your goal is search discovery without training collection, use separate groups. Merge these rules with your existing file and test the exact paths that matter.

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

Robots.txt is a crawl preference, not access control. Protect private content with authentication. If Cloudflare sits in front of the site, compare the robots rule with a synthetic OAI-SearchBot response before treating an allowed rule as reachable.

What this free check proves

  • It fetches the public robots.txt response for the submitted homepage.
  • It applies the longest matching rule for the exact OAI-SearchBot token.
  • It shows the matched directive and keeps the other OpenAI roles visible.
  • It does not claim that OpenAI will crawl, index, cite, rank, or recommend the page.

Primary source

OpenAI’s publisher and developer guidance distinguishes OAI-SearchBot search discovery from GPTBot training controls. The page was reviewed in July 2026; crawler behavior and documentation can change.