Free tool · updated July 2026

Allow ChatGPT search without automatically allowing model training.

OpenAI documents separate crawlers for search, model training, and ad validation. Build an explicit robots.txt policy for each use instead of copying a blanket AI-bot block.

Interactive generator

Choose each use independently.

The default policy allows ChatGPT search discovery while blocking model-training and advertising crawlers. Change it to match your actual publishing policy.

01 ChatGPT search

OAI-SearchBot helps surface public pages in ChatGPT search features.

02 Model training

GPTBot controls content crawling that may be used to improve OpenAI's foundation models.

03 Ad validation

OAI-AdsBot validates landing pages submitted as ads. Most sites do not need it.

Generated robots.txt blockReview before publishing
# OpenAI search visibility

User-agent: OAI-SearchBot
Allow: /

# OpenAI model training

User-agent: GPTBot
Disallow: /

# OpenAI ad landing-page validation

User-agent: OAI-AdsBot
Disallow: /
Check the public response

What does your live site allow now?

Fetch the public /robots.txt file and resolve the homepage rule for each OpenAI crawler. This checks crawler instructions, not firewall access.

No account. We fetch the public robots.txt file only.

What each rule controls

OAI-SearchBotAutomatic crawling used to surface websites in ChatGPT search results. Blocking it opts pages out of those search answers, though navigational links may still appear.
GPTBotCrawling for content that may be used to improve OpenAI's generative foundation models. This setting is independent from search.
OAI-AdsBotSafety and relevance validation for landing pages submitted as ChatGPT ads. It is not a general search crawler. See the advertiser access checklist.
ChatGPT-UserA user-triggered fetch, not an automatic crawler. OpenAI notes that robots.txt rules may not apply to these requests.

Crawler purposes and current user-agent guidance: OpenAI's official crawler documentation.

Installation checklist

  1. 1

    Copy the generated block and merge it into the existing file at https://yourdomain.com/robots.txt. Do not erase unrelated rules.

  2. 2

    Open the public file and confirm the exact rules you intended are present in the server response.

  3. 3

    Check your CDN, firewall, and bot-management settings. A permissive robots file cannot override a network-level block.

  4. 4

    Keep private or paid material behind authentication. The Robots Exclusion Protocol is a crawler preference, not access control.

  5. 5

    Recheck after CMS, security-plugin, or CDN changes; those systems can generate or replace crawler rules.

Protocol semantics and security limits: IETF Robots Exclusion Protocol (RFC 9309).

What this cannot guarantee

Allowing a crawler makes access possible; it does not guarantee crawling, indexing, citation, recommendation, ranking, or traffic. Blocking a named crawler also does not protect content from every bot or user-directed request. Test the public response and your infrastructure policy together.

Need other providers too? Use the multi-provider AI crawler checker for OpenAI, Anthropic, Perplexity, and Google policy tokens.