GPTBot vs OAI-SearchBot vs ChatGPT-User: which one decides if ChatGPT cites you
By Alex Bouchard · 6 September 2026
Most robots.txt templates written since 2023 have one line for OpenAI: block GPTBot. OpenAI now documents three user agents with three different jobs, and the one that decides whether ChatGPT can cite you is not GPTBot. Here is what each bot does, straight from OpenAI's documentation, and the robots.txt that matches the policy most companies actually want.
The three bots
| Token | Job | Training? | robots.txt |
|---|---|---|---|
GPTBot | Crawls content that may be used to train OpenAI's foundation models. | Yes | Honoured. Disallow = do not train on this site. |
OAI-SearchBot | Surfaces websites in ChatGPT's search features. | No | Honoured. Disallow = absent from ChatGPT search answers. |
ChatGPT-User | Fetches a page when a person asks ChatGPT about it, or a GPT Action calls it. | Not stated | User-initiated; OpenAI says robots.txt rules may not apply. |
OpenAI also documents OAI-AdsBot, which only visits pages submitted as ad landing pages and is not used for training. It is not a crawler of the open web.
Why sites disappear from ChatGPT without noticing
The failure is a template, not a decision. A "block all AI" rule set — from a plugin, a CDN toggle, or a list copied in 2023 — disallows GPTBot and OAI-SearchBot together. The site keeps ranking in Google, traffic looks normal, and nobody checks ChatGPT. The training opt-out was the intent; the search opt-out came along for the ride. In our scans this pattern shows up regularly: sites that explicitly disallow the retrieval bots while their marketing pages are otherwise perfectly readable.
The reverse mistake is rarer but real: allowing everything and assuming that settles it. A robots.txt that says Allow: / is only consulted if the crawler gets through your edge in the first place; a bot-management rule that 403s OAI-SearchBot makes the robots.txt line irrelevant.
The robots.txt for "cite me, don't train on me"
User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Disallow: /
That is the whole change for OpenAI. ChatGPT-User needs no line: OpenAI says it may not honour one, and it only fetches when a person asks. If you want the same split for Anthropic and Perplexity, the tokens are in the AI crawler list.
The scan reads your robots.txt for every OpenAI, Anthropic and Perplexity token and fetches your homepage as each of them, so you can see whether ChatGPT is allowed in and whether there is anything to read once it is.
Scan free →Sources
Questions people ask
If I block GPTBot, will ChatGPT stop citing my site?
Not according to OpenAI. GPTBot is the training crawler; OAI-SearchBot is the crawler behind ChatGPT search, and OpenAI says sites that disallow OAI-SearchBot will not appear in ChatGPT search answers. You can disallow GPTBot and allow OAI-SearchBot.
What is ChatGPT-User?
The user agent ChatGPT uses when a person asks about a specific page or a GPT Action fetches one. OpenAI says that because these fetches are initiated by a user, robots.txt rules may not apply to them.
What user agent should I allow to appear in ChatGPT search?
OAI-SearchBot. OpenAI's documentation recommends allowing OAI-SearchBot in robots.txt to help ensure your site appears in search results.
Does OAI-SearchBot use my content for training?
OpenAI states OAI-SearchBot is not used to train generative AI models. GPTBot is the crawler whose collected content may be used for training.
Sixty seconds, no signup: fetch your homepage the way GPTBot, ClaudeBot and PerplexityBot fetch it, count the words that survive, and read what your robots.txt actually tells each AI bot.
Scan free →Corrections: every claim on this page is attributable to vendor documentation or to our own published measurement. If one is wrong, tell us and it gets fixed with a dated note.