ChatGPT-User vs OAI-SearchBot: user agents and robots.txt
Both agents come from OpenAI, and both can end with your page cited in ChatGPT, but they work differently. OAI-SearchBot crawls on its own schedule and follows robots.txt. ChatGPT-User fetches a page because a person's request needs it, and robots.txt may not apply. Mixing them up leads to rules that block the wrong thing.
The short answer
- Manage ChatGPT search with OAI-SearchBot. It crawls automatically, follows robots.txt and decides whether your pages can appear in ChatGPT search answers.
- Treat ChatGPT-User as a person's request routed through ChatGPT. OpenAI says it is not used for automatic crawling and not used to decide search eligibility, and that robots.txt rules may not apply to it.
- Verify both by IP address, using OpenAI's published lists, rather than trusting the user-agent string.
Side by side
| Question | OAI-SearchBot | ChatGPT-User |
|---|---|---|
| What triggers a visit | Automatic crawling | A user's request in ChatGPT or a custom GPT, including GPT Actions |
| What it is for | Surfacing websites in ChatGPT search | Visiting a page to answer or act on that request |
| robots.txt | Followed; changes take about 24 hours to apply | May not apply, because a user initiated the visit |
| Effect on search eligibility | Decides it | None, according to OpenAI |
| Published IP list | https://openai.com/searchbot.json | https://openai.com/chatgpt-user.json |
OpenAI also says that sites opted out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links.
The exact user-agent strings
OpenAI's crawler documentation gives these strings, checked on 2026-10-21:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/botWhen OAI-SearchBot fetches your robots.txt file, OpenAI may add a robots.txt; marker before the URL, for example compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot, so you can tell robots.txt requests from page requests. Version numbers change from time to time, so match on the tokens OAI-SearchBot and ChatGPT-User rather than the full string.
Find them in your logs
In a standard combined access log, the request path is the seventh space-separated field and the status code is the ninth. These commands count visits, list the pages ChatGPT-User fetched and show the status codes OAI-SearchBot received:
grep -c "OAI-SearchBot" access.log
grep "ChatGPT-User" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20
grep "OAI-SearchBot" access.log | awk '{print $9}' | sort | uniq -cHow to read them:
- Steady OAI-SearchBot requests mean ChatGPT search is crawling the site. Gaps after a firewall or CDN change are worth investigating.
- A ChatGPT-User request means a conversation needed that page at that moment, for example because someone pasted the link or asked a question that led ChatGPT to it. Pages with many such requests are pages ChatGPT relies on; keep them current.
- 403 or 429 responses to either agent point to a firewall rule, bot protection or rate limit, not to robots.txt.
Verify the IP address
A user-agent string is easy to copy. Before you allowlist traffic, or decide that OpenAI is crawling heavily, compare the source IP address with OpenAI's published JSON lists. Fetch the lists regularly instead of copying addresses once. OpenAI's guidance for advertisers adds a broader point that applies here too: avoid relying only on IP addresses observed in your logs, and combine user-agent identification, published IP ranges and provider-level bot verification where your CDN supports it.
A third kind of ChatGPT traffic: signed agent requests
OpenAI's help center describes a cloud browser in ChatGPT Work that signs its requests with Web Bot Auth, using the HTTP Message Signatures standard (RFC 9421). Each request carries Signature, Signature-Input and Signature-Agent headers, with Signature-Agent set to "https://chatgpt.com". The public keys are published at https://chatgpt.com/.well-known/http-message-signatures-directory.
- Cloudflare recognises this traffic as a signed agent, with the bot tag
chatgpt-agentand detection ID129220581. - Akamai and HUMAN list it under the existing bot name ChatGPT Agent.
- Vercel recognises and verifies it automatically.
- Other CDNs can verify the signature directly against the published keys before allowing the request.
OpenAI notes that at launch this cloud browser cannot sign in to websites or complete payments. Verify it by its signature, not by its user-agent string.
Rules that match what you want
To be eligible for ChatGPT search while keeping content out of model training:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /There is no ChatGPT-User line, because OpenAI says robots.txt may not apply to it. Then make sure your firewall or CDN lets OAI-SearchBot through from its published IP ranges; OpenAI's help page lists that as a condition for appearing in ChatGPT search results.
- To opt out of ChatGPT search, disallow
OAI-SearchBot. Your pages can still appear as navigational links. - To stop ChatGPT-User, you would need a firewall rule. That stops ChatGPT reading your page for the person who asked, for instance when they paste your URL, and it does not change search eligibility.
- Training is a separate decision. See GPTBot vs OAI-SearchBot for the training crawler, the OAI-AdsBot guide for the crawler that reviews ad landing pages, and the AI crawler list for other companies' agents.
To check what ChatGPT does with your pages once they are reachable, see how to track brand mentions in ChatGPT.
Sources
- OpenAI: Overview of OpenAI crawlers
- OpenAI IP list for OAI-SearchBot
- OpenAI IP list for ChatGPT-User
- OpenAI Help: ChatGPT Work's Cloud browser allowlisting
- OpenAI Help: Searching the web with ChatGPT
- OpenAI Help: Advertiser guidance for allowing OpenAI web crawlers
- IETF: RFC 9421, HTTP Message Signatures
Reviewed on 2026-10-21. OpenAI changes user-agent versions and help articles from time to time; the robots.txt tokens have stayed the same.
Frequently asked questions
What is ChatGPT-User?
The user agent ChatGPT uses when a person's request in ChatGPT or a custom GPT needs a web page, for example to read a link or answer a question. OpenAI says it is not used to crawl the web automatically.
Does ChatGPT-User respect robots.txt?
OpenAI says robots.txt rules may not apply, because the visit is initiated by a user. To manage whether your site appears in ChatGPT search, use OAI-SearchBot in robots.txt.
What is the difference between OAI-SearchBot and ChatGPT-User?
OAI-SearchBot crawls automatically to surface websites in ChatGPT search and follows robots.txt. ChatGPT-User visits a page only when a user's request needs it, and OpenAI says it is not used to decide whether content may appear in search.
How do I verify that a request really came from OpenAI?
Compare the source IP address with OpenAI's published lists: openai.com/searchbot.json for OAI-SearchBot and openai.com/chatgpt-user.json for ChatGPT-User. A user-agent string alone is easy to fake. Requests from ChatGPT's signed cloud browser can be verified by their HTTP message signatures.
Should I block ChatGPT-User?
Usually not. Blocking it does not take you out of ChatGPT search, which depends on OAI-SearchBot. It only stops ChatGPT from reading your page for the person who asked about it.