How to Optimize for AI Search (and Get Cited by ChatGPT)

How AI search finds your pages

AI search tools don't invent their sources. When someone asks a question, the tool searches an index, reads the pages it finds, and writes an answer that cites some of them. OpenAI says ChatGPT search uses third-party search providers, such as Bing, alongside its own crawler. Google's AI Overviews come from Google's regular search index. So the basics still matter: if a tool can't crawl and index your page, it can't cite it.

1. Let the right AI crawlers in

The big AI companies each run separate crawlers for separate jobs. Some collect data to train models, and others build the index that AI search cites. You can allow one and block the other:

• OpenAI: GPTBot collects training data. OAI-SearchBot builds the index behind ChatGPT search. ChatGPT-User fetches a page when a user asks ChatGPT to open it. (OpenAI's crawler docs)
• Anthropic: ClaudeBot collects training data, Claude-SearchBot indexes for search, and Claude-User fetches pages for users. (Anthropic's crawler docs)
• Perplexity: PerplexityBot builds its search index, and Perplexity-User fetches pages for users. (Perplexity's crawler docs)
• Google: Google-Extended is not a crawler but a robots.txt setting that controls Gemini training. It does not affect AI Overviews, which use the regular Googlebot.

If your robots.txt doesn't mention these bots, they're allowed by default. You only need rules to block something. Here is a common setup that stays in AI search but opts out of model training:

# Let AI search tools find and cite you
User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

# Optional: opt out of model training
# while staying in AI search
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

Check your CDN too. Since July 2025, Cloudflare blocks known AI crawlers by default on new domains and asks new sites to choose (Cloudflare's announcement). If your site is on Cloudflare, look at its AI crawler settings. Your robots.txt can say "allow" while your firewall quietly says "no."

2. Get indexed in Bing, not just Google

Because ChatGPT search leans on third-party providers like Bing, a page Bing hasn't indexed is harder for ChatGPT to find. Add your site to Bing Webmaster Tools, submit your sitemap, and use IndexNow to tell Bing about new pages the moment you publish them. It takes about ten minutes and most sites skip it.

3. Write answers that can be quoted on their own

AI tools lift sentences and short passages. Make yours easy to lift:

• Lead with the answer. Put the direct answer in the first sentence under each heading, then explain.
• One question per section. Use the words people actually ask as headings.
• Be specific. Real numbers, dates, prices, and names, with the source linked. Vague pages give AI nothing to cite.
• Use tables for comparisons. "X vs Y" questions are common, and a clear table is easy to quote.
• Keep it current. Update prices, versions, and dates, and show when the page was last updated.

4. Add structured data

Schema markup, like Organization, Service, Product, Article, and FAQ, tells machines what your page is about without guessing. It won't get you cited by itself, but it removes ambiguity about who you are, what you sell, and where you are.

5. Get mentioned beyond your own site

AI answers lean heavily on what other sites say. When someone asks "best accountant in Pasadena" or "best AI receptionist for dentists," the answer draws on reviews, directories, comparison articles, and community discussions. Make sure you're listed where your buyers look, that your details are consistent everywhere, and that you're part of the comparisons in your field.

6. Track it

Google Analytics 4 now separates visits from AI assistants, so you can see traffic from ChatGPT, Gemini, Perplexity, and others. Once a month, also ask the major AI tools the questions your buyers ask and note who gets cited. That is the closest thing to a ranking report AI search has.

What doesn't matter much

• llms.txt. It's a proposed standard, but Google's John Mueller said in 2025 that no AI system currently uses it. Adding one is harmless; expecting it to get you cited isn't realistic.
• Stuffing pages with "AI" keywords. AI tools look for clear answers, not repeated phrases.
• Anyone who guarantees citations. AI answers vary by question and change over time. Nobody controls them.

What this looked like on my own site

This site allows AI search crawlers, submits every new page to Bing through IndexNow, and writes most pages answer-first with sources. Over three months, monthly visits from AI assistants went from 94 to 273, and visits from ChatGPT alone went from 23 to 174. The numbers are still small, but the trend is clear, and it came from the basics above, not from tricks.