How Google AI Overviews Get Answers Wrong and Why That Matters

AI Overviews are Google's AI-written summaries at the top of many search results. They're often useful, but they can be confidently wrong, and because they sit above the links, many people never check them. Here's how the mistakes happen and how to catch them.

Famous mistakes

Soon after AI Overviews launched in the US in May 2024, screenshots spread of the feature suggesting glue to keep cheese on pizza (taken from an old joke on Reddit) and saying people should eat a small rock a day (from a satirical article). Google said it made changes, including showing fewer Overviews built from joke and forum content. The underlying problem hasn't gone away, because it comes from how the summaries are made.

Why AI Overviews get things wrong

• They can't always tell a joke from a fact. Satire, sarcasm and forum banter read like any other text to a language model.
• They blend sources. An Overview can combine two pages that are each right about different things into one statement that's wrong, for example mixing up two products with similar names.
• They fill gaps. When few good sources exist, the model may still produce a fluent answer, sometimes with invented details. This is often called hallucination.
• They can be out of date. Prices, opening hours, laws and software menus change, and a summary can lag behind the page it came from.
• They drop the caveats. A medical or legal page might say "in most cases" or "check with your doctor". A one-paragraph summary often leaves that out.

Why it matters

For trivia, a wrong answer is harmless. For health, money, legal questions or anything you're about to act on, it isn't. The Overview's position makes it look like Google's official answer, and its smooth writing makes it sound certain. It also means fewer people click through to the sites that did the work, which is a big complaint from publishers.

How to check an answer in 30 seconds

1. Open the sources. Click the links attached to the Overview and find the sentence it's based on. Make sure the page really says that.
2. Check who the source is. An official site, a manufacturer's manual or a professional body counts for more than a forum post.
3. Check the date. For anything that changes, make sure the source is recent.
4. Compare with one more result. If the top regular results disagree with the Overview, trust the pages, not the summary.

If you find a mistake, use the feedback buttons under the Overview. Google uses those reports.

Searching without AI Overviews

Google has no setting to switch them off, but the Web filter under the search box shows plain links without them, and you can make it Chrome's default. See how to search Google without AI answers for the steps.