Google’s efforts to serve up AI-generated answers in search results hasn’t exactly gone according to plan. When AI Overviews rolled out last summer, the feature surprised users by crafting embarrassing responses, telling them to glue cheese onto pizza, eat rocks and boogers, and set their birthday as a password. Though Google made fixes to address some of the most absurd answers, AI Overview still occasionally presents inaccurate information. But rather than retreat from AI search results, Google is doubling down.
This week, the company announced it’s testing a new “AI Mode” in search that replaces the typical web links that follow an Overview with a more comprehensive AI-generated summary. Google says these more thorough responses, powered by its newer Gemini 2.0 model, should be better equipped to answer more complex, multifaceted questions as well as queries related to coding and advanced math. It’s the latest example of generative AI makers leaning into the technology as a tool to search the web despite concerns from researchers who worry AI hallucinations may make these results unreliable.
“You can ask nuanced questions that might have previously taken multiple searches—like exploring a new concept or comparing detailed options—and get a helpful AI-powered response with links to learn more,” Google Search vice president of product Robby Stein wrote in a blog post.
Stein went on to say that this new feature utilizes a “query fan-out” method, which runs several related searches simultaneously, pulling from multiple sources before synthesizing them into a single answer. Users can also ask follow-up questions to their initial queries. While users can still click through to websites for more context, the interface in this mode no longer displays the running list of links. Google says it will provide web links when it does not have “high confidence” that an AI-generated answer will be helpful. The company has already acknowledged that the tool may make mistakes.
“As with any early-stage AI product, we won’t always get it right,” Stein said. “For example, while we aim for AI responses in Search to present information objectively based on what’s available on the web, it’s possible that some responses may unintentionally appear to take on a persona or reflect a particular opinion.”
For now, AI Mode is only available through Google’s Search Labs. Users with access can select AI Mode from the list of tabs where they would typically find other features like Images or News. Google is also expanding the overall scope of AI Overviews. Moving forward, Overviews will be powered by Gemini 2.0 and will appear even more frequently. Additionally, AI Overviews will now be available to teen users and those who are not signed into a Google account.
“With Gemini 2.0’s advanced capabilities, we provide faster and higher quality responses and show AI Overviews more often for these types of queries,” Stein added.
AI models keep hallucinating false facts
The promise of AI search sounds appealing on the surface. By simply using conversational speech, a wandering internet searcher can quickly receive answers in an easy to understand format. AI search also means, in theory at least, that users can ask more open-ended questions than they could with typical search engines that work best with springs of keywords. But even the newest, most advanced models offered by AI companies continue to hallucinate and fabricate facts. Removing drop down links to web pages where the AI-generated information is pulled from risks making it more difficult for users to verify any given claim.
Google isn’t the only one dealing with less-than-perfect AI responses. In January, Apple was forced to suspend a feature that provided AI summaries of news stories after it generated multiple false claims. Some Apple users received push notifications falsely claiming Luigi Mangione, the man accused of killing UnitedHealthcare CEO Brian Thompson, had shot himself. Others received an alert erroneously saying Israel’s prime minister Benjamin Netanyahu had been arrested. After suspending the feature Apple told The Guardian it was “working on improvements” which would be made available in a future software update.
Related: [How to avoid AI in your Google searches]
In its blog post, Google said it was experimenting with the new AI mode in part due to requests from “power users” to add AI responses to even more searches. AI Overviews, the post claimed, are amongst Google’s most popular search features. Google did not respond to Popular Science’s request for comment seeking more data or details that could illustrate that point. For now, at least, it is still possible to search Google without AI summaries. To do that users can click on the “more” tab below search results and select the “Web” option.
