Google’s AI Overviews feature, which uses generative AI (GenAI) to provide concise answers to queries, confidently interprets made-up idioms. Users have found that simply typing in a random phrase and adding the word “meaning” will give them a confident explanation of what the phrase means, regardless of whether it is real. The system not only interprets nonsense constructs as set expressions, but also indicates their supposed origin, sometimes even providing hyperlinks to enhance the effect of authenticity.
Image source: Shutter Speed / Unsplash
As a result, examples of obvious fictions began to appear on the Internet, which were processed by AI Overviews as genuine phraseological units. For example, the phrase “a loose dog won’t surf” was interpreted as “a humorous way of expressing doubt about the feasibility of an event.” The construction “wired is as wired does” was explained by AI as a statement about how human behavior is determined by its nature, just as the functions of a computer depend on its circuits. Even the phrase “never throw a poodle at a pig” was described as a proverb with biblical origins. All of these explanations sounded plausible and were presented by AI Overviews with complete confidence.
The AI Overview page has a disclaimer at the bottom that it is powered by “experimental” generative AI. Such AI models are probabilistic algorithms that select each subsequent word based on the highest possible predictability, based on training data. This allows for coherent texts, but does not guarantee factual accuracy. This is why the system is able to logically explain what a phrase might mean, even if it has no real meaning. However, this property leads to the creation of plausible, but entirely fictitious interpretations.
As Ziang Xiao, a computer scientist at Johns Hopkins University (JHU), explains, the word predictions in these AI models rely solely on statistics. However, even a logically appropriate word does not guarantee a reliable answer. In addition, generative AI models have been shown to be user-pleasing, adapting responses to perceived expectations. If the system “sees” in the query an indication that a phrase like “you can’t lick a badger twice” should be meaningful, it will interpret it as such. This behavior was observed in a study led by Xiao last year.
Xiao points out that such failures are especially likely in contexts where the training data is sparse, such as rare topics and languages with a limited number of texts. The error can also be amplified by cascading, since the search engine is a complex, multi-layered mechanism. At the same time, AI rarely admits its ignorance, so if the AI encounters a false premise, it is likely to produce a fictitious but plausible-sounding answer.
Google spokesperson Meghann Farnsworth explained that when searching based on absurd or untenable premises, the system tries to find the most relevant content based on the limited data available. This is true for both traditional search and AI Overviews, which can be activated in an attempt to provide useful context. However, AI Overviews does not work for every query. As cognitive scientist Gary Marcus noted, the system produces inconsistent results because GenAI relies on specific examples in its training sets and is not prone to abstract thinking.