In the recently released video game Trash Goblin, players take on the role of a little goblin who opens sacks of trash, retrieves debris-encrusted mirrors, blades, and other baubles, then chisels and scrubs them clean. They then sell them to visitors of their shop.
It’s important to note—because it is true—that, during Trash Goblin’s chiseling phase, as the player click-click-clicks their mouse to reveal the trinkets encased within the game’s messy junk, you cannot accidentally damage the trinkets.
Last week, Google’s “AI Overview,” the AI snippet that now commonly appears atop the search results for people who Google for information about anything, incorrectly said you could break them.
Google’s AI-powered search results didn’t just make one mistake about Trash Goblin, a well-reviewed game that’s sold 10s of thousands of copies for UK-based development studio Spilt Milk.
It made reams of errors, delivering faulty advice to players who Googled for game tips.
In some cases, Google’s search-topping AI Overview was blatantly wrong.
In others, it was right initially, then displayed bad information to support it.
In some cases, Google’s AI confused Trash Goblin with another game, and started giving advice tied to that game instead.
The result has been confusion for players and headaches for Trash Goblin’s developers, who are unsure how to solve the problem of the world’s most popular search engine telling people the wrong things about their game.
“It talks to you in a way that it sounds like it understands your question, whereas that’s the big lie about AI, right?” Andrew Smith, Trash Goblin’s creative director told me over a call last week
“It doesn’t understand your question at all. Not in a human way, anyway.”
Google says its AI Overview feature is used by 1.5 billion users around the world. It uses the company’s Gemini AI in conjunction with search results, and it produces, Google says, results on par with other search engine features.
“The vast majority of AI Overviews are factual, and we’ve continued to make improvements to both the helpfulness and quality of responses,” a company spokesperson told Game File yesterday. “When issues arise—like if our features misinterpret web content or miss some context—we use those examples to improve our systems.”
Nevertheless, at the bottom of each of Google’s AI Overview answers the fine print states: “AI responses may include mistakes.”
As I spoke with Smith and two of his colleagues last week, we tested Google AI’s ability to give accurate advice about Trash Goblin. And, even when we asked it the same question repeatedly, as you’ll see below, we kept getting different answers.