We all know AI tools can do a lot these days. But how good are they at searching the internet?
It’s a big question, and it matters. Because according to Future’s own research (Future publishes TechRadar), almost a third of US respondents said they now use AI instead of traditional search engines, like Google.
Some turn to ChatGPT and other chatbots. Others use AI tools built specifically for searching and researching, like Perplexity. And even if we’re not actively choosing AI tools, AI is showing up in our searches anyway, like those little Google summaries that now pop up at the top of your results.
In short, AI-powered search is everywhere. But here’s the real question: is it actually any good? That’s what researchers are starting to ask, and let’s just say, the early reviews aren’t glowing.
Is ChatGPT the new Google? Not so fast…
People might be using AI tools for search. But that doesn’t mean they should.
In our recent testing, we pitted four of the top AI chatbots against each other to see how well they handled search: OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, and Perplexity AI.
The results were… mixed. Not a total disaster – they could all retrieve some relevant info – but accuracy wasn’t great. And the way they summarized that information? Often confusing or just not that helpful.
More extensive testing from the Tow Center for Digital Journalism, as reported in the Columbia Journalism Review, backs this up. Their team tested eight major AI models – including ChatGPT, Perplexity, Copilot, Grok, and Gemini – and found consistent issues: “confident presentations of incorrect information, misleading attributions to syndicated content, and inconsistent information retrieval practices.” Ouch.
In total, AI models gave an incorrect answer to more than 60% of queries. Perplexity was the most accurate (which tracks, it’s marketed as a research tool), but it still got 37% of answers wrong. Grok fared the worst, with a frankly shocking 94% failure rate. No comment.
The Tow Center marked results on a scale from completely correct to completely incorrect (or no answer at all). Some of the worst examples were clear hallucinations, where AI just… makes stuff up. But even when answers weren’t entirely wrong, there were still big issues with how the tools repackaged news and search results.
Search without sources isn’t search at all
This repackaging gets to the heart of the problem. Even if AI doesn’t serve up false info, it still summarizes and reshapes content in ways that are unhelpful – and often misleading. As the Tow Center puts it:
“While traditional search engines typically operate as an intermediary, guiding users to news websites and other quality content, generative search tools parse and repackage information themselves, cutting off traffic flow to original sources.”
“These chatbots’ conversational outputs often obfuscate serious underlying issues with information quality. There is an urgent need to evaluate how these systems access, present, and cite news content.”
One of the clearest issues is how poorly AI tools cite their sources. ChatGPT, for example, often links to the wrong article, sends you to a site’s homepage, or skips citations altogether.
That’s a problem for two key reasons. Firstly, publishers lose out on traffic, even though their content is being used.
Secondly, fact-checking becomes a pain. One of the only ways to verify AI results is to go back to the original source – and that’s a lot harder if it’s not provided.
So if AI gives you information but no clear source, is it really worth using for search – especially if you end up fact-checking everything via Google anyway? Probably not.
Yes, AI tools are improving all the time. And yes, some, like Perplexity, perform better than others. But even the best still need human oversight. For now, when it comes to search, the bots just aren’t ready to fly solo.