It’s been a year since Google rolled out AI Overviews, and over two years since the launches of ChatGPT and Perplexity — tools that have collectively upended the traditional relationship between publishers, platforms and audiences.
The spread of AI technology is having an increasingly clear impact on publishers’ traffic and their ability to control content scraping.
Digiday has sifted through the data to find the numbers that help show where publishers stand now when it comes to AI. Here’s what you need to know:
The steady chipping away of traffic
AI bots are driving dramatically less traffic to publishers than traditional Google search — 95.7% less on average, according to a report published in February by Tollbit, an analytics and licensing platform for publishers and AI companies.
While AI search engines refer users back to publishers 0.74% of the time per content scrape — better than chatbots, which refer users to publishers at a rate of just 0.33% — they still generate 91% fewer referrals than the average clickthrough rate from the top 100 organic Google search results, per the Tollbit report.
Naturally, chatbot traffic (from tools like ChatGPT or Perplexity) is relatively small and experimental in scale. While chatbots do matter, publishers are currently more concerned about how Google’s AI Overviews might replace or dilute traditional search traffic, on which they rely heavily.
A Semrush study published this week analyzing over 10 million keywords found that Google is showing more AI Overviews in search results. Over 13% of all search queries triggered AI Overviews in March 2025, up from 6.49% in January 2025. The science, health, and people and society industries experienced the largest share growth of AI Overviews (up about 20%) from January to March, likely due to the “high-trust, information dense” nature of these categories, according to the report.
Meanwhile, software company Ahrefs analyzed 300,000 keywords and found that Google AI Overviews in search results correlated with a 34.5% lower average CTR for the top-ranking page, compared to similar keywords without an AI Overview. (This is how some are calculating AI Overviews’ impact, as there is still no way to separate Google AI Overviews clicks and impressions from the results of publishers’ Google search data.)
Even when AI chatbots and search engines do cite their sources, the links in which they do so don’t always work to drive a user to those pages. A Tow Center for Digital Journalism report found that more than half of responses from Google’s Gemini and Grok 3 cited fabricated or broken URLs that led to error pages.
Generative AI in the Newsroom, a project from Northwestern University, also found that sites in the news publisher category only received 3.2% of ChatGPT’s filtered traffic and 7.4% of Perplexity’s filtered traffic in March 2025. Only 5 out of 143 news sites in the study’s sample received 100 or more unique visitors from ChatGPT and Perplexity from July to November 2024.
AI bot scraping is getting worse
AI crawlers are continuing to scrape publishers’ web content for free — and the situation is getting worse.
Scrapes per website doubled from Q3 to Q4 in 2024, and scrapes per page more than tripled in that time, according to Tollbit’s report. The top scrapers per page in Q4 were from ChatGPT, Meta and Perplexity.
Publishers who don’t want AI bots scraping their sites for content to train tech companies’ large language models can use robots.txt — a few lines of code that tell web crawlers which URLs they can access — to disallow access to their sites.
But that mechanism isn’t working all that well. AI bot scrapes bypassing robots.txt grew by over 40% between Q3 and Q4 2024, according to Tollbit’s report.
AI bot scraping activity increased significantly between Q3 and Q4 2024 for a cohort of websites “with constant bot-blocking strategies,” according to the report. Average scrapes per website for that cohort rose from 3.13 million in Q3 to 5.05 million in Q4, an increase of more than 61%. AI bot traffic as a percentage of total traffic grew by 117% between Q3 and Q4 2024 for those sites. For all sites Tollbit measured in Q4, 3.3% of scrapes bypassed robots.txt.
The largest share of AI bot traffic compared to total AI traffic was coming from OpenAI, TikTok and Meta, per Tollbit.
Lawsuit revelations
Lawsuits by publishers like The New York Times and Ziff Davis shed some light on the level of AI bot scraping, despite publishers’ efforts to block those bots.
Ziff Davis’ lawsuit claims that its domains “accounted for 168,091 documents within the 19 million web pages from the top 1,000 domains in WebText,” which is the training dataset for OpenAI’s large language model GPT-2.
Ziff Davis also claims that OpenAI training datasets contain over one million Ziff Davis articles.
Meanwhile, The New York Times’ lawsuit alleges that NYTimes.com is the fifth top domain in the WebText dataset, with 333,160 entries. Times content accounts for 1.23% of all sources listed in OpenWebText2, with a total of 209,707 unique URLs.
And to add insult to injury, publishers may be facing a new blow: Google can still use publishers’ content to train its search-related AI features — like AI Overviews — even if those publishers have opted out of AI training, a Google executive testified at the platform’s antitrust trial hearing on May 3, according to Bloomberg.
AI deals start to show glimmers of returns
Publishers who have signed deals with AI tech companies are seeing increases in referral traffic from AI search engines and chatbots — but data shows the traffic isn’t all that significant.
For example, referrals from ChatGPT to The Atlantic’s site rose by more than 80% from December to January, according to Digiday reporting. The Atlantic signed a deal with OpenAI in May 2024.
However, the Generative AI in Newsroom report found that OpenAI’s partner sites don’t receive significantly more traffic than non-partners. Partner sites averaged 18 additional visitors from July to November 2024 — and while that was double the number of visitors compared with non-partner sites, the difference wasn’t significant.
Chartbeat data shared with Digiday shows referrals from ChatGPT are growing. Across more than 3,700 news and non-news publishers, page views from ChatGPT increased from 363,000 in August 2024 to 3 million in January 2025. In April, that number jumped to 5.3 million page views. But that comes out to only about 1,400 page views per publisher.
Page views from Perplexity have grown from about 306,000 in August 2024 to 693,000 in April 2025, according to Chartbeat — which comes out to about 187 page views per publisher.
Independent publishers will be the first casualties of AI
A number of independent publishers have reportedly seen their website traffic tank since the expansion of AI bots, search engines and chatbots — especially since the rollout of Google’s AI Overviews.
World History Encyclopedia — the world’s second-most visited history website — saw traffic drop 25% in November after appearing in AI Overviews, CEO Jan van der Crabben told Big Technology.
About 25 independent publishers told Bloomberg that they had seen traffic plummet as a result of AI Overviews, with one website owner reportedly seeing traffic from Google fall more than 70% following AI Overviews’ rollout.
