A new report measuring bot activity on the Web, as well as related data shared with WIRED by Internet infrastructure company Akamai, shows that AI bots already account for a meaningful portion of Web traffic. The findings also shed light on an increasingly sophisticated arms race, as bots adopt clever tactics to bypass website security to keep them out.
“The majority of the Internet in the future is going to be bot traffic,” says Toshit Pangrahi, co-founder and CEO of Tollbit, a company that tracks web-scraping activity and publishes the new report. “It’s not just a copyright problem, a new entrant is emerging on the Internet.”
Most large websites try to limit what content bots can scrape and feed into AI systems for training purposes. (WIRED’s parent company, Condé Nast, as well as other publishers, are currently suing several AI companies over alleged copyright infringement related to AI training.)
But another type of AI-related website scraping is also now on the rise. Many chatbots and other AI tools can now get real-time information from the web and use it to enhance and improve their outputs. This might include the latest product prices, movie theater schedules or a summary of the latest news.
According to Akamai data, training-related bot traffic has been growing steadily since last July. Meanwhile, the global activity of bots bringing web content to AI agents is also increasing.
“AI is changing the web as we know it,” Akamai chief technology officer Robert Blumoffe told WIRED, “The upcoming arms race will determine the future look, feel, and functionality of the web, as well as the basics of doing business.”
In the fourth quarter of 2025, Tolbit estimates that on average one in every 50 visits to its clients’ websites was from an AI scraping bot. In the first three months of 2025, the figure was just one in every 200. The company says that in the fourth quarter, more than 13 percent of bot requests were bypassing robots.txt, a file that some websites use to indicate which pages bots should avoid. Tolbit says the share of AI bots that ignore robots.txt increased 400 percent from the second quarter to the fourth quarter of last year.
Tollbit has reported a 336 percent increase in the number of websites attempting to block AI bots over the past year. Pangrahi says scraping techniques are becoming more sophisticated as sites try to control how bots access their content. Some bots disguise themselves by making their traffic appear as if it is coming from a normal web browser or send requests designed to mimic how humans typically interact with websites. Tolbit’s study states that the behavior of some AI agents is now almost indistinguishable from human web traffic.
Tollbit markets tools that website owners can use to charge AI scrapers for accessing their content. Other companies, including Cloudflare, also offer similar tools. “Anyone who depends on human web traffic – starting with publishers, but basically everyone – is going to be affected,” says Pangrahi. “There needs to be a faster way for machine-to-machine, programmatic exchange of value.”
<a href