While talk of an AI bubble is prevalent in the air these days, fears of overinvestment that could emerge at any time are creating something of a paradox on the ground: Companies like Google and OpenAI can barely build the infrastructure fast enough to meet their AI needs.
During an all-hands meeting earlier this month, Google’s AI infrastructure chief Amin Vahdat told employees that the company needs to double its service capacity every six months to meet demand for artificial intelligence services, CNBC reported. Vahdat, vice president of Google Cloud, presented slides showing how the company needs to scale “1000x in the next 4-5 years.”
While a thousandfold increase in compute capacity sounds ambitious in itself, Vahdat noted some key hurdles: Google needs to be able to deliver this increase in capacity, compute and storage networking “at essentially the same cost and faster, for the same power, same energy levels,” he told employees during the meeting. “It won’t be easy but through collaboration and co-design, we will get there.”
It’s unclear whether this “demand” cited by Google reflects natural user interest in AI capabilities while the company integrates AI features into existing services like Search, Gmail, and Workspaces. But whether users are using the features willingly or not, Google isn’t the only tech company struggling to keep up with its growing user base of customers using AI services.
Major technology companies are racing to build data centers. Google competitor OpenAI is planning to build six huge data centers across the US through its Stargate partnership project with SoftBank and Oracle, which will commit more than $400 billion over the next three years to reach about 7 gigawatts of capacity. The company faces similar hurdles in serving its 800 million weekly ChatGPT users, with even paid customers regularly exceeding usage limits for features like video synthesis and simulated reasoning models.
“Competition in AI infrastructure is the most important and most expensive part of the AI race,” Vahdat said at the meeting, according to CNBC viewing of the presentation. The infrastructure executive explained that Google’s challenge goes beyond simply spending more than competitors. “We’re going to spend a lot more,” he said, but he added that the real objective is to build infrastructure that is “more reliable, more performant and more scalable than what’s available elsewhere.”