Enterprise AI is entering a new phase – where the central question is no longer what can be built, but how to get the most from our AI investments.
In VentureBeat’s latest AI Impact Tour session, Brian Gresley, director of portfolio strategy at Red Hat, described the operational reality inside large organizations: AI sprawl, rising inference costs, and limited visibility into what returns those investments are actually delivering.
This is the “Day 2” moment – when pilots give way to production, and cost, administration, and sustainability become harder than building the system in the first place.
"We’ve seen customers who say, ‘I have 50,000 CoPilot licenses. I don’t really know what people are getting from it. But I know I’m paying for the most expensive computing in the world, because it’s a GPU." Said politely. "’How will I keep this under control?’"
Why is enterprise AI cost now a board-level issue?
Over the past two years, cost was not the primary concern for organizations evaluating generic AI. The experimental phase gave teams cover to spend freely, and the promise of productivity gains justified aggressive investment, but this dynamic is changing as enterprises enter their second and third budget cycles with AI. out of focus "Can we make something?" To "Are we getting what we paid for?"
Enterprises that made big, early bets on managed AI services are taking a hard look at whether those investments are delivering measurable value. The issue isn’t just that GPU computing is expensive. This is because many organizations lack the tools to link spending to results, making it nearly impossible to justify innovation or scale responsibly.
Strategic shift from token consumer to token producer
The dominant AI procurement model of the last few years has been straightforward: pay a vendor per token, per seat, or per API call, and let someone else manage the infrastructure. That model made sense as a starting point, but it is being questioned by organizations without enough experience to compare alternatives.
Enterprises that have gone through an AI cycle are starting to rethink that model.
"How can I start being a token generator, instead of purely being a token consumer?" Said politely. "Are there use cases and workloads that are suitable for me to own more? This could mean operating GPU. This may mean renting a GPU. And then ask, ‘Does that workload require the largest state-of-the-art model?’ Are there more capable open models or smaller models that fit?’"
The decision is not binary. The right answer depends on the workload, organization, and risk tolerance involved, but the math is becoming more complex as the number of open models enabled increases, from DeepSeek to models now available through cloud marketplaces. Now enterprises actually have real alternatives to the handful of providers that dominated the landscape two years ago.
Falling AI costs and rising usage create a paradox for enterprise budgets
Some enterprise leaders argue that locking in infrastructure investments now could mean paying significantly more in the long run, pointing to Anthropic CEO Dario Amodei’s statement that AI inference costs are falling by about 60% per year.
The emergence of open-source models like DeepSeek and others over the past three years has meaningfully expanded the strategic options available to enterprises willing to invest in the underlying infrastructure.
But while the cost per token is decreasing, usage is increasing at a pace that offsets increased efficiency. This is a version of the Jevons paradox, the economic theory that improving resource efficiency increases rather than reduces total consumption, because lower costs enable wider adoption.
For enterprise budget planners, this means that declining unit costs do not translate into a decline in total billings. An organization that triples its AI usage, while halving costs, still has to spend more than before. The idea is to identify which workloads actually require the most capable and most expensive model, and which can be handled just as well by smaller, cheaper options.
The business case for investing in AI infrastructure resilience
The prescription is not to slow down AI investments, but to build with resilience in mind. The organizations that win will not necessarily be those that move fastest or spend the most; They are building infrastructure and operating models capable of absorbing the next unexpected growth.
"The more you can create some abstractions and give yourself some flexibility, the more you can experiment without increasing costs, but also without putting your business at risk. This is just as important as asking if you are doing everything best practice right now," Explained politely.
But despite how deeply AI discussions have become in enterprise planning cycles, most organizations’ practical experience is still measured in years, not decades.
"It seems like we’ve been doing this forever. We have been doing this for three years," Added politely. "It’s early and it’s moving really fast. You don’t know what’s going to happen next. “But the specifics of what’s going to happen next – you have to have some sense of what that looks like.”
For enterprise leaders who are still calibrating their AI investment strategies, this may be the most practical solution: The goal is not to optimize for today’s cost structure, but to build organizational and technical flexibility to adapt whenever it changes again.
<a href