ScaleOps has expanded its cloud resource management platform with a new product aimed at enterprises running self-hosted large language model (LLM) and GPU-based AI applications.
The AI Infra product announced today expands the company’s existing automation capabilities to meet the growing need for efficient GPU utilization, predictable performance and reduced operational burden in large-scale AI deployments.
The company said the system is already running in enterprise production environments and is delivering major efficiency gains for early adopters, reducing GPU costs by between 50% and 70%. The company does not publicly list enterprise pricing for this solution and instead invites interested customers to receive a custom quote based on their operation size and needs.
Explaining how the system behaves under heavy load, Yoder Shafrir, CEO and co-founder of ScaleOps, said in an email to VentureBeat that the platform “uses proactive and reactive mechanisms to handle sudden spikes without impact on performance,” noting that its workload entitlement policies “automatically manage the ability to keep resources available.”
He said reducing GPU cold-start latency was a priority, emphasizing that the system “ensures immediate response as traffic increases,” especially for AI workloads where model load times are substantial.
Expanding Resource Automation into AI Infrastructure
Enterprises deploying self-hosted AI models face performance variability, long load times, and persistent underutilization of GPU resources. ScaleOps introduced the new AI Infra product as a direct response to these issues.
The platform allocates and scales GPU resources in real-time and adapts to changes in traffic demand without requiring changes to existing model deployment pipelines or application code.
According to ScaleOps, the system manages production environments for organizations including Viz, DocuSign, Rubrik, Coupa, Alkamy, Vantour, Grubhub, Island, Chewy, and several Fortune 500 companies.
The AI Infra product introduces workload-aware scaling policies that proactively and reactively adjust capacity to maintain performance during spikes in demand. The company said these policies reduce cold-start delays associated with loading large AI models, improving responsiveness when traffic increases.
Technical integration and platform compatibility
The product is designed for compatibility with common enterprise infrastructure patterns. It works across all Kubernetes distributions, major cloud platforms, on-premises data centers, and air-gapped environments. ScaleOps emphasized that deployment does not require code changes, rewrites of infrastructure, or modifications to existing manifests.
Shafrir said the platform “seamlessly integrates into existing model deployment pipelines without requiring any code or infrastructure changes,” and he added that teams can immediately begin optimizing with their existing GitOps, CI/CD, monitoring, and deployment tooling.
Shafrir also explained how automation interacts with existing systems. He said the platform operates without disrupting workflows or creating conflicts with custom scheduling or scaling logic, explaining that the system “does not change the manifest or deployment logic” and instead enhances schedulers, autoscalers, and custom policies by incorporating real-time operational context while respecting existing configuration limitations.
Performance, Visibility and User Control
The platform provides full visibility into GPU utilization, model behavior, performance metrics, and scaling decisions at multiple levels, including pods, workloads, nodes, and clusters. While the system enforces default workload scaling policies, ScaleOps noted that engineering teams retain the ability to tune these policies as needed.
In practice, the company aims to reduce or eliminate the manual tuning that DevOps and AIOps teams typically do to manage AI workloads. Installation requires minimal effort, described by ScaleOps as a two-minute process using a single Helm flag, after which customizations can be enabled through a single action.
Cost Savings and Enterprise Case Studies
ScaleOps reported that early deployment of the AI Infra product has resulted in a 50-70% reduction in GPU costs in customer environments. The company gave two examples:
-
A major creative software company operating thousands of GPUs had an average utilization of 20% before adopting ScaleOps. The product increased utilization, consolidated underutilized capacity, and enabled the reduction of GPU nodes. These changes reduced overall GPU costs by more than half. The company also reported a 35% reduction in latency for key workloads.
-
A global gaming company used the platform to optimize dynamic LLM workloads running on hundreds of GPUs. According to ScaleOps, the product increased usage sevenfold while maintaining service-level performance. The customer estimated annual savings of $1.4 million from this workload alone.
ScaleOps said the expected GPU savings typically exceed the cost of adopting and operating the platform, and customers with limited infrastructure budgets have reported rapid returns on investment.
Industry Context and Company Perspective
The rapid adoption of self-hosted AI models has created new operational challenges for enterprises, particularly around GPU efficiency and the complexity of managing large-scale workloads. Shafrir described the broader scenario by saying that “cloud-native AI infrastructure is reaching a breaking point.”
“Cloud-native architectures unlocked great flexibility and control, but they also introduced a new level of complexity,” he said in the announcement. “Managing GPU resources at large scale has become chaotic – waste, performance issues and skyrocketing costs are now the norm. The ScaleOps platform was built to fix this. It provides an end-to-end solution for managing and optimizing GPU resources in a cloud-native environment, enabling enterprises to run LLM and AI applications efficiently, cost-effectively, and improve performance.”
Shafrir said the product brings together the full set of cloud resource management functions needed to manage diverse workloads at scale. The company has positioned the platform as a holistic system for continuous, automated optimization.
An integrated vision for the future
With the addition of the AI Infra product, ScaleOps aims to establish a unified approach to GPU and AI workload management that integrates with existing enterprise infrastructure.
The platform’s early performance metrics and reported cost savings suggest a focus on measurable efficiency improvements within the expanding ecosystem of self-hosted AI deployments.