Standard guidelines for building large language models (LLM) optimize only for training costs and ignore inference costs. This is a challenge for real-world applications that use estimation-time scaling techniques to increase the accuracy of model responses, such as drawing multiple logic samples from a model upon deployment.
To bridge this gap, researchers at the University of Wisconsin-Madison and Stanford University have introduced Train-to-Test (TTE).2) scaling law, a framework that jointly optimizes a model’s parameter size, the amount of its training data, and the number of test-time estimation samples.
In practice, their approach proves that it is computation-optimal to train a significantly smaller model on much more data than traditional rules and then use the saved computational overhead to generate many repeated samples at inference.
For enterprise AI application developers who are training their own models, this research provides a proven blueprint for maximum return on investment. This shows that there is no need to spend huge amounts on frontier models for AI reasoning. Instead, smaller models can provide stronger performance on complex tasks while keeping per-query estimation costs manageable within real-world deployment budgets.
conflicting scaling laws
Scaling laws are an important part of developing large language models. Pretraining scaling laws decide the best way to allocate computation during model building test-time scaling law Provide guidance on how to allocate computation during deployment, such as letting the model “think long” or generating multiple logic samples to solve complex problems.
The problem is that these scaling laws, despite being fundamentally interconnected, were developed completely independently of each other.
A model’s parameter size and training period directly determine both the quality of its inference samples and the per-query cost. Currently, the industry’s gold standard for pretraining is chinchilla ruleswhich suggests a compute-optimal ratio of approximately 20 training tokens for each model parameter.
However, creators of modern AI model families, such as Llama, Gemma, and Quon, routinely break this rule by deliberately overtraining their small models on massive amounts of data.
As paper co-author Nicholas Roberts told VentureBeat, the traditional approach falters when creating complex agentic workflows: "In my view, the estimation stack breaks down when each individual estimation call is expensive. This is the case when models are large and you need to sample frequently." Instead of relying on massive models, developers can use supertrained compact models to run this repeated sampling at a fraction of the cost.
But because training and test-time scaling laws are examined separately, there is no rigorous framework for calculating how much more a model should be trained based on how many argument samples it will need to generate during deployment.
As a result, there was previously no formula that jointly optimized model size, training data volume, and test-time estimation budget.
The reason this framework is difficult to formulate is that pretraining and test-time scaling speak two different mathematical languages. During pretraining, a model’s performance is measured using “loss”, an intuitive, continuous metric that tracks prediction errors as the model learns.
When testing, developers use real-world, downstream metrics to evaluate a model’s reasoning capabilities, such as pass@k, which measures the probability that a model will give at least one correct answer in k independent, repeated attempts.
train-to-test scaling law
To solve the gap between training and deployment, researchers have developed a method called train-to-test (T2) scaling law. At a high level, this framework predicts the reasoning performance of a model by treating three variables as a single equation: the size of the model (N), the amount of training tokens it learns from (D), and the number of reasoning samples generated during inference (K).
Tea2 Combines the pretraining and inference budgets into an optimization formula that takes into account both the baseline cost to train the model (6ND) and the compounding cost to repeatedly query it at inference (2Nk). Researchers tried different modeling approaches: whether to model pre-training loss or test-time performance (pass@k) as functions of n, d, and k.
The first approach takes the familiar mathematical equation used for Chinchilla scaling (which calculates a model’s prediction error, or loss) and directly modifies it by adding a new variable that accounts for the number of repeated test-time samples (k). This allows developers to see how increasing guess count reduces the overall error rate of the model.
The second approach directly models the accuracy of the downstream pass@. It tells developers the probability that their application will solve a problem given a specific computation budget.
But should enterprises use this framework for every application? Roberts explains that this approach is highly specialized. "I think you won’t see as much benefit for knowledge-heavy applications like chat models," He said. instead, "Tea2 Designed to suit logic-heavy applications like coding, where typically you would use repeated sampling as your test-time scaling method."
What does this mean for developers
To validate t2 Scaling the laws, the researchers created an extensive test bed of more than 100 language models, ranging from 5 million to 901 million parameters. They trained 21 new, highly trained checkpoints to test whether their mathematical predictions held true in reality. They then benchmarked the models across eight diverse tasks, including synthetic tasks designed to test arithmetic, spatial reasoning, and knowledge recall, as well as real-world datasets such as SciQ and OpenBookQA.
Both of their mathematical models proved that the computation-optimal bounds deviate significantly from standard Chinchilla scaling. To maximize performance under a fixed budget, the optimal choice is a model that is significantly smaller and trained on much more data than the traditional 20-tokens-per-parameter rule.
In their experiments, the highly trained small model consistently outperformed the larger, chinchilla-optimal model in all eight evaluation tasks when test-time sampling costs were accounted for.
For developers wanting to implement these findings, the technical barrier is surprisingly low.
"There is no need for anything fancy to do test-time scaling with our current models," Roberts said. "At deployment time, developers can fully integrate infrastructure that makes the sampling process more efficient (e.g. KV caching if you’re using Transformer)."
KV caching helps by storing the already processed context so that the model does not have to re-read the initial signal for each new logic sample.
However, excessive overtraining comes with practical trade-offs. While overtrained models can be extremely stubborn and difficult to fine-tune, Roberts says that when he applied supervised fine-tuning, "While this effect was present, it was not a strong enough effect to pull the optimal model back to Chinchilla." The computation-optimal strategy is definitely inclined towards compact models.
Still, teams taking this to its full extent should be wary of reaching physical data limits. "The flip side is that if you take our overtraining recommendations to extremes, you may actually end up losing your training data," Roberts said, referring to the impending crisis. "data wall" Where high quality internet data has ended.
These experiments confirm that if an application depends on generating many test-time logic samples, aggressively overtraining a compact model is practically and mathematically the most efficient way to spend the end-to-end compute budget.
To help developers get started, the research team plans to soon open-source their checkpoints and code, allowing enterprises to plug in their own data and test scaling behavior immediately. Ultimately, this framework serves as an equalizing force in the AI industry.
This is especially important because the high cost of marginal models can become a barrier when you rely on logical models.
"Tea2 Fundamental changes that create stronger logic models," Roberts concluded. "You may not need a massive compute budget to achieve state-of-the-art logic. Instead, you need good data and smart allocation of your training and inference budget."
<a href