When an enterprise LLM retrieves a product name, technical specification, or standard contract clause, it is using expensive GPU calculations designed for complex logic – just to access static information. This happens millions of times every day. Each lookup wastes cycles and increases infrastructure costs.
DeepSeek’s new ongoing research "conditional memory" Addresses this architectural limitation directly. The work introduces engrams, a module that separates static pattern retrieval from dynamic logic. This gives results that challenge the notion of what memory is really for in neural networks. The paper was co-authored deepseek Founder Liang Wenfeng.
Through systematic experiments DeepSeek found the optimal balance between compute and memory, allocating 75% of sparse model capacity to dynamic logic and 25% to static lookups. This memory system improved reasoning more than knowledge retrieval.
Complex reasoning benchmarks increased from 70% to 74% accuracy, while knowledge-focused tests improved from 57% to 61%. These improvements came from tests including Big-Bench Hard, ARC-Challenge, and MMLU.
The research comes as enterprises face increasing pressure to deploy more capable AI systems while reducing GPU memory bottlenecks and infrastructure costs. DeepSeek’s approach offers a potential way forward by fundamentally rethinking how models should be structured.
How conditional memory solves a different problem than agentic memory and RAG.
Agentic memory systems, sometimes called episodic memory – e.g. massa, memoOr memp – Focus on episodic memory. They store records of past conversations, user preferences, and interaction history. These systems help agents maintain context throughout the session and learn from the experience. But they are external to the forward pass of the model and do not optimize how the model internally processes stable linguistic patterns.
For Chris Latimer, founder and CEO of Vectorize, which developed Hindsight, the conditional memory approach used in Engrams solves a different problem than agentic AI memory.
"This is not solving the problem of linking agents to external memory such as conversation history and knowledge stores," Latimer told VentureBeat. "It is more geared towards squeezing performance out of smaller models and getting more out of scarce GPU resources."
Conditional memory deals with a fundamental problem: Transformers lack basic knowledge lookup primitives. When processing text, they must simulate the retrieval of stable patterns through expensive neural computations across multiple layers. These patterns include named units, technical terminology, and common phrases.
The DeepSeek paper shows this with a concrete example. recognize "Diana, Princess of Wales" Generating features sequentially requires consuming multiple layers of attention and feed-forward networks. The model uses deep, dynamic logic circuits to essentially perform a simple hash table lookup. It’s like using a calculator to remember your phone number instead of just looking it up.
"The problem is that Transformer lacks ‘Basic Knowledge Search’ capability," The researchers write. "Many tasks like recovery that must be solved in O(1) time have to ‘simulate for recovery’ through a large amount of computation, which is very inefficient."
How does conditional memory work?
engram introduces "conditional memory" Working with conditional calculations of MoE.
The mechanism is straightforward. The module takes a sequence of two to three tokens and uses a hash function to look them up into a huge embedding table. Retrieval occurs in constant time, regardless of the size of the table.
But the retrieved patterns need to be filtered. a hash lookup for "Apple" May collide with unrelated content, or the word may mean fruit rather than company. Engram solves this with a gating mechanism. The model’s current understanding of the context (accumulated through earlier attention layers) acts as a filter. If the retrieved memory contradicts the current context, the gate suppresses it. If it fits, the gate lets it in.
The module does not apply to every layer. Strategic placement balances performance gains against system latency.
This dual-system design raises an important question: How much capacity should each get? DeepSeek’s main finding: The optimal partitioning is 75-80% for compute and 20-25% for memory. Testing found the net MoE (100% calculation) proved to be less than optimal. Too much computation wastes the depth of reconstruction of static patterns; Too much memory loses reasoning ability.
Infrastructure Efficiency: GPU Memory Bypass
Perhaps the most practical contribution of Engram is its infrastructure-aware design. Unlike MoE’s dynamic routing, which relies on runtime hidden states, engram’s retrieval indices depend entirely on input token sequences. This deterministic nature enables the prefetch-and-overlap strategy.
"The challenge is that GPU memory is limited and expensive, so using larger models becomes expensive and difficult to deploy," Latimer said. "The clever idea behind Engram is to keep the main model on the GPU, but load a large portion of the model’s stored information into a separate memory on regular RAM, which the model can access on an appropriate time basis."
During inference, the system can asynchronously retrieve embeddings from the host CPU memory via PCIe. This happens when the GPU computes the preceding Transformer block. Strategic layer placement takes advantage of the calculation of early layers as a buffer to hide communication latency.
The researchers demonstrated this with a 100B-parameter embedding table that was completely offloaded to host DRAM. They achieved less than 3% throughput penalty. Separating storage from compute solves a significant enterprise bottleneck as GPU high-bandwidth memory remains expensive and scarce.
What this means for enterprise AI deployment
For enterprises evaluating AI infrastructure strategies, DeepSeek’s findings suggest several actionable insights:
1. Hybrid architectures outperform pure approaches. The 75/25 allocation law indicates that the optimal model should divide sparse capacity between compute and memory.
2. Infrastructure costs may shift from GPU to memory. If engram-style architecture proves viable in production, infrastructure investment patterns may change. The ability to store 100B+ parameters in CPU memory with minimal overhead shows that memory-rich, compute-moderate configurations can provide better performance per dollar than pure GPU scaling.
3. Improvement in reasoning outweighs knowledge gains. The surprising finding that reasoning benefits more than knowledge retrieval suggests that the value of memory goes beyond the obvious use cases.
For enterprises leading AI adoption, Engram shows that the next frontier may not just be larger models. These are better architectural choices that respect the fundamental difference between static knowledge and dynamic logic. Research suggests that optimal AI systems will increasingly resemble hybrid architectures.
Organizations waiting to adopt AI until later in the cycle should monitor whether major model providers incorporate conditional memory principles into their architectures. If the 75/25 allocation law applies across all scales and domains, next-generation Foundation models can provide significantly improved reasoning performance at lower infrastructure costs.
<a href