AI agents forget. Every time a coding assistant loses track of a debugging thread, or a data analysis agent re-accesses the same context it has already processed, the team pays in the form of latency, token costs, and brittle workflows. The solution most teams reach for – expanding the context window or adding more RAGs – is increasingly expensive and still doesn’t work reliably.
To address this, researchers at Mind Lab and several universities proposed Delta-Mem, an efficient technique that compresses a model’s historical information into a dynamically updated matrix without changing the model itself. The resulting module adds only 0.12% of the parameters of the backbone model – compared to 76.40% for a leading alternative – while it performs better on memory-heavy benchmarks. Delta-MEM allows models to continuously accumulate and reuse historical data, reducing reliance on large-scale reference windows or complex external retrieval modules for behavioral consistency.
long memory challenge
The traditional solution is to simply dump all the information into the model’s context window.
But as paper co-author Jingdi Lei told VentureBeat, current systems treat memory only as a context-management problem. “Either we keep expanding the context window, or we retrieve more documents through RAG,” Lei explained. “These approaches are useful and will remain important, but when agents need to operate on long-lasting, multi-step interactions, they become increasingly expensive and brittle, and they don’t really [work] Like human memory because they prefer to look at documents.”
In enterprise settings, the constraint is not just whether the model can access the history, but whether it can reuse that history efficiently, consistently, and with low latency. Standard attention mechanisms incur quadratic computational cost as the sequence length increases. Furthermore, expanding the context window does not guarantee that the model will actually remember the information effectively. Models often suffer from context erosion or context rot as they become overwhelmed with excess (and often conflicting) information, even if they can support one million tokens in theory.
Researchers argue for advanced memory mechanisms that can compactly represent historical information and maintain it dynamically across interactions. Existing solutions come with heavy transaction costs and generally fall into three paradigms:
- Text memory: Stores history as text injected into context – constrained by window boundaries and prone to information loss under compression.
-
Outer-channel (RAG): Encodes and retrieves from external modules – adds latency, integration complexity, and potential misalignment with the backbone.
-
Parametric: Encodes model weights in memory via adapter – static after training, cannot adapt to new information during live interactions.
inside delta-mem
To achieve a compact and dynamically updated memory, delta-mem compresses an agent’s past interactions into an “online state of associative memory” (OSAM). This state is maintained as a fixed-size matrix that preserves historical information while the underlying language model remains stable.
For enterprise workflows, this simply means solving operational bottlenecks. Lei notes that a persistent coding assistant, for example, “may need to remember project conferences, recent debugging steps, user preferences, or intermediate decisions in the workflow.” Similarly, a data analysis agent “may need to maintain task state, beliefs, and prior observations while iterating over multiple tool calls.”
Instead of repeatedly retrieving and recomputing all the relevant history for these tasks, the delta-mem matrix provides a low-overhead way to carry forward useful interaction states inside the model’s further computations.
During generation, the system does not retrieve raw text segments to add to the prompt. Instead, the current hidden state of the backbone LLM is projected into the matrix to retrieve the old memory. This operation extracts context-relevant associative memory signals from the delta-mem. These signals are then converted into numerical corrections that are applied to the model’s calculations. It controls the logic of the model at the time of estimation without changing its internal parameters.
After each interaction, Delta-Mem updates the online state using “Delta-Rule Learning”. When new information arrives, the previous state makes predictions about the resulting attention values. It then compares this prediction with the actual value and corrects the memory matrix based on the discrepancy.
This updating mechanism relies on the “gated delta-rule”. Basically, memory modules have separate knobs that control how much previous memory is kept and how much new memory is put in. This error correction with controlled obsolescence allows the matrix to evolve over time, maintaining stable historical associations without being derailed by short-term noise.
The researchers explored three strategies for determining when and how the matrix updates:
- write token-state Captures fine changes but is sensitive to short-term noise.
-
write sequence position Average tokens within a message segment, smoothing updates at the expense of some local detail.
-
multi-state writing Decomposes memory into sub-states for different information types such as facts or work progress.
Delta-mem in action
The researchers evaluated delta-mem across three LLM backbones: Qwen3-8B, Qwen3-4B-Instruct, and SmolLM3-3B. They configured the framework with a compact 8×8 matrix. The system was tested on common capability benchmarks including HotpotQA, GPQA-Diamond and IFEVAL. It was also evaluated on memory-heavy tasks such as Locomo, which tests long-term conversational memory, and the Memory Agent Bench, which assesses retention, retrieval, selective forgetting, and test-time learning over extended interactions.
The framework was compared to representative models of three existing memory paradigms: textual memory baselines (e.g., BM25 RAG, LLMLingua-2, and MemoryBank), parametric systems (Context2LoRA and MemGen), and the outer-channel approach MLP memory.
According to the researchers, across the board, delta-mem outperformed the baseline. On the Qwen3-4B-instruction backbone, the token-state write variant achieved an average score of 51.66%, easily surpassing the frozen vanilla backbone at 46.79% and the strongest baseline, Context2LoRA, at 44.90%. On the memory-heavy memory agent bench, the average score increased from 29.54% to 38.85%. Performance on the specific test-time learning subtask nearly doubled from 26.14 to 50.50.
However, the most compelling finding is the operational efficiency of the system. The researchers tested the framework in a de-context setting where the historical text was completely removed from context. Even without explicit text replay, Delta-MEM successfully recovered context-relevant evidence in multi-hop tasks. The researchers argue that the model remembers past interactions without the need to take large amounts of instant tokens.
The framework only adds 4.87 million trainable parameters, which is only 0.12% of the Qwen3-4B-Instruct backbone. By comparison, the MLP memory baseline requires 3 billion parameters, which adds up to 76.40% of the backbone size, giving the following results. When the prompt length increased to 32,000 tokens during inference tests, the framework maintained almost the same GPU memory footprint as a standard, unmodified model. This eliminates the massive memory bloat that plagues other advanced memory systems like MemGen and MLP memory.
Different updating strategies proved beneficial depending on the underlying model capability. The sequence-position writing strategy was most effective for a strong backbone such as Qwen3-8B. These more efficient models use segment-level writes to smooth updates and reduce token-level noise. In contrast, the multi-state write strategy led to massive performance leaps for smaller backbones like SmolLM3-3B. For these low capacity models, separating the memory into multiple states proved important to reduce information interference.
Implementing Delta-MEM in an Enterprise Stack
The researchers have released the code for the delta-meme and the weights for their trained adapter on hugging faces on GitHub. For AI engineering teams who want to integrate this framework into their existing inference stack, the process requires minimal computing resources.
“In practice, an engineering team would start with an existing instruction-tuned backbone, attach delta-mem adapter modules to selected attention layers, train only the adapter parameters on domain-relevant multi-turn or long-context data… and then infer with the memory state updated online during the conversation,” Lei said. Importantly, teams do not need massive pre-training funds. The training data only needs to reflect the target memory behavior, such as multi-turn dialogue, agent trace, or domain workflow where earlier information should influence later decisions.
While compressing the interaction history into a fixed-size mathematical matrix produces immense efficiency, it comes with trade-offs. Delta-MEM is not a lossless replacement for clear text logs or document retrieval. Because different pieces of information compete in the same limited state, there is a risk of memory mixing.
“Delta-MEM is useful when systems require fast, online, continuously updated behavioral state,” Lei said. “RAG is better when the system requires accurate factual recall, citation, compliance, audit, or access to a large external knowledge base.” Remembering a user’s actions or multi-step logic trajectories is a perfect fit for a delta-mem, while retrieving legal contracts or medical guidelines should reside in a vector database.
This means that the most realistic enterprise architecture going forward is a hybrid approach. Delta-MEM acts as a lightweight internal working memory, reducing the need to retrieve or re-run everything all the time, while RAG acts as an explicit, high-capacity memory layer.
“Looking ahead, I don’t think vector databases will become obsolete,” Lei said. “Instead, I expect enterprise AI stacks to become more layered. We’ll likely see short-term working memory inside models, long-term explicit memory in retrieval systems, and policy or audit layers that decide what should be stored, retrieved, forgotten, or exposed to the user.”
<a href