The Metis memory foundation model is an early research prototype that gives a language model a persistent latent memory state inside its forward computation. Instead of retrieving old text and adding it to every new prompt, Metis compresses interaction history into dynamic matrices attached to the model. Later queries read those matrices through a dedicated memory-attention path while the ordinary model weights remain frozen.
The authors call this arrangement native memory: memory is represented by model-internal state, and learned procedures decide how to store and use it. Their released family uses Qwen3.5 backbones at 4B, 9B, and 27B scales. The reported results show that useful information can survive after the original text leaves the context window, with lower tail latency on some long-history workloads. They also show why native memory is not yet a replacement for external agent memory: fixed-size compression loses older information, similar facts interfere, and irrelevant stored material can degrade unrelated tasks.
What “native” means
Metis draws the boundary around memory more narrowly than most agent systems. Information learned before an interaction is treated as knowledge in static model weights. Information acquired during interaction is memory stored in a dynamic state.
| Layer | External memory | Metis native memory |
|---|---|---|
| Persistent state | Text, records, vectors, files, or database rows outside the model | Dense matrices and normalization state inside selected Transformer layers |
| Write path | Explicit insertion, summarization, indexing, or retrieval-system logic | A learned state update during forward computation |
| Read path | Retrieve, rerank, and place selected text into context | Query the latent state through memory attention |
| Online gradients | Usually none, except explicit test-time adaptation methods | None; the deployed weights stay frozen and memory updates are forward-only |
| Inspectability | Stored items can remain addressable and reviewable | Individual remembered claims are entangled in a latent matrix |
This is different from ordinary test-time training. Metis is mid-trained to carry state across separate interaction steps and to interpret operations such as remembering, updating, forgetting, and reflecting. It is also different from memory-augmented neural networks whose learned controller reads and writes a separate slot store. The paper places both the state and its update procedure directly in the backbone computation.
Architecture
A Metis block augments selected Transformer layers with two components.
The local memory block holds the changing session state: a memory matrix and a normalization vector . These are the fast, dynamic parameters. Resetting them starts an independent memory session.
The hyper memory block contains static parameters learned during mid-training. It scores intermediate token representations, selects a compact subset, projects those representations into memory keys and values, and computes the next local memory state. The released implementation uses a gated-delta update rather than gradient descent.
The runtime cycle is:
- Process an interaction using the ordinary model and the current memory state.
- Select informative hidden states from that interaction.
- Transform them into an update for and .
- On a later query, use a separate learned memory query to read the state.
- Blend the memory readout with the layer's normal attention output.
The released code exposes this lifecycle directly. A caller can commit no message, only the user message, or the complete user-assistant exchange, and can reset the state between sessions. The historical text does not need to be replayed during the later query.
Training the memory procedure
The authors synthesize supervision from 27 public benchmarks. The primary corpus contains 357,137 samples and about 406 million tokens across four operations:
- remember: store a fact and answer a later question about it;
- update: replace an earlier value before it is queried;
- forget: revoke information so it no longer supports the answer;
- reflect: combine facts introduced at separate steps.
Each operation appears in explicit, implicit, and distractor-heavy forms. A further 609,443 auxiliary samples target two failure modes: interference among similar facts and memory pollution in ordinary dialogue. The auxiliary tasks bind multiple entities, forget one fact without erasing another, return to normal conversation after a memory operation, and answer questions that should ignore memory entirely.
During mid-training, the Qwen3.5 backbone remains frozen and only native-memory parameters are optimized. The reported runs used eight H100 GPUs. The project currently releases model checkpoints, architecture code, inference examples, and the training harness, but its repository labels the release a research preview and says the training data is not yet included.
Reported evidence
The paper compares models that can see the full original context, retrieval systems that see selected context, and models that must answer with no historical text in the query. The most useful comparison is within that last group: it tests whether information actually survived in model state.
| Benchmark average | Plain Qwen3.5-27B, no context | Temp-LoRA-27B | Metis-27B | Qwen3.5-27B, full context |
|---|---|---|---|---|
| MemOps Gold operations | 1.69 | 9.70 | 24.76 | 87.90 |
| Metis test-set operations | 16.87 | 23.86 | 73.77 | 78.87 |
| LoCoMo Gold question answering | 0.07 | 4.24 | 26.74 | 65.03 |
| NextMem contextual generation | 17.75 | 30.97 | 50.82 | 78.80 |
Metis leads the no-context baselines in these reported averages, but full context remains substantially stronger on three of the four evaluations. The external MemOps benchmark is especially sobering: the best native-memory result is less than one-third of the full-context score. Forgetting is the weakest operation on that benchmark.
At 4B scale on LoCoMo Gold evidence sessions, Metis reports 0.562 seconds average end-to-end latency and 0.926 seconds at the 95th percentile, compared with 0.607 and 3.012 seconds for full context. Retrieval into a partial context was still faster at 0.268 and 0.456 seconds. In a controlled history-length sweep, Metis became faster than full context only at the first measured 64K-token point; at 128K it was 1.50× faster for 32 generated tokens and 1.60× faster for 128. Native memory moves work into a history-ingestion stage rather than making that work disappear.
These are results reported by the system's authors on a recent preprint. Many answers are scored with an LLM judge, the training corpus is synthetic and not yet released, and the benchmarks do not establish long-lived production behavior.
Failure modes
Metis makes the compression boundary unusually visible.
In the paper's capacity study, recall declines when a single update grows beyond several hundred words. Recall also degrades over repeated updates: early facts fade, newer facts remain unstable, and accumulated state transitions introduce interference throughout the memory matrix.
Stored noise can damage ordinary capability. With an empty state, Metis-4B remains close to its Qwen3.5-4B backbone on four general benchmarks. After an irrelevant message is committed, performance drops on every benchmark. The largest reported change is IFEval, which falls from 76.71 for the backbone with the same message in context to 54.53 for Metis with that message stored, a 22.18-point gap.
The latent representation also changes what can be inspected. A text record can be cited, corrected, permissioned, and deleted as a distinct object. A blended matrix does not naturally preserve those object boundaries. The paper identifies semantic confusion, capacity, controllability, and interpretability as open problems and explicitly describes hybrid native-external systems as future work.
What changes for agent architecture
Metis shows that one layer of memory can move from an agent harness into the model without online gradient updates or repeated context replay. That is a real architectural option, especially for dense short-term personalization and long histories whose full text would be expensive to prefill repeatedly.
It does not eliminate the rest of the memory lifecycle. Long-lived agents still need durable identity, source provenance, permissions, correction history, portability, restoration, and evidence about which state caused an action. Those are system properties represented by context repositories, explicit records, and trajectories rather than by recall accuracy alone.
The useful distinction is therefore not native memory versus external memory as mutually exclusive camps. It is latent learned state for efficient behavioral continuity alongside inspectable external state for custody and truth. Metis is evidence that the first half can work. Its own failure analysis is evidence that the second half remains necessary.