How AI Memory Actually Works

Open ChatGPT in a browser and ask it to remember that you are vegetarian. It can carry that fact into a later conversation. Open Claude, work inside a project, and it can maintain a memory for that project without mixing it with another one.

Now move into a terminal. Open Claude Code or Codex on a real repository. What counts as memory there is often a markdown file: CLAUDE.md for Claude Code, or an equivalent instruction file read by the coding agent.

These two worlds use the same word for very different mechanisms.

In the browser, the memory system lives on the provider's side. In your terminal, the file lives on your disk. The browser system can select information and bring it forward. The local file gives you ownership and legibility. But a file does not rank its contents, understand that one fact replaced another, or decide which three lines matter for the question you just asked. It is a document, read as a document.

That contrast is the cleanest place to start, because it removes a common mistake: memory is not whatever text happens to survive between prompts.

ChatGPT itself has two memory mechanisms, according to OpenAI's published documentation. Saved memories are the explicit items you tell it to remember. They are visible, editable, and deletable. Reference chat history is the implicit mechanism: it selects useful information from earlier conversations to carry forward. These mechanisms are controlled separately, and saved memories are stored separately from chat history. Deleting a conversation does not, by itself, delete a saved memory created from that conversation.

Claude's published documentation describes separate memory per project. Its memory summary can be viewed and edited, while incognito chat provides a way to avoid carrying a conversation into memory. That project boundary matters. Client work and personal work should not become one undifferentiated pool.

Claude Code is a different case. Its memory mechanism is hierarchical markdown files named CLAUDE.md. The file is client-side, inspectable, and yours. That is useful. It is also static. If an old instruction remains after the architecture changes, the file does not know it is stale. If it grows to several pages, it does not know which paragraph deserves attention now. It has no retrieval system because it is not a retrieval system.

So the real engineering problem is not “how do I preserve text?” The problem is: how do I build something that can decide what is worth keeping, recover it by meaning, understand time and relationships, stay inside the right boundary, and become better without silently becoming worse?

That is an AI memory system.

A database is not a memory

Suppose I save every conversation for six months. Nothing is lost. I now ask: “What did we decide about payment retries?”

The database can search for the words “payment retries.” But the decision may have been written as: “If the card fails, wait and try again, but only twice.” The meaning matches. The words do not.

A conventional keyword lookup finds what matches. Memory has to find what means the same thing.

That is where vectors enter.

A vector is a position on a map of meaning. Put “king” and “queen” on that map and they should sit near each other. Put “pizza” on it and it should sit elsewhere. A real map has hundreds of directions, sometimes more than a thousand, because the system needs enough room to separate fine shades of meaning. You do not need to picture every direction. The useful idea is simply that related text receives nearby coordinates.

An embedding is the operation that produces those coordinates. Text goes in. A position comes out. People often use “embedding” and “vector” as if they mean the same thing. In casual discussion that is harmless. Technically, the vector is the position; embedding is the process of working out that position.

Once stored text has positions, “payment retry logic” can land near “if the card fails, wait and try again.” Retrieval no longer depends on shared spelling. The system searches a neighbourhood of meaning.

That is semantic search. It is necessary. It is not sufficient.

There is also an operational trap here. The map is not universal. Different embedding models draw different maps. Change the embedding model and you change the coordinate system used to interpret stored material. The underlying memories did not change, but the ground beneath their positions did. Any production memory design has to treat embedding choice and migration as system concerns, not as a hidden implementation detail.

The three holes in vectors alone

Vector search looks so convincing in a demo that teams mistake it for the whole system. It fails in three predictable ways.

Similar is not the same

Compare these two statements:

We decided to retry twice.

We considered retrying twice and rejected it.

They use almost identical language. Their vectors can sit close together. Their operational meanings are opposite. Distance can tell us that both concern the same subject. Distance alone cannot tell us which decision became valid.

This is why a nearest-neighbour result is a candidate, not an answer.

Vectors have no clock

Imagine that a team says in March, “We use Postgres.” In June, the team migrates. In July, an agent asks memory which database the project uses.

Both statements can be semantically relevant. The March statement may even be a closer wording match. But it is no longer current. A vector does not understand that March preceded June or that a later fact superseded an earlier one.

Time cannot be pasted on as decorative metadata. It has to participate in ingestion, contradiction detection, invalidation, retrieval, and ranking.

Proximity is not a relationship

Vectors tell us which things are near one another. They do not tell us that a bug came from a decision made in a meeting by a particular person, or that a workaround belongs to a specific release and was retired by a later fix.

Those are edges, not distances.

A useful memory system therefore needs three structures at once: a semantic map, a graph of connections, and a clock. The map finds related meaning. The graph explains how pieces relate. The clock tells the system what was true when, and whether it is still true now.

That combination is the beginning of memory. A vector database by itself is still storage with an unusually good search function.

The three types of memory

The next mistake is to treat every retained item as the same kind of object. Human memory gives us a cleaner model.

Episodic memory is what happened. Your first day at a job is an episode: people, place, sequence, and time belong together. In an agent system, a debugging session, a decision meeting, or a failed deployment is episodic. The timestamp is part of the event, not an optional label attached later.

Semantic memory is what is true. Paris is the capital of France. You may not remember when you learned that fact because the fact survived while the original episode disappeared. In engineering work, “this service owns invoice generation” is semantic memory. It may have originated in a conversation, but the useful retained object is the claim.

Procedural memory is how to do something. Riding a bicycle is the standard human example: you can perform the skill without being able to write a complete description of balance. For an AI agent, a verified workflow, a proven recovery sequence, or a reusable procedure belongs in this category.

These three types have different shapes and different retrieval needs. A chat archive is mostly episodic. It records what was said and when. Calling that complete memory is like calling a server log an operating manual and a knowledge base at the same time.

The distinction matters for AI Reliability Engineering because reliability depends on feeding the agent the right kind of evidence. An event can explain why a decision happened. A fact can state the current decision. A procedure can tell the agent what to do next. Flatten them into one text pile and the agent has to reconstruct those differences every time it answers.

How a memory gets made: the seven-stage ingestion pipeline

Storing a memory is not one write. In SuperLocalMemory v3.8.10, the ingestion path is a sequence of gates and transformations. Each stage answers a different question.

1. Decide whether the input is worth keeping

Most text is not durable information. Greetings, repeated acknowledgements, transient tool noise, and duplicated context can overwhelm retrieval if everything is retained. The first stage asks whether the input carries enough information to justify its future cost.

This is the role represented by entropy_gate.py. The principle is plain: do not make retrieval harder by storing noise.

Every accepted item will cost storage, indexing work, retrieval time, and possibly prompt tokens later. A memory system that accepts everything has avoided judgment at ingestion and pushed the entire burden into recall.

2. Classify what arrived

Is the item a fact, an event, a preference, or another memory shape? Classification controls what later stages should do with it. The verified module here is type_router.py.

This is not filing for filing's sake. A preference may remain valid until explicitly changed. An event belongs on a timeline. A fact may contradict an existing fact. Routing lets the system apply the correct rules instead of treating every sentence as a generic chunk.

3. Extract the actual claim

A paragraph is not a fact. It may contain context, hedging, alternatives, and one load-bearing assertion. fact_extractor.py pulls out the claim that should be represented.

Consider: “We tested three options. Redis was fastest, but because this service must survive a cold restart without another dependency, we chose the local store.” Saving the entire paragraph may be useful as an episode. The semantic claim is narrower: the service uses the local store, with a stated reason.

Extraction makes the retained unit explicit enough to compare, connect, invalidate, and retrieve.

4. Resolve entities

“The client,” “Rahul,” and “that customer” may refer to one entity across several months. If the system stores them as three unrelated names, it does not have one memory of the person or organisation. It has fragments that cannot reliably meet.

entity_resolver.py handles this stage. Entity resolution gives later graph and retrieval operations a stable thing to point at.

This is also where careless systems create false joins. Two people can share a name. A good resolver has to avoid turning linguistic similarity into identity without enough evidence.

5. Parse time and validate temporal truth

temporal_parser.py attaches time. temporal_validator.py checks whether the new information invalidates something already believed.

This stage is what separates accumulation from learning. New information does not always sit beside old information. Sometimes it overrules it.

Crucially, invalidation should not mean erasure. If the project used Postgres in March and migrated in June, the March fact was true in March. The system may need that history to explain an old incident or reproduce an earlier release. The correct state is superseded, with a timeline, not deleted as if it had never existed.

6. Connect the memory

auto_linker.py creates relationships to related memories, facts, entities, and events. This builds the web that vectors cannot provide.

The relationship can answer questions that similarity cannot: which decision caused this change, which event confirmed a claim, which person owns the component, or which procedure resolved the incident.

The graph is valuable because reasoning often travels through a connection. A question may not resemble the target memory closely in vector space, but an entity or event path can still lead to it.

7. Consolidate

consolidator.py performs the final stage. Human memory does not keep every sensory detail forever. Repeated episodes become patterns; details fade while a useful summary remains.

An artificial memory system needs the same discipline. Without consolidation, it grows into a warehouse of near-duplicates. The retrieval problem becomes harder with every accepted item, even if every item was reasonable on its own.

Consolidation turns accumulated experience into something more compact and reusable. It is not deletion with a nicer name. It is the conversion of repeated or related material into a stronger representation while preserving what remains important.

How memory comes back: six channels, fusion, and re-ranking

When a user asks a question, a capable memory system does not run one search. It runs several searches in parallel because relevance has more than one shape.

SuperLocalMemory v3.8.10 has six verified retrieval channel modules.

semantic_channel.py searches the meaning map. This is the vector path. It finds material that expresses related ideas even when the wording differs.

bm25_channel.py searches keywords. Semantic retrieval did not make literal text useless. Exact names, error strings, identifiers, and rare terms often need lexical search.

entity_channel.py retrieves around a person, project, customer, component, or other resolved entity. It answers “what do we know about this thing?” even when the individual memories use different language.

temporal_channel.py searches by time. It can prefer the relevant period and help distinguish current truth from historical truth.

hopfield_channel.py follows connections. It uses the web rather than only the map, letting retrieval reach related material through stored relationships.

profile_channel.py applies scope. It keeps retrieval inside the active world instead of allowing a relevant-looking memory from the wrong project or identity to leak into the answer.

These six channels will disagree. That is expected. Each produces scores with its own meaning and scale. A semantic similarity score cannot be averaged naively with a keyword score or a graph score.

The verified fusion.py module uses Weighted Reciprocal Rank Fusion. The important move is to combine rank positions rather than pretend raw scores are comparable. Each channel returns an ordered list. Fusion rewards candidates that appear strongly across several lists, with weights reflecting the channels trusted for the query. In the current verified implementation, the default fusion constant is k=15.

This gives the system a useful property: a memory that appears partway down several independent lists can beat a memory that appears first in only one. Agreement across retrieval views becomes evidence.

Fusion still produces candidates, not final truth. The first stages are designed to be broad and fast. The last stage can spend more compute on fewer items.

That is the job of reranker.py: a subprocess-isolated cross-encoder reads the question and each top candidate together, then judges whether the candidate actually answers the question. Unlike the original vector lookup, this model gets to inspect the relationship between query and candidate directly.

The order matters. Running the expensive judge over the full store would be wasteful. Running only fast retrieval would leave too many semantic near-misses. Broad retrieval narrows the field. Fusion combines different kinds of evidence. Re-ranking performs the careful final selection.

There is a useful scar in the code: an earlier fusion version re-fused results three times and destroyed the rankings. That detail is more instructive than a perfect architecture diagram. Retrieval components do not become correct merely because each one sounds reasonable. Their composition has to be measured.

Forgetting is a requirement, not a defect

A thought experiment makes the scaling problem obvious. Imagine a store with a million memories. This is not a claimed benchmark or measured capacity result. It is a way to expose what breaks.

Every retained memory is another candidate that can look relevant. A useful result can be buried under a large number of things that resemble it. Perfect retention therefore does not produce perfect recall. It can produce noise.

Deleting by age is not enough. An architecture decision from a year ago may still govern the system. A message from minutes ago may already be worthless. Age and importance are different variables.

SuperLocalMemory couples two mechanisms to deal with this problem.

The first is the Ebbinghaus forgetting curve. Ebbinghaus's work dates to 1885. The shape is the point: forgetting is steep early and then flattens. In the verified coupling code, retention contributes to forgetting drift as:

lambda_forget = (1 - R) * forgetting_drift_scale

The curve tells the system how memory fades over time. It does not, alone, tell the system which memory deserves to resist that fade.

The second mechanism couples Fisher confidence to Langevin dynamics. Picture a memory as a particle moving within a boundary. If it reaches the boundary, it is archived. Temperature controls how strongly that particle moves.

The conceptual relationship is:

T_eff = T0 / confidence

The implementation adds an epsilon guard:

T_eff = T_0 / (fisher_confidence + epsilon)

High confidence makes the denominator larger, so effective temperature drops. The memory moves less and stabilises toward the active region. Low confidence makes effective temperature higher. The memory moves more and drifts toward archival.

The two mechanisms are coupled. The verified implementation combines Fisher temperature and Ebbinghaus forgetting drift as:

T_combined = T_fisher * (1 + lambda_forget)

This gives the system a way to forget without a hand-written cleanup schedule deciding each record's fate. Confident memories stabilise. Uncertain memories are more likely to fade. The resulting behaviour is based on both time and learned confidence, not on “delete everything older than this date.”

That distinction is central to AI Reliability Engineering. Forgetting is safe only when it is governed, inspectable, and coupled to evidence about usefulness. An unbounded store is unreliable because noise grows. A blunt retention rule is unreliable because it can remove old but governing knowledge. The system needs controlled decay.

Temporal invalidation: preserve history without serving stale truth

Forgetting and invalidation solve different problems.

Forgetting manages value under scale. Invalidation manages truth under change.

Return to the database example. “We use Postgres” was true in March. A later migration makes another statement true in June. If both facts remain active with equal standing, the memory system can retrieve obsolete architecture with complete confidence.

The wrong fix is to erase March. Historical questions still need it. An incident from April may make sense only under the old architecture.

The correct model is a timeline with supersession. The earlier fact remains available as historical truth, while the later fact becomes current truth. Retrieval can then answer two distinct questions correctly:

“What database do we use now?”

“What database were we using when the April incident happened?”

This is why time belongs inside the memory object and the retrieval logic. A timestamp column added after the fact does not automatically create temporal reasoning. The ingestion pipeline must detect a possible contradiction, validate it, link the new and old states, and change which one is treated as current.

A learning system needs a system that can stop it

The six retrieval channels need weights. Those weights can be guessed once and frozen, or they can learn from actual recall outcomes.

Learning sounds obviously better. It is also where a memory system can quietly degrade.

A new ranking model may look promising on a small sample and perform worse after promotion. Without a guardrail, “self-improving” means the system is authorised to reduce its own quality without an alarm.

SuperLocalMemory's verified learning discipline uses shadow testing and rollback.

Queries are routed deterministically using a hash, so the same query goes to the same lane even across a daemon restart. That makes the comparison reproducible rather than random.

Phase A is a fast triage at n=100. Early promotion requires both a strong effect and statistical significance. If that gate is not met, Phase B continues to n=885 paired comparisons. That sample size is set for a minimum detectable effect of 0.02, power 0.8, two-sided alpha 0.05, and sigma 0.15.

Promotion is not the end of validation. The next 200 recalls are watched against the pre-promotion baseline. If mean NDCG@10 drops by at least 2%, the system automatically rolls back. The model flag changes happen in one BEGIN IMMEDIATE transaction, and retraining is disabled for 24h after rollback so the system cannot immediately repeat the same failure.

There is also a defined failure path for a missing previous model. The code does not demote the active model and leave the user with nothing. It logs the error, enters safe mode, and falls back to the Phase-2 heuristic.

This is the pattern I care about: learning is allowed only inside a reversible control loop.

The numbers are not decoration. n=100 is triage, not final proof. n=885 is the full paired validation under the stated power and significance assumptions. 200 is the post-promotion watch. A 2% mean NDCG@10 drop is the rollback threshold. Each number corresponds to a different failure mode.

Anyone can add retraining. Reliable systems define what evidence permits promotion, what evidence triggers reversal, and what happens when reversal itself cannot complete normally.

Profile isolation: walls at the record level

Memory becomes dangerous when several worlds share one system.

Client project. Personal project. Day job. A query in one should not retrieve a plausible answer from another. Semantic relevance does not grant permission.

SuperLocalMemory's verified profile model scopes every memory, fact, entity, and learning record with profile_id. The boundary exists at the record level, including the learning data, rather than only at the conversation or interface level.

This is columnar isolation, not separate stores. That distinction matters because the wrong mental model leads to the wrong operational claims.

Switching profiles is config-only and moves zero data. Records stay where they are. The active profile changes which scoped records the system can operate on. There is no copy, export, or migration during a switch.

The design rule is private by default and shared only by an explicit scope decision. If a memory that should have been shared remains private, the failure is reduced availability and can be corrected. If a private memory leaks into another profile, the failure may be irreversible.

At organisational scale, profile isolation is only part of the boundary. Role-based access determines who may read, write, delete, or inspect the audit trail. Retrieval quality cannot compensate for weak access control. A highly relevant result from the wrong profile is still the wrong result.

Caching and compression: the unglamorous economics

Memory costs compute when text is embedded. It costs time during retrieval. It costs prompt tokens when retrieved material is injected into a model call.

Two practical levers control that cost.

First, do not repeat work. Exact caching can reuse a result for the same question. Semantic caching can reuse work when differently worded questions mean the same thing. The verified cache modules cover exact and semantic paths, centroid storage, invalidation, and stampede control: exact.py, semantic.py, centroid_store.py, invalidation.py, and stampede.py.

Cache invalidation matters because memory changes. A cached answer that ignores a newly superseding fact is fast and wrong. The cache has to participate in the same truth lifecycle as the underlying memory.

Second, reduce what is sent. Retrieved memories are prose, and prose can be compressed while retaining the useful meaning. The verified compression path includes ccr.py, prose_llmlingua.py, router.py, and align.py.

I am deliberately not attaching a compression ratio or cache hit rate. Those figures were not verified in the source material for this article. The engineering point does not need an invented percentage: repeated retrieval wastes compute, and verbose context consumes tokens on every call.

Caching prevents repeated work. Compression reduces the payload. Both become more important as memory stops being a demo and becomes infrastructure used every day.

Multi-agent shared memory

Most developers no longer use one AI surface. An editor agent, a terminal agent, and a background process may all touch the same project. Without shared state, each works from a partial view. One can repeat a failed approach. Another can undo a decision made minutes earlier. The human becomes the message bus between tools.

Putting memory below the agents changes that shape.

An agent records a verified decision into the shared layer. Another agent retrieves it through the same scoped system. The memory is not trapped inside either agent's private transcript. SuperLocalMemory's verified mesh modules include mesh/broker.py and mesh/remote_sync.py for this shared-memory direction.

Shared does not mean unbounded. The profile and permission rules still apply. The value is that authorised agents can coordinate through one memory layer instead of maintaining conflicting local histories.

This is also why memory belongs outside the model. Models and tools can change. A durable memory layer can serve several agents while keeping the truth lifecycle, retrieval pipeline, isolation policy, and learning controls consistent.

The reference implementation

Memory is not storage. Storage is the easy part.

Memory is the system that decides what deserves to survive, what kind of thing it is, which entity it belongs to, when it was true, what it connects to, whether it has been superseded, how confidently it should remain active, which profile may see it, and whether it actually earned its place in an answer.

That is a large claim, so I prefer an implementation you can inspect over a diagram you have to trust.

SuperLocalMemory is the open-source reference implementation for the architecture described here. The verified source for this article is v3.8.10: the seven-stage ingestion path, six retrieval channels, Weighted Reciprocal Rank Fusion, cross-encoder re-ranking, Ebbinghaus and Fisher-Langevin forgetting, deterministic shadow tests, automatic rollback, per-record profile_id isolation, cache and compression modules, and shared-memory mesh components.

This is what AI Reliability Engineering looks like at the memory layer: not a promise that the model will remember, but a set of explicit mechanisms for deciding what memory means, measuring whether recall improved, and recovering when it did not.

Read the code. The scars are part of the design.