Companion article
Context window versus memory: what actually persists?
Separate what a model can use now from what an application saves, retrieves, compacts, or leaves in your files.
Available now and saved for later are different promises
An agent can leave a file on disk, keep a conversation in a session store, and still fail to use either in its next answer. The context window is a capacity limit for a model invocation. Context is the material actually supplied. A memory system saves information and provides a way to retrieve it later. Storage alone does not put that information in front of the model.
Imagine a coding task with a requirement to preserve old term URLs. The requirement may be visible in yesterday’s chat and written in a specification. If today’s harness loads neither, the model has no direct evidence of that requirement. Increasing the context limit will not retrieve it. A useful next action is to read the current spec and compare the relevant routes, rather than ask the model to remember harder.
| Layer | What can survive? | How it reaches the next decision |
|---|---|---|
| Context | Nothing is guaranteed beyond the request by this concept alone. | The harness assembles or restores input. |
| Session history | Messages and tool activity, subject to the store’s policy. | History is replayed, filtered, or compacted. |
| Saved memory or files | Notes, preferences, source code, and artifacts. | A loader or tool retrieves selected contents. |
| Model parameters | Patterns learned through training. | Inference uses the weights; reading a note does not retrain them. |
Follow the state through a real boundary
Start by asking what crossed the boundary: another model request, another user turn, a restarted application, or a new workspace? These are different events. A session may span many requests while a temporary filesystem disappears with its environment. A repository file may survive a cleared conversation while an unsaved plan does not.
For a concrete diagnosis, inspect the assembled input or the harness’s retrieval log. Find the requirement itself, a trustworthy summary of it, or a resolved pointer to its source. A filename listed in context is only a context pointer. It does not demonstrate that the file’s contents were loaded. If the evidence was loaded and the agent still ignored it, the failure is no longer explained by storage alone.
Provider features implement these ideas differently
In OpenAI’s Responses API, a conversation or a previous_response_id can connect requests without the caller manually repeating the entire history. Those are OpenAI API mechanisms, not universal meanings of “memory.” They do not turn the model’s window into unlimited storage. Other harnesses manage their own session history and send selected items explicitly.
Compaction reduces the context carried forward. A harness may make a readable summary; OpenAI’s documented compaction can return encrypted, opaque state alongside retained items. Do not assume every compaction artifact is a summary you can edit. Keep exact requirements in a maintained source and reload them when precision matters.
A prefix cache is a separate optimization: it reuses computation for eligible matching input. It neither retrieves a forgotten requirement nor frees those input tokens from the context window. A cache hit is also not a promise that the next answer will match the previous one.
Try it: saved does not mean supplied
This small Python 3 experiment isolates context assembly without a model, network call, or account. Run it in a terminal. It creates a temporary note, builds a request without the note, then explicitly loads it. The assertions describe expected program behavior; this is not a measurement of model recall.
In a coding assistant, the equivalent follow-up is to start a fresh task and inspect which repository instructions or memory items the harness loads. Record the product and settings, the saved location, and the observed tool reads. Do not infer that storage failed solely because an answer omitted the note: retrieval, selection, and use are separate steps.
from pathlib import Path
from tempfile import TemporaryDirectory
with TemporaryDirectory() as directory:
note = Path(directory) / "requirements.txt"
note.write_text("Preserve /term/token/", encoding="utf-8")
request = ["Review the routes."]
assert note.exists()
assert "Preserve /term/token/" not in "\n".join(request)
request.append(note.read_text(encoding="utf-8"))
assert "Preserve /term/token/" in "\n".join(request)
print("Saved note became context only after loading.")Design for recovery, not apparent recall
Before clearing or handing off a long task, save the objective, constraints, completed edits, checks, and next action in a handoff artifact. Link to exact sources and identify assumptions that still need checking. A short, maintained note is easier to inspect than a claim that the agent remembers everything.
When a later answer contradicts a decision, first check whether the decision was saved, whether it was retrieved, and whether it was still current. Then inspect whether the model used it correctly. This separates a missing-state problem from a reasoning or verification problem and gives you a concrete repair at each step.