The Real Context the AI Saw Is Bigger Than What You Typed — Snapshot the Whole Assembly, Not Just Your Prompt

#124 was the raw text you typed. This piece is what the harness actually shipped — that plus the system prompt, every tool definition, every prior turn, and every retrieved chunk. The two are companions, not duplicates. Your typed prompt is 40 words; the API request body is often 4,000. Snapshot the whole assembly.

The vendor UI hides the shape of what it sent. The API reference does not. Anthropic's Applied AI team named the discipline last September: "Context engineering is the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information."¹ The essay enumerates the components in one line — "system instructions, tools, Model Context Protocol (MCP), external data, message history, etc."¹ Every one of those items is a separate top-level field in the request body. Every one is billed against the same window. And every one is trivial to save as a plain file on your device, next to the answer, so the next time a wrong answer arrives at 11pm you can read what the model actually read.

That is the personal-scale version of a discipline the enterprise already buys.

What most people believe: the prompt is what you typed

Most people equate the prompt with the last thing they typed into a chat box. The API contract calls that a single field among several, and the other fields carry weight. The mental model that survives is that your typed prompt is one layer of the assembly; the assembly is what the model reads.

The frame shifted this week without much noise. On dev.to, an indie developer named Jackson Ly put it in one line: "Your context window is a fixed budget. Every token you spend on a stale tool output is a token you did not spend on the thing that actually decides the next step."² The essay's title is precise about what fell out — What belongs in an agent's context window (and what to evict) — and its argument is that once you accept the assembly is real, the harder problem is not what to add but what to drop.

Testimony is not evidence. A chat scrollback tells you what the model said in reply. It does not tell you what the model was sent to compose that reply.

The pivot: the API's real input is an assembly

Every major API today ships the assembly as separate, serialisable fields. Anthropic's Messages API request body has three top-level arrays, each with its own schema. OpenAI's Responses API has two, a polymorphic input and a stand-alone instructions, plus a tools array. The Model Context Protocol standardises the tool-definition wire. The names differ per vendor; the shape does not.

Layer 1, the system prompt. Anthropic's Messages API reference documents the system field as "optional string or array of TextBlockParam" and defines it plainly: "System prompt. A system prompt is a way of providing context and instructions to Claude, such as specifying a particular goal or role."³ OpenAI's Responses API reference is unambiguous about what happens to the equivalent parameter: instructions is "a system (or developer) message inserted into the model's context. When using along with previous_response_id, the instructions from a previous response will not be carried over to the next response."⁴ Two vendors, one behaviour — the system prompt is in-context bytes, sent in the same request as your turn.

Layer 2, every prior turn in the conversation. Anthropic's messages parameter is "array of MessageParam (Required)" and its own description names the pattern: "Input messages. Our models are trained to operate on alternating user and assistant conversational turns. When creating a new Message, you specify the prior conversational turns with the messages parameter, and the model then generates the next Message in the conversation."³ The same reference caps the array at "a limit of 100,000 messages in a single request."³ OpenAI's input is polymorphic — a string, or "a list of one or many input items to the model, containing different content types."⁴ The doc is explicit about the echo: "Messages with the assistant role are presumed to have been generated by the model in previous interactions."⁴ Turn history is not implied by a user prompt; it is serialised into the request body every time.

Layer 3, every tool definition. Anthropic's tools field is "optional array of ToolUnion" — "Definitions of tools that the model may use. If you include tools in your API request, the model may return tool_use content blocks that represent the model's use of those tools."³ Each tool schema carries a name, a description, and an input_schema (a JSON Schema for the tool's arguments). The Model Context Protocol writes the same shape at the protocol layer: "To discover available tools, clients send a tools/list request. This operation supports pagination."⁵ Every conforming server declares its capability under { "capabilities": { "tools": { "listChanged": true } } }.⁶ The tool array is not free. It is bytes, in the request, per turn.

Layer 4, every retrieved chunk. Whatever the retrieval layer picks — a vector search, a keyword match, a file read the agent triggered — the payload is injected inline into messages (or input) content before the call goes out. It gets its own bytes and its own place in the assembly. A companion piece on this blog covers the which-files-did-the-agent-read side of the same question.⁷

The receipts are how you know all of this is real. Anthropic's Prompt Caching docs price the whole prefix — system, tools, prior turns, whichever is stable — as a first-class object: "Cache read tokens are 0.1 times the base input tokens price." "5-minute cache write tokens are 1.25 times the base input tokens price. 1-hour cache write tokens are 2 times the base input tokens price."⁸ OpenAI's parallel guide names its minimum and its retention window: caching "is enabled automatically for prompts that are 1024 tokens or longer" and "cached prefixes generally remain active for 5 to 10 minutes of inactivity, up to a maximum of one hour," with extended retention up to twenty-four hours; the mechanism can "reduce latency by up to 80% and input token costs by up to 90%" and "caching happens automatically, with no explicit action needed."⁹ Both vendors bill the assembly, not just your typed turn.

The argument: snapshot the whole assembly, on your own disk

The vendor's UI hides the layers. The API's usage response does not. Every conforming client has enough visibility to serialise each layer of the assembly to a file next to the answer. That file is the receipt, and the receipt is what makes tomorrow's answer reviewable.

The personal-vault owner's discipline is one shell command per turn. Concatenate system.md, tools.json, retrieved-chunks/*.md, and user-prompt.md into a single snapshot-<timestamp>.md before the API call fires. When the model replies, save the reply next to it. Six months later — or six minutes later, when a bad answer arrives — the file is a full transcript of the input the model received, not a summary of what the vendor decided to show you.

# minimal snapshot one-liner
cat system.md tools.json retrieved-chunks/*.md user-prompt.md > snapshot-$(date +%Y%m%dT%H%M%S).md

The equivalent of the same discipline at team scale is already a market — LLM-observability platforms sell dashboards that answer what did the model read? for engineering teams paying invoices. Their existence is the signal that the question is real; you do not need their invoice to run it at personal scale. The layers are already yours to save.

The Anthropic Effective Context Engineering essay puts the framing in one line: "the art and science of curating what will go into the limited context window."¹ Curation happens before the model reads. Auditing happens after. The snapshot is the artefact that lets you do both. It makes tomorrow's model call reviewable and yesterday's model call re-runnable.

The practice: what to save, in what order, and where

Save five things per turn as plain files in the vault next to the answer. The system prompt goes to system.md; the tools to tools.json; the prior turns append to a session file; the retrieved chunks to a timestamped folder; your typed turn to user-prompt.md. Concatenate; the snapshot is one file you can grep.

• system.md, the system prompt, verbatim. Anthropic exposes it in the request body as the system field; OpenAI exposes it as instructions.³⁴ If your client sets it silently on your behalf, print it once and paste it in.
• tools.json, the array of tool definitions with their name, description, and input_schema for each. On Anthropic this is your tools parameter; on MCP, it is the response to tools/list.⁵ Snapshot the array once per session, and grep in on schema drift.
• session-<date>.md, the append-only log of every prior turn. Anthropic's messages array is up to 100,000 messages long in principle; the local file only has to be as long as this conversation.³ A companion piece on session persistence covers the file discipline.¹⁰
• retrieved-<timestamp>/*.md, the retrieved chunks, one file per chunk, with the source pointer. Save them before the API call, not after. When the answer surprises you, the chunk that was actually pulled is the first thing worth reading.
• user-prompt.md, the raw turn you typed. This is #124's discipline; this piece is the union of the four layers above with it.¹¹

Then log the vendor's usage field alongside the snapshot. On Anthropic, the response body carries usage.cache_read_input_tokens and usage.cache_creation_input_tokens; on OpenAI, usage.input_tokens_details.cached_tokens. Those numbers are proof: they tell you the size of the assembly the vendor billed, in the vendor's own accounting. A companion essay on token cost covers why the size matters at all.¹²

The layering pairs with three other disciplines. Pinning the exact model version records what model read this, so the model is reproducible.¹³ The working-memory frame is the principle for which this piece is the practice.¹⁴ The chunking-shapes-retrieval argument sets up what the retrieval layer picks in the first place.¹⁵ Each layer catches what the layer above lets through, and the write-side record of what the agent changed closes the loop on the other end.¹⁶

The honest caveats: three failure modes to name

The snapshot narrows the surprise. It does not eliminate it. Three failure modes survive every tier of the discipline, and every reader should hold them. The snapshot captures what the client sent, not what the model finally attended to. A larger assembly is not a smarter answer. And the receipts — the vendor's cache prices — move under it.

One, the snapshot captures the CLIENT-SENT input, not the FINAL model-attention input. Whatever the client sent is on your disk. Whatever the vendor's safety layer, policy wrapper, or post-hoc system addition inserted server-side is not. The vendor may append a preamble the client never sees, wrap the request with an internal instruction, or attach a moderation directive that only their infrastructure can read. The discipline captures most of what the model read, perhaps ninety percent of the answer, and honestly named as most, not all. The last piece is opaque unless you are the vendor.

Two, a bigger context window does not mean a smarter answer. ValeryKot, writing on dev.to this week, put the honest observation as bluntly as it deserves: "Large context windows are incredibly useful. They just don't compensate for weak retrieval. If anything, they make weak retrieval look convincing."¹⁷ The same essay reminds the reader that "retrieval is a selection problem" and that "coherence is not the same thing as faithfulness."¹⁷ Logging the assembly makes noise visible; it does not remove the noise. A larger snapshot with the wrong chunks in it is still the wrong answer, just longer. The chunking-shapes-retrieval discipline is the corollary: audit the assembly, and shape what feeds it.¹⁵

Three, cache pricing drifts, and the receipts move under it. Anthropic's Prompt Caching page today prices cache reads at 0.1× base input tokens and cache writes at 1.25× (5-minute) or 2× (1-hour); the minimum cacheable prefix is 1,024 tokens for Sonnet 4.5 and Opus 4, and 2,048 for Haiku 3.5.⁸ OpenAI's guide today names 1,024-token minimums, five-to-ten-minute inactive retention up to one hour, and an extended-retention option up to twenty-four hours.⁹ Both vendors have moved these numbers before. Anthropic added the 1-hour tier after launch; OpenAI added the 24-hour extended retention. Any essay that quotes today's rate should stamp today's date on the footnote. Re-verify at publish, and re-verify again when the answer that comes back is a dollar different from what you expected.

Layered defence is the shape. The snapshot shrinks the surface a wrong answer can hide inside. The write-audit tells you what changed after the reads.¹⁶ The read-audit tells you which files the agent opened before the request went out.⁷ The typed-turn record tells you what you asked for.¹¹ Each is a layer. The assembly snapshot is the one that watches the request body.

Frequently asked questions

What is actually in my AI's context window besides my message? Four layers, all serialised into the same request body. The system prompt (Anthropic exposes this as system; OpenAI exposes it as instructions).³⁴ The full history of prior turns (Anthropic's messages array; OpenAI's input array of role-typed items).³⁴ Every tool definition with its JSON Schema (Anthropic's tools array; MCP's tools/list response).³⁵ And whatever the retrieval layer injected inline. Snapshot all four, plus your typed turn.

How do I log the full prompt, not just the user turn? Concatenate the four layers before the API call. A shell one-liner works: cat system.md tools.json retrieved-chunks/*.md user-prompt.md > snapshot-$(date +%Y%m%dT%H%M%S).md. Each layer is separately serialisable per the vendor spec — Anthropic's system/messages/tools are three top-level body fields³; OpenAI's instructions, input, and tools are three parameters at the same level.⁴ Save the receipt on your own disk, in the vault, next to the answer.

How big is my prompt really? Use the vendor's own usage response, not your typed word count. Anthropic returns usage.input_tokens on every response, plus usage.cache_read_input_tokens and usage.cache_creation_input_tokens when caching is on.⁸ OpenAI returns usage.input_tokens_details.cached_tokens.⁹ Those numbers are the assembly the vendor billed. The delta between the vendor's number and your typed word count is the iceberg the vendor already priced.

Does the AI see the system prompt? Yes, verbatim. Anthropic's reference defines system as an "optional string or array of TextBlockParam" separate from messages but sent in the same request body.³ OpenAI is even more explicit: instructions is "a system (or developer) message inserted into the model's context."⁴ The system prompt is in-context bytes, billed at the same rate as your turn (or discounted when cached).

Do tool definitions count as context tokens? Yes. Anthropic's tools parameter is "an array of ToolUnion" where each tool has a name, a description, and an input_schema (JSON Schema for the tool's inputs).³ MCP's tools/list response returns the same shape at the protocol layer.⁵ Prompt caching prices the tools block at the same 0.1× cache-read rate as the system prompt and priors.⁸

What tokens does prompt caching actually read? The cached prefix is whatever is stable across turns — usually the system prompt, the tool definitions, and the earliest turns of the conversation. Anthropic requires a 1,024-token minimum for Sonnet 4.5 and Opus 4 (2,048 for Haiku 3.5) and prices reads at 0.1× base.⁸ OpenAI requires a 1,024-token minimum with automatic caching and up to 90% cost reduction.⁹ Both vendors price the assembly, not just your typed turn.

Does a bigger context window mean a smarter answer? No. ValeryKot, writing on dev.to, said it plainly: "Large context windows are incredibly useful. They just don't compensate for weak retrieval. If anything, they make weak retrieval look convincing."¹⁷ The same piece names the underlying problem, "retrieval is a selection problem," and the honest limit, "coherence is not the same thing as faithfulness."¹⁷ Logging the assembly makes noise visible; only better retrieval removes it.

Steph Ango's version is the philosophical anchor: "Apps are ephemeral, but your files have a chance to last. If you want to create digital artifacts that last, they must be files you can control, in formats that are easy to retrieve and read."¹⁸ The corollary is that the snapshot of what the app read from your files is also yours to keep. The vendor's chat window will roll away when the vendor does. The plain file the shell wrote survives.

Your notes live on your device as plain Markdown files, so when the retrieval layer picks a chunk, the chunk is already a file — mnmnote.com is the same shape at the layer above.

References