Let the AI Build a Wiki of Your Notes, but Keep Search

Anthropic published measurements on November 6, 2026 showing that an AI-maintained wiki, used alone, answered questions about a document corpus less accurately than a plain file-search tool — 48% of graded criteria versus 53%.¹ The wiki only pulled ahead when both tools ran together, rising from 54% to 61%, and the extra help concentrated on questions that required information from several documents at once.

The result runs against the popular intuition that a curated summary layer should outperform raw retrieval. It is also the first published head-to-head measurement of wiki versus search over a real document corpus — a question that note-keepers had been debating with nothing but intuition on either side.

What the study measured

Anthropic's research team ran what they call "dreaming" — a process in which a Claude agent reads a document corpus and writes wiki pages summarizing its contents, session by session.¹ The corpus came from Stainless, a software company Anthropic acquired, holding about 350,000 files and roughly 420 million tokens.¹ The evaluation set comprised 25 questions.

Dreaming is a form of offline pre-processing. The agent is not answering questions when it writes the wiki; it is condensing what it has read into persistent pages it can later consult. Andrej Karpathy described a structurally similar approach in his LLM Wiki gist: "the wiki is a persistent, compounding artifact" — a summary layer that accumulates rather than recomputes.² The earlier post on this pattern, Skip the Vector Database: Markdown Notes as AI Memory, explains why that framing attracted so much attention: a folder of plain-text wiki pages is a memory a model can read with no special infrastructure.

The Anthropic test asked what happens when you actually measure it.

Scores are reported as "percentage of graded criteria met," not as fully-correct answers — a distinction that matters when interpreting the numbers.¹

The numbers, plainly

Wiki alone vs search alone. "Claude's best score was 48% when it used only the wiki to answer questions."¹ The search tool alone scored 53%. The wiki on its own finished below the simpler tool.

Both together. "When it used both search and the wiki, Claude's aggregate score rose from 54% to 61% as that wiki grew, compared with 53% when it had access to the search tool alone."¹ Search with the wiki beat search alone by eight percentage points of graded criteria at the wiki's largest scale — 128 dreaming sessions.

Token savings. "Using search and the wiki also saved about 20,000 tokens per question."¹ The wiki gave Claude a head start on where to look, so the search calls were shorter.

Where the gain lives. The test included a merger task covering 24 questions, more than half of which required Claude to find information across several documents. On those items, "Claude scored 87% when it consulted the wiki, compared with 73% when it used search alone."¹ An improvement of fourteen percentage points on multi-document questions.

Where it did not. On one laboratory task, "brief criteria often could be retrieved with a text search from a single document, and the wiki showed no clear gain in performance relative to search."¹ The wiki added nothing — and cost additional context — when the answer lived in one place.

When the wiki grows larger than what it summarised

One finding does not appear in the headline numbers but belongs in any honest account of the test. On another laboratory task, the wiki expanded past the size of the original documents: "Claude had made an extremely detailed wiki that effectively abstracted nothing,"¹ and the figure caption records the result directly: "The wiki grew to be larger than the raw files."¹ Anthropic invokes Borges's famous map — the one so accurate it has the same scale as the territory — without naming the story.

The figure Anthropic provides is concrete: 10 source documents totaling 115 KB produced a wiki of 68 pages totaling 203 KB — 1.77 times the size of the material it was meant to summarize.¹ An independent byte-count receipt for this post confirmed the same ratio: 10 notes at 115,000 bytes expanded to 68 wiki pages at 202,980 bytes under equivalent conditions.³

A wiki is a compression of a corpus. When it exceeds the corpus in size, it is no longer doing that job. This is not a failure of the dreaming idea; it is a failure of the prompting and chunking strategy the agent used for that particular task. But it is a failure mode a note-keeper should expect and check.

One measurement note: du -sh overstates the ratio for many small files, because each file rounds up to a 4 KB disk block — producing a 2.27× result instead of the true 1.77× by byte content. If you run this check yourself, use du -sb on GNU systems, du -A on BSD, or the portable find . -exec cat {} + | wc -c.⁴⁵

A second finding the figures obscure

Anthropic also tested two wiki layouts: a shared summary table that merged facts from multiple contracts, and per-document pages that quoted contract terms verbatim.

The shared table retained 72% of the facts (41 out of 57) from the contracts it covered.¹ The per-document pages quoting terms word for word retained 100%.¹

The lesson generalises: a wiki page that summarises across sources drops facts. A wiki page that quotes and attributes them does not. This suggests that when an agent writes wiki pages, it should quote key claims rather than paraphrase them — a constraint that applies to AI-written summaries with the same force it applies to human note-taking.

The RAPTOR architecture (Recursive Abstractive Processing for Tree-Organized Retrieval), published at ICLR 2024, found that adding a summary tree alongside original chunks improved accuracy on complex, multi-step reasoning by roughly 20 percentage points on the QuALITY benchmark using GPT-4.⁶ But HippoRAG 2 (ICML 2025) found that "structure-augmented methods' performance on more basic factual memory tasks drops considerably below standard RAG."⁷ RAPTOR, specifically, "deteriorates substantially on the simple and multi-hop QA tasks due to the noise introduced … by its LLM summarization."⁸ The Anthropic result is consistent with both findings: the summary layer gains on complex multi-document questions and fails to help on simple single-document ones.

How to apply this to your own notes

The Anthropic study used a corporate document corpus and a single model family. The transfer to a personal notebook is an inference, not a measurement. But the structural conditions are testable: if most of your questions span several notes, a wiki layer should help; if most of them live in one note, it probably will not.

Three checks worth running on your own setup:

Count multi-note questions. Look at the last twenty questions you asked your notes-search tool, your AI assistant, or yourself. How many needed information from two or more notes to answer? If the fraction is below a third, the Anthropic data suggests a wiki will not reliably beat your existing search.

Keep search switched on. The key finding is that search plus wiki outperformed both tools alone, and that "search alone" is already the stronger baseline. Whatever you build on top of your notes, the file-search tool is not the thing to remove.

Check the wiki's size against its sources. If you are running an agent that writes wiki pages for you, compare the byte count of the wiki folder to the byte count of the source notes. A wiki approaching its sources in size is probably not compressing them — and the shared-table result suggests it may also be dropping facts.

The caveats the numbers cannot settle

Single vendor, single corpus, single model family. This is research from Anthropic about Anthropic's "dreaming" feature running on Claude. The corpus is a software company's internal documents plus legal-merger files, not a personal notebook. The evaluation set is 25 questions. The results have not been independently replicated.

Graded criteria, not correctness. The scoring measures how many criteria a response meets, not whether the response is fully correct. An answer can meet most criteria and still be wrong on the decisive fact.

HN thread critique. One substantive comment in the HN discussion (item 50402212, 77 comments) made the same point about scale: "dreaming adds only marginal gains to the wiki + search configuration … search is much cheaper."⁹ That observation is consistent with the data — the 54% → 61% improvement spans 128 dreaming sessions, which is a nontrivial inference budget for a 420-million-token corpus.

What "search" means here. The search tool in the test is a file-search tool — keyword and content search over the corpus. It is not vector-embedding retrieval. The comparison is wiki versus grep, not wiki versus semantic search. Those are different baselines.

Frequently Asked Questions

Is an AI-maintained wiki better than search for my notes? In Anthropic's test, the wiki alone scored 48% of graded criteria versus 53% for search alone — search was the stronger individual tool. Combined, they scored 61%, eight points above search alone. The combined advantage concentrated on questions requiring information from several documents at once.¹

When does an LLM wiki actually help with questions about my notes? The measured gain appeared on the merger task where most questions needed facts from across multiple documents — 87% with the wiki versus 73% with search alone.¹ On a task where most answers lived in a single document, the wiki showed no measurable gain. The practical test is whether your questions typically span several notes.

Can I remove the search tool once I have a wiki? The data suggests no. "In our tests, we found that the wiki helped Claude the most when we also gave it access to a search tool."¹ The wiki alone scored below the search tool alone. The two worked best together.

What is Anthropic's "dreaming" feature? Dreaming is an offline process in which a Claude agent reads a document corpus session by session and writes wiki pages summarizing what it has read. Those pages are stored and consulted at query time alongside a live search tool. It is conceptually similar to the Karpathy LLM Wiki pattern — a persistent summary layer beside the source files.²

What goes wrong when an AI builds a wiki without constraints? On one task, the agent wrote such detailed pages that the wiki grew larger than the original source documents — 10 documents (115 KB) producing 68 wiki pages (203 KB). The pages "effectively abstracted nothing," according to the researchers.¹ A wiki that exceeds its source corpus in size is no longer summarising it.

Does the wiki style matter — shared summaries or per-document pages? Significantly. The shared-table format, which merged facts from multiple contracts, retained 72% of facts. Per-document pages quoting contract terms verbatim retained 100%.¹ If an agent is writing wiki pages for you, it should quote key claims rather than paraphrase across sources.

Does the Karpathy LLM Wiki pattern work? Karpathy's framing — the wiki as a persistent, compounding artifact — was a description of the pattern, not a measurement of it.² The Anthropic test is the first measurement. The results are more nuanced: the wiki helps at scale, in combination with search, specifically on multi-document questions. It does not replace search.

Is this the same as RAPTOR or GraphRAG? Structurally related but distinct. RAPTOR builds a hierarchical summary tree and shows gains on complex multi-step questions and losses on simple factual ones.⁶ HippoRAG 2 uses a knowledge-graph structure over entities.⁷ Anthropic's dreaming writes flat wiki pages. All three are "summary layer plus retrieval" architectures; the tradeoffs differ.

The finding is uncomfortable for anyone building wiki layers as a replacement for search: the first measurement says they are not a replacement. Keep search, add the wiki if most of your questions span several notes, and check the wiki's size against what it covers.

Notes stay in plain files you can measure. MNMNOTE keeps them on your own device, open to any tool — mnmnote.com.