Don't Build a RAG Over Your Vault. Grep It.

For a personal vault under about a million words and known-vocabulary queries, a regex over plain files — piped to the model you already pay for — beats a vector database on cost, latency, and the thing that matters most when retrieval misbehaves: being able to read why it returned what it did. Start with rg.

This is the upstream decision the "build a RAG over your notes with Pinecone" tutorials skip. Wang et al.'s February 2026 GrepRAG paper measured a regex-driven retriever against a vanilla embeddings RAG on a published code-completion benchmark and reported the grep variant ahead on exact-match accuracy at roughly one-fortieth the per-query latency¹². A peer-reviewed result on code is not a peer-reviewed result on notes — but it is directional evidence against the assumption that vectors are the floor. The follow-on argument is about scale and vocabulary, and the evidence for it lives in the tool's own benchmark table³ and in Anthropic's own published acknowledgement that lexical signals belong in the answer⁴.

The peer-reviewed result that quietly inverted the default

A peer-reviewed February 2026 paper — Wang et al.'s GrepRAG — measured a regex-driven retriever against vanilla embeddings RAG on the DevEval code-completion benchmark and found the grep variant ahead on exact-match accuracy at roughly one-fortieth the retrieval latency¹². The result is on code, not on notes — but it is the first peer-reviewed inversion of the vector-default.

The headline number is straight from the paper. "In the Python subset under the DeepSeek-V3.2-EXP setting, our method achieves an EM rate of 38.61%, substantially exceeding the traditional Vanilla RAG (24.99%) and RLCoder (36.59%)"¹. The Java number reads the same way — "Similarly, in the Java subset, Naive GrepRAG establishes a new benchmark with an EM of 41.70%"⁵. Two languages, the same direction.

The retrieval cost is the part that ought to be staring back at anyone who has paid an embeddings bill. The paper reports "the average retrieval latency of less than 0.02s" for the grep-driven variant². The round-trip is dominated by the model call, not the index — the same shape as a rg -n piped to a chat client.

The transfer to a personal vault is not free, and we will name that bound. A code corpus is structured with shared vocabulary; a notes vault is messier and more synonymic. The paper's evidence is directional — it falsifies the assumption that vectors are required for any retrieval that an LLM consumes. It does not prove the personal-vault case on its own. The case for that rests on the tool's latency envelope below and on Anthropic's own published hybrid-retrieval result⁴.

Ripgrep on a single machine is sub-second on a real corpus

A regex over a personal notes folder takes less time than the model takes to reply, on any laptop made in the last five years. Andrew Gallant's ripgrep README publishes the comparison: ripgrep returns 536 matches for [A-Z]+_SUSPEND on the entire Linux kernel source tree in 0.082s — against git grep at 0.273s (3.34×) and ack at 2.935s (35.94×)³.

The whole table is on the README, and the numbers are reproducible on a fresh linux/ checkout. The kernel tree is not a personal notes vault — it is much bigger. The latency envelope is therefore the ceiling, not the floor, of what a small corpus pays.

The honest cap matters more than the headline. When the corpus is no longer a directory tree but a single ~13GB compressed text file — the OpenSubtitles English dump the README uses — "passing the -n flag (for showing line numbers) increases the times to 1.664s for ripgrep and 9.484s for GNU grep"⁶. Even at thirteen gigabytes of single-file text, the wait is under two seconds. A personal notes vault is not thirteen gigabytes of single-file text. The latency budget is hardly the constraint.

The implication for an agent is the same as for a person. The model is already waiting on the network; the regex is not what makes the round-trip slow. The model receives a head -50 of matched lines and reads them. That is the whole retrieval pipeline.

The 198-point Show HN that productionized exactly this

A developer named alexmrv posted a working version of the idea to Hacker News in August 2025 — "Show HN: I replaced vector databases with Git for AI memory (PoC)" — and it drew 198 points and 45 comments⁷. The repository it pointed to, Growth-Kinetics/DiffMem, today carries 890 stars under an MIT license⁸.

The thread is ten months old, not yesterday — but the repository is live, the design is documented, and the comment thread is the public record of what the small-corpus audience actually said. The submission, item 44969622 on the front-page index, is the canonical reference page that Google indexes for "DiffMem vs vector database" queries today⁷.

The DiffMem README states the design in one paragraph. The project is "a lightweight, git-based memory backend for AI agents and conversational systems" that "uses Markdown files for human-readable storage, Git for tracking temporal evolution through differentials, and a git-native retrieval agent that explores the repository via shell commands (grep, git log, git diff, git blame)"⁹. The README's punchline is shorter: "No vector databases, no embeddings, no BM25 — just git and an LLM"¹⁰.

The thread's named reply is the one to take to heart. mattnewton wrote, on the same Show HN, that "agentic retrieval where you simply pass an annotated list of files to an LLM and ask it which ones it wants to look at is probably better than traditional RAG for small datasets"¹¹. The phrase to keep is small datasets — the spine of this whole post.

The five-minute version

Five lines and a model — the smallest version that does real work. The folder is the source; the regex is the retriever; the model is the reasoner. No embedding API, no chunker, no index to maintain. When the model cites a wrong line, you open the file and see why — the failure mode is legible by construction.

1. Open a terminal in the folder that holds your notes. The folder is the source of truth; everything downstream reads from it.
2. Run a single regex over the tree. rg -n -w 'topic' ~/notes | head -50 returns up to fifty matched lines with line numbers and filenames. The -w flag enforces whole-word matches; the head -50 cap keeps the prompt small.
3. Paste the output into the chat client you already use, with one prompt: "Here are the matched lines from my notes for topic. Answer my question using only these matches; tell me if the matches do not contain the answer."
4. Ask the question.
5. If the answer disappoints, open the file at the line number the model cites and read the surrounding paragraph. The matched line is a coordinate; the file is the page.

That is the retrieval system. The model is the reasoner; the regex is the retriever; the file is the record. When the model cites a wrong line, you can open the file and see why — which is more than the vector store will tell you.

The thirty-minute version

The thirty-minute upgrade is the same loop with three small additions — a saved prompt, a wrapping shell function, and the habit of writing the vocabulary you actually use into the notes. Each is a one-time cost that widens the regex's reach without adding a dependency. The model decides what to look at next; the file stays where it is.

Save the prompt as a plain text file in the same folder. Wrap the rg call in a shell function — ask_notes() { rg -n -w "$1" ~/notes | head -50; } — so the regex is one keystroke away. Then, when you notice a note you wrote uses vacation and your query says PTO, add the synonym as a parenthetical in the note. The vocabulary you write is the vocabulary the regex can find.

The second addition is to let the model widen the regex on its own. After the first rg returns nothing, pass the same prompt with the user's plain-English question and ask the model to suggest two more search terms — synonyms, common abbreviations, related verbs — and re-run those. This is the agentic retrieval mattnewton named: an annotated index, the model deciding what to look at next¹¹. The DiffMem README writes the same loop in shell commands rather than English: grep, git log, git diff, git blame, each chosen by the model⁹.

The third addition is to write a manifest. A 30-line plain text file at the root of the folder lists what is in each top-level subdirectory — one line per folder, written by you, not by an indexer — so the model has a map before it has the matches. The manifest is the cheapest possible index, and it is the one that will outlive every retrieval architecture you try.

When grep is the wrong tool

There are three honest cases where grep is the wrong primary tool — a wide vocabulary gap between query and source, a corpus past the single-machine envelope, or a genuinely conceptual question whose words you never wrote down. Each is a real bound. The honest upgrade is rarely "embeddings alone" — it is hybrid, in Anthropic's published shape.

• The vocabulary gap is wide. You wrote "vacation leave" everywhere; the query is "PTO." The regex finds nothing. Anthropic's Daniel Ford, writing on Anthropic's own blog, reported that on their evaluation corpus "Contextual Embeddings reduced the top-20-chunk retrieval failure rate by 35% (5.7% → 3.7%)" and "Combining Contextual Embeddings and Contextual BM25 reduced the top-20-chunk retrieval failure rate by 49% (5.7% → 2.9%)"⁴. The honest upgrade is both signals — the lexical pass picks up the exact strings the embedding pass misses, and adding it to embeddings cuts the failure rate by another roughly forty percent on their workload.
• The corpus is larger than a personal vault. Single-machine regex is sub-second on the Linux kernel tree and under two seconds on a thirteen-gigabyte single file³⁶. Past that envelope — a multi-machine corpus, a team wiki of a hundred contributors, an enterprise document store — the operational story changes. The thesis here is the personal vault.
• The query is genuinely conceptual. "Find the note where I argued against premise-as-conclusion reasoning" is not a known-vocabulary query if you never wrote the phrase. A vector pass with a small embedder is a reasonable lookup for the literal idea. The grep pass is best when the query is a word you wrote.

The thread's honest counter, from the same Show HN, is petesergeant's reframing — if you are not using embeddings, why would you need a vector DB? Because you want to use embeddings. The spine is the if. The choice is upstream of the index.

The decision rule, written down once

The Pareto frontier on a personal corpus has two coordinates — the corpus has to fit single-machine regex latency, and the typical query has to be a term you remember writing. When both hold, regex-and-pipe wins on cost, latency, and debuggability. When either fails, the upgrade is a lexical-plus-embedding hybrid in Ford's shape⁴ — not embeddings alone.

The villain is the shape, not the vendor. Pinecone, Weaviate, and Chroma are serious products built by serious people — they are the right answer for the corpus that is not a personal vault. They are the wrong answer for a hundred markdown files a single person owns.

Common mistakes

Five common mistakes account for almost every "grep didn't work for my notes" story — embedding before measuring, treating a code benchmark as a notes benchmark, dropping the lexical signal when you do go to embeddings, chunking before a real question, and closing the corpus to make regex easier. Each is a one-line fix.

• Embedding the whole vault before measuring grep. The vendor tutorials start at the chunker; the honest comparison starts at rg. Run the regex first; pay for embeddings second — if at all.
• Treating the GrepRAG numbers as a personal-vault number. They are a code-completion benchmark¹. The transfer is directional — the personal-vault claim rests on the latency envelope and the vocabulary-overlap argument, not on the EM rate.
• Dropping lexical signals when you do go to embeddings. Ford's published numbers say the hybrid cuts the failure rate by an additional forty percent over embeddings-only⁴. Adding BM25 to embeddings is not nostalgia — it is what the vendor data says works.
• Chunking before you have a question. A folder of plain markdown files is already the index for a person's queries. The chunk is a function of the retrieval architecture, not of the notes.
• Closing the corpus to make the regex easier. Synonyms in the source — vacation / PTO, invoice / receipt — are a one-line write in the note that pays out forever in retrieval. Write the vocabulary you search by.

How this fits with MNMNOTE

The substrate is the point. The file MNMNOTE's editor saves is the file rg searches — a folder of plain markdown a regex can read, and the model you pipe matches to runs under your own key. Steph Ango's "apps are ephemeral, but your files have a chance to last"¹² is the same observation, seen from the retrieval side.

The architectural critique that sits upstream of this decision is the companion argument that the vector database is the wrong abstraction for a personal corpus; this post is its operational consequence. The honest privacy line is the qualified one — your notes stay on your device; the matched lines you choose to pipe to a model are the lines that go to your chosen provider. The retrieval decision and the egress decision are the same decision, made once, in writing. There is no second store to sync, no third party in the loop you did not put there.

Frequently asked questions

These six questions are the verbatim shapes of what people search before they reach for a vector store. The answers are short on purpose — the long forms live above. Each one names the bound, the source, and the moment the upgrade path begins.

Do I need a vector database for my notes? Probably not. For a personal vault under about a million words where you mostly search for terms you remember writing, a regex over the folder plus the model you already use is on the Pareto frontier on cost, latency, and debuggability. Wang et al.'s February 2026 paper measured a grep-driven retriever ahead of vanilla embeddings on a code-completion benchmark at one-fortieth the retrieval latency¹². Use a vector store when the vocabulary gap is wide or the corpus is larger than a single machine.

Is RAG overkill for personal notes? Often, yes — the vendor-default shape is. RAG-the-pattern (retrieval-then-generation) is sensible; embedding-store RAG on a hundred markdown files is the part that is overkill. The hybrid mattnewton named on the DiffMem thread — "pass an annotated list of files to an LLM and ask it which ones it wants to look at" — is "probably better than traditional RAG for small datasets"¹¹. The framing is small datasets, not no retrieval.

Grep vs RAG — which one is faster? Retrieval latency for ripgrep is under two seconds on a thirteen-gigabyte single-file English-text corpus⁶ and under 0.02 seconds on the GrepRAG benchmark². Vector retrieval is typically hundreds of milliseconds plus an embedding API call per query. The model call dominates either total time; the practical difference is the embedding API call you no longer make.

What is the best way to give an AI access to my notes without a vector database? Three small pieces: a rg -n -w 'term' ~/notes | head -50 to find the matches, a saved prompt that says "answer using only these matches; tell me if the matches do not contain the answer," and a habit of opening the cited file at the cited line when the answer disappoints. The DiffMem README documents the agent version of the same loop with grep, git log, git diff, and git blame⁹.

Is RAG dead? No. RAG-the-pattern is fine; the mandatory vector store part is what is no longer the default for small corpora. Anthropic's own published guidance is hybrid — their reported Contextual Embeddings cut the top-20 chunk failure rate by 35% on their corpus, and adding Contextual BM25 on top cut it by 49%⁴. Lexical signals are part of the answer, not the past.

DiffMem vs a vector database — what is the trade-off? DiffMem is a working illustration of the git-as-store version of the small-corpus pattern — markdown files in a repository, an agent reading them with shell commands, no embeddings or vector store⁹¹⁰. The trade-off is the one this post names: it wins on cost, latency, and debuggability where the queries are known-vocabulary and the corpus fits on one machine; it loses to a hybrid pass when the vocabulary gap is wide. The repo is MIT-licensed at 890 stars⁸; the Show HN that introduced it drew 198 points and 45 comments in August 2025⁷.

The first retrieval architecture is the folder. The second is whatever the folder needs that the folder cannot do alone. Most personal vaults never need the second; the ones that do are honest about why — a specific vocabulary gap, a specific corpus size, a specific class of conceptual query — and add only the lever that fixes it. The credit on the small-corpus framing belongs to Wang et al. for the peer-reviewed number¹, to Andrew Gallant for the tool that makes the latency a non-issue³, to Daniel Ford for the honest hybrid caveat⁴, and to alexmrv for shipping the working pattern at the small end⁹.

The substrate is the plain file. The retrieval is whatever the plain file accepts. The same plain file lives at mnmnote.com.