How qmd Searches Your Markdown Entirely On-Device

Reference: tobi/qmd — MIT · TypeScript

qmd is a command-line search engine that indexes your markdown notes, docs, and meeting transcripts, then answers keyword or natural-language queries. It fuses BM25 full-text search, vector semantic search, and an LLM reranker — and every model runs locally on your machine, with nothing sent to a cloud.

It is also, slightly improbably, written by Tobi Lütke, the founder and CEO of Shopify. That pedigree explains some of its 27,485 stars,¹ but the engineering is what makes it worth reading. qmd packages a retrieval pipeline that most teams ship as a hosted service into a single npm install that never leaves your laptop.

What problem does qmd solve?

Most "search your notes with AI" tools ship your documents to a cloud embedding API, store the vectors in a managed database, and call a hosted reranker. That works, but your private notes leave your machine, you pay per token, and you cannot run offline. The retrieval quality is good; the ownership story is not.

qmd takes the opposite stance. Its tagline is blunt: "An on-device search engine for everything you need to remember."² The README is equally direct about where the work happens — the three search techniques are "all running locally via node-llama-cpp with GGUF models."³ Your files stay in their folders, the index is a local SQLite file at ~/.cache/qmd/index.sqlite,⁴ and the embedding, reranking, and query-expansion models are downloaded once and cached on disk.

The trade is deliberate. qmd accepts the cost of running models on your hardware in exchange for keeping a state-of-the-art retrieval pipeline entirely under your control. That is a rare combination, and it is the reason the project describes itself as "tracking current sota approaches while being all local."⁵

How does qmd's hybrid search pipeline work?

The query command — qmd's best-quality mode — runs a multi-stage pipeline. A fine-tuned local model expands the query into variants. Each variant hits two indexes in parallel — a BM25 full-text index and a vector index. The lists are merged with Reciprocal Rank Fusion, trimmed to the top 30, reranked by an LLM, then blended back together.

flowchart TD
    Q[User query] --> EX[LLM query expansion<br/>original + variants]
    EX --> FTS[BM25 / SQLite FTS5]
    EX --> VEC[Vector search / sqlite-vec]
    FTS --> RRF[RRF fusion k=60<br/>+ top-rank bonus]
    VEC --> RRF
    RRF --> TOP[Top 30 candidates]
    TOP --> RR[LLM reranker<br/>qwen3-reranker]
    RR --> BLEND[Position-aware blend<br/>75% / 60% / 40% RRF]
    BLEND --> OUT[Ranked results]

Figure: qmd's query pipeline — the query is expanded into variants, each variant runs BM25 and vector search in parallel, RRF fuses the lists with a top-rank bonus, the top 30 candidates go to the LLM reranker, and a position-aware blend produces the final ranking.

The fusion step is plain Reciprocal Rank Fusion, the standard information-retrieval method for combining ranked lists. qmd's implementation uses the canonical constant k = 60 and sums a reciprocal-rank contribution per list:

export function reciprocalRankFusion(
  resultLists: RankedResult[][],
  weights: number[] = [],
  k: number = 60
): RankedResult[] {
  // ...
  const rrfContribution = weight / (k + rank + 1);
  // ...
}

Nothing exotic so far. The interesting decisions come in how qmd refuses to trust any single signal completely.

What stops the reranker from ruining good results?

Rerankers are powerful but overconfident — they will happily demote an obviously perfect exact match because the surrounding semantics looked weak. qmd guards against this with two mechanisms built into the fusion code. First, documents that rank #1 in any list get a small score bonus; documents at #2 and #3 get a smaller one:

if (entry.topRank === 0) {
  entry.rrfScore += 0.05;
} else if (entry.topRank <= 2) {
  entry.rrfScore += 0.02;
}

Second, the final score is a position-aware blend: the higher a document already ranks after fusion, the more qmd trusts retrieval over the reranker. The thresholds are hardcoded — top 3 results keep 75% of their retrieval score, ranks 4 to 10 keep 60%, and everything below 10 leans on the reranker at 40%.⁶

The README states the intent plainly: "Position-aware blending prevents the reranker from destroying high-confidence retrieval results."⁷ It is a small, opinionated piece of glue, and it is the kind of detail a hosted black box rarely exposes.

How does qmd chunk documents for embedding?

Before any vector search can happen, documents have to be split into chunks. Naive chunkers cut at a fixed token count and slice through headings, code blocks, and sentences. qmd instead scores candidate break points and cuts at the best one near its target size.

Each break point gets a base score — an H1 heading scores 100, an H2 scores 90, a code-fence boundary 80, a blank line 20, a list item 5. As the chunker approaches its 900-token target (with 15% overlap), it searches a 200-token window and applies a squared-distance decay so a strong-but-distant heading still beats a weak-but-near line break. Code fences are protected so snippets stay whole.

For source files, qmd can additionally use tree-sitter to add AST-derived break points at function, class, and import boundaries — enabled with --chunk-strategy auto for TypeScript, JavaScript, Python, Go, and Rust.

The chunk size is a single constant, not a magic number buried in a service:

export const CHUNK_SIZE_TOKENS = 900;
export const CHUNK_OVERLAP_TOKENS = Math.floor(CHUNK_SIZE_TOKENS * 0.15);  // 135 tokens (15% overlap)

What is the context tree, qmd's headline feature?

qmd lets you attach a description to a collection or sub-path; that text is returned alongside any matching document beneath it. Tag qmd://meetings as "Meeting transcripts and notes" and every hit arrives pre-labelled. The README calls it "the key feature of QMD as it allows LLMs to make much better contextual choices when selecting documents."⁸

The idea maps cleanly onto how agents actually consume search results. A raw filename tells an LLM little; a path plus a human-written description of what lives there tells it whether the hit is worth opening. It is a cheap, file-tree-shaped way to inject the metadata that makes plain markdown a usable retrieval target — the same reason markdown notes work so well as durable AI memory.

What would I steal from qmd?

Three ideas travel well beyond this repo. The position-aware blend is a reusable rule: never let a downstream reranker fully override a confident upstream signal. The score-based chunker beats fixed-window splitting on markdown. And the agent-first I/O surface — --json output plus an MCP server — makes a CLI a first-class tool for an LLM.

The SDK design is quietly disciplined too. qmd refuses to assume a default index path: "The SDK requires explicit dbPath — no defaults are assumed. This makes it safe to embed in any application without side effects."⁹ That is the same own-your-data instinct that runs through the whole project, and it rhymes with the broader case for keeping retrieval local and readable.

Verdict

qmd is worth learning from whether or not you adopt it. It is a clean, MIT-licensed demonstration that a modern hybrid-retrieval stack — query expansion, BM25, vectors, RRF, and a reranker — fits in a 16,000-line TypeScript codebase that runs on your machine.¹⁰ If you index docs for an agent and care where data sleeps, it gives you quality and ownership.

Frequently Asked Questions

What is qmd?

qmd is a command-line search engine for your local markdown notes, documentation, and meeting transcripts. It combines BM25 keyword search, vector semantic search, and an LLM reranker, exposing the results through a CLI, an MCP server, and a TypeScript SDK. It is MIT-licensed and written by Tobi Lütke.

Is qmd really made by the Shopify CEO?

Yes. The repository lives at tobi/qmd and its MIT license reads "Copyright (c) 2024-2026 Tobi Lutke," the founder and CEO of Shopify. The fine-tuned query-expansion model is published under his tobil HuggingFace namespace, and the npm package is @tobilu/qmd.

Does qmd send my notes to the cloud?

No. qmd is explicitly on-device. Its embedding, reranking, and query-expansion models all run locally through node-llama-cpp using GGUF files cached on disk, and the index is a local SQLite database. Your documents stay in their folders on your machine; no cloud API is required to search.

What models does qmd use?

By default qmd downloads three local GGUF models: embeddinggemma-300M for embeddings, qwen3-reranker-0.6b for reranking, and a fine-tuned qmd-query-expansion-1.7B model for expanding queries. They are fetched from HuggingFace once and cached under ~/.cache/qmd/models/. You can swap the embedding model via an environment variable.

How does qmd's hybrid search work?

The query command expands your query, runs BM25 and vector search in parallel, fuses the lists with Reciprocal Rank Fusion (k=60), keeps the top 30, reranks them with an LLM, and blends scores position-aware so confident retrieval matches are protected. The simpler search and vsearch commands run BM25 or vectors alone.

Can qmd index code, not just markdown?

Yes. With --chunk-strategy auto, qmd uses tree-sitter to chunk source files at function, class, and import boundaries for TypeScript, JavaScript, Python, Go, and Rust. Markdown and other file types fall back to its score-based regex chunker. Tree-sitter grammars are optional dependencies that degrade gracefully if missing.

How do I use qmd with an AI agent?

qmd ships an MCP server (qmd mcp) exposing query, get, multi_get, and status tools, plus a Claude Code plugin. For looser integration, its --json and --files output formats are built for agentic workflows, letting an agent call the CLI directly and parse structured results.

Is qmd free and open source?

Yes. qmd is released under the MIT license, the most permissive common open-source license, allowing commercial use, modification, and redistribution. As of 2026-07-06 the repository has 27,485 stars and 1,722 forks, and is installable from npm as @tobilu/qmd.

Search you can audit beats search you have to trust.

If you want notes that stay yours and stay searchable, mnmnote.com keeps every word local and in plain markdown.