Own the Vector Index Next to Your Files

If you build semantic search over your own notes, keep the resulting index as a file on your own disk, next to the notes, not inside a vendor. The index is derived data. Your notes are the source of truth; the embeddings are a convenience built from them. Owning the file means you can move, version, or rebuild it.

The usual advice, "embed your notes," quietly assumes a managed vector database. You sign up, you push your vectors, you query an endpoint. It works, and for a while it feels like progress. Then the index is no longer a thing you have. It is a tenant row in a system you rent, governed by someone else's export policy and uptime. The mechanism that searches your private writing now lives somewhere you do not control.

This is the branch for people who genuinely want semantic search and want to keep it theirs. The companion argument, that you can build AI memory from plain Markdown with no vectors at all, lives in Markdown notes as AI memory. Read that first if you are not sure you need embeddings. If you have decided you do, the question shifts: not whether to embed, but where the index should live.

The index is derived, not the source

An embedding index stores vectors computed from your text. It is downstream of your notes, not a copy of them and not a replacement. Meta's FAISS library calls itself "a library for efficient similarity search and clustering of dense vectors,"¹ built "around an index type that stores a set of vectors."² It stores numbers. The words stay in your files.

That single fact changes what ownership means here. Lose the index and you lose a search shortcut, not your data. The notes still sit on disk in a format you can read. You can recompute the vectors whenever you want, because the inputs never left.

So the question to ask of any embedding setup is not how clever the search is, but how easily the index leaves the tool. If it leaves as a file, you own it. If it can only be queried through an endpoint and exported on the vendor's terms, you are renting the one artifact that maps your private notes to each other.

A vector index is a file you can write to disk

In the libraries most people actually use, the index already is a file. There is nothing exotic to build. FAISS spells it out: write_index(index, "large.index") "writes the given index to file large.index," and read_index("large.index") "reads a file."³ The Hugging Face Datasets docs wrap the same step in save_faiss_index and load_faiss_index.⁴ Write, then read. That is the whole portability story.

You can go even more compact. The sqlite-vec extension, written by Alex Garcia and backed by Mozilla's Builders program, is "an extremely small, 'fast enough' vector search SQLite extension that runs anywhere!" — "written in pure C, no dependencies, runs anywhere SQLite runs."⁵ Your entire vector index becomes one SQLite file sitting beside your notes. No server, no account, no tenant row. Just a file you can copy to a backup drive or check into a folder.

This is not a fringe corner of the ecosystem. FAISS carries tens of thousands of GitHub stars, sqlite-vec thousands more, and llama.cpp — the project that anchors much of local inference — well over a hundred thousand.⁶ Owning the artifact is the mainstream path, not the contrarian one. The tools to do it are mature and widely used.

Pin the model, not just the index

Owning the index is not the same as owning the model's behavior. Embeddings are only comparable if they came from the same model. Change the model and the numbers shift, so a rebuild against a different version can return different neighbors. The honest move is to keep the model as a portable artifact too, alongside the index.

Model weights travel as files. Hugging Face's safetensors format is "a new simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy),"⁷ a portable, inspectable file for the model that builds your vectors. The GGUF format, introduced by the llama.cpp project on 21 August 2023, goes further toward self-containment: "a single file holds the tensors, the tokenizer and all metadata needed to load and run the model, eliminating the need for accompanying configuration files."⁸

Pin the exact model file, record which one you used, and your index becomes reproducible. Lose the index and you can rebuild an identical one. Skip this step and a rebuild is a guess. The model artifact is the second half of the thing worth owning, and it is the half most people forget.

Losing the index costs compute, not data

If you delete your vector index, you do not lose anything irreplaceable. You lose the time it takes to rebuild. The embeddings were derived from notes you still have, using a model you can still run. Rebuilding reads the notes, runs the model, writes a new index file. The cost is compute, measured in minutes, not lost writing.

Local embedding models are small enough to make this routine. Ollama lists nomic-embed-text at 137M parameters, all-minilm at 23M, and mxbai-embed-large at 334M, with the workflow being to "first pull a model."⁹ These run on a laptop. A rebuild over a personal vault is an errand, not a migration.

Which is why you can treat the index file the way you treat any cache: keep it for speed, version it if you like, or .gitignore it knowing it regenerates from source. The freedom to delete without fear is the clearest sign you own the right thing. You are protecting the notes; the index is downstream of them.

When a managed database is the right call

Owning the index next to your files is the durable move for a personal vault. But managed vector databases genuinely win at scale, team, and high-traffic production. If you run embeddings behind a product with real load and automatic scaling needs, a hosted system earns its keep. This argument is scoped to your own notes, on your own machine.

The distinction is purpose, not loyalty to a tool. A hosted vector database is the right abstraction for a production service; a file on your disk is the right abstraction for a private knowledge base. The villain here is never a vendor — it is a shape: an index you cannot export, holding the map of your private notes, in a place you do not control. That shape is wrong for personal use regardless of which logo is on it.

For the personal case, the local route is genuinely private, not merely a fallback. Mozilla, in sponsoring sqlite-vec, framed the goal as "a chatbot that can answer questions about your personal data without letting a single byte of that data leave the privacy and safety of your laptop."¹⁰ Not a single byte leaving — that is the privacy property a file on your own disk gives you by default, and a remote endpoint cannot.

Files outlast tools

Keep the index as a file for the reason you keep anything as a file: it can outlast the tool that made it. Steph Ango, who runs Obsidian, put it plainly: "Apps are ephemeral, but your files have a chance to last."¹¹ An index inside a managed service inherits that service's lifespan. An index in a file inherits your disk's.

Ango's fuller statement is the design rule: "if you want to create digital artifacts that last, they must be files you can control, in formats that are easy to retrieve and read."¹² A vector index is a digital artifact. So is the model that built it. Both can be files you control, and the libraries above make them so without any heroics on your part.

This is the local-first principle applied to the AI layer. The Ink & Switch essay that named local-first software argued, in its own subtitle, that "You own your data, in spite of the cloud,"¹³ and held that a user should retain "ultimate ownership and control"¹⁴ over it.

Your notes already qualify when they are open Markdown stored locally on your own device. The derived index should not be the one piece that escapes that ownership. Kept as a file beside the notes, with the model pinned, it stays inside the same boundary.

There are limits worth stating plainly. Semantic search over your notes is a convenience layer, not a system of record. If your notes are clinical or legal, do not rely on it for anything that demands authority; it offers no guarantees about sensitive data and no legal or medical advice.

And "runs locally" is not "runs on anything": the small models above fit a laptop, but local embedding still asks for capable hardware. Own the artifact, and own its boundaries too.

Frequently asked questions

Where should I store my embeddings for a personal RAG?

Store them as a file on your own disk, next to your notes. A FAISS index writes to a single .index file you control,³ or your whole index can live in one SQLite file via the sqlite-vec extension, which "runs anywhere SQLite runs."⁵ Both keep the index portable and under your control, rather than as a row in a hosted database.

Can I save a FAISS index to disk and load it later?

Yes — that is the built-in behavior. In FAISS, write_index(index, "large.index") "writes the given index to file," and read_index "reads a file" back.³ At a higher level, Hugging Face Datasets offers save_faiss_index(...) to save and load_faiss_index(...) to "reload it later."⁴ Saving and reloading the index is a one-line operation in both.

If I delete my vector index, do I lose my data?

No. The index is derived data — FAISS "stores a set of vectors" computed from your text,² not the text itself. Your notes remain on disk. Deleting the index costs only the compute to rebuild it, which over a personal vault using small local models is minutes, not a data-loss event. The notes are the source of truth.

Do I need a managed vector database for a personal knowledge base?

For a personal vault, no — a FAISS index file or a single sqlite-vec SQLite file on your own disk handles semantic search without a hosted service. Managed databases genuinely win when you need production scale, high traffic, automatic scaling, or a team. Match the abstraction to the job: a file for personal notes, a hosted system for a product under load.

What is the difference between safetensors, GGUF, and pickle?

Safetensors is "a new simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy)."⁷ GGUF, introduced by llama.cpp on 21 August 2023, is self-contained: "a single file holds the tensors, the tokenizer and all metadata needed to load and run the model."⁸ For a portable, inspectable embedding model, both beat pickle.

How do I keep my embeddings portable and avoid vendor lock-in?

Keep three things as files you own: the index (a FAISS .index or one sqlite-vec SQLite file),³⁵ the model that built it (safetensors or GGUF),⁷⁸ and the notes themselves as open Markdown. Pin the exact model so the index is reproducible, version or .gitignore the index knowing it rebuilds, and nothing about your search depends on a vendor staying online.

Can I run embeddings locally without sending my notes to the cloud?

Yes. Local embedding models are small — Ollama lists options from 23M to 334M parameters that you "first pull"⁹ and run on a laptop. Mozilla described the goal of local vector tooling as answering questions about your data "without letting a single byte of that data leave the privacy and safety of your laptop."¹⁰ The notes never leave your machine.

Embeddings are worth building when plain search stops finding what you mean. But the index they produce is the most revealing artifact in the whole setup — it maps your private notes to one another — and it is the one most easily handed to a vendor by default. Keep it as a file, pin the model that made it, and treat it as rebuildable. Then the clever part of your search is yours, the same way the writing is.

If you also want the retrieval layer to be one you can inspect rather than trust, the case for a personal RAG you can actually audit follows directly from owning the index. And the broader move — running private AI on your own notes, keeping a second brain that is not just a RAG, owning the derived structure of a GraphRAG — is the same instinct the local-first ownership case makes for your files at large.

The notes are the source of truth; the index is a convenience built from them. To keep your notes as open Markdown on your own device — the source every index is rebuilt from — write them in mnmnote.com.