Tagging Notes Doesn't Need a Frontier Model

The work of organizing your notes — tagging, routing, classifying, pulling a date or a name out of a paragraph — is not thinking. It is sorting. Sorting is constant, narrow, and repetitive, and a small language model on your own device handles it well. A frontier model is the wrong tool.

That is a claim about fit, not a claim about capability. Frontier models are extraordinary at open-ended reasoning, and nothing here disputes that. But most of what an app does over a pile of notes is small, mechanical, and endless: this note is a recipe, that one is a meeting, this line holds a phone number, that paragraph belongs under a project. In June 2025 a group of NVIDIA researchers argued the same thing about AI agents in general, that "the majority of agentic subtasks in deployed agentic systems are repetitive, scoped, and non-conversational" ¹. A vault of notes is the household version of that observation. The moment you separate the rare act of thinking from the constant act of sorting, the hardware question answers itself.

What people think reading their notes takes

The common assumption is that "AI for your notes" means the biggest model you can reach. Understanding your own writing takes the same engine that writes essays and debugs code, or so the thinking goes. It is a reasonable instinct. The best-known models are the biggest ones, and they are genuinely good.

The instinct fails on economics and location. A frontier model lives in a data center, bills per token, and needs a network round-trip for every request. That is acceptable when you ask a hard question once. It is absurd as the thing that watches your vault and quietly files each new note. The NVIDIA group put the general case plainly: small language models are "sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations" ¹. Reserve the expensive engine for the question that actually deserves it.

Thinking about your notes is rare. Sorting them is constant.

Draw one line and everything downstream changes. Thinking about your notes — "synthesize my last year of research into an argument" — is rare, hard, and worth a big model. Sorting your notes — tag this, route that, classify the other — happens on every save, forever. The first is a query. The second is a background job.

Almost all of the AI work a note app does falls on the sorting side of that line, and sorting has a specific shape. It is classification (what kind of note is this?), routing (which folder or project?), extraction (what dates, names, tasks live inside?), and tagging (what is this about?).

None of it is conversation. None of it needs world knowledge. It needs to be fast, private, and cheap enough to run thousands of times without anyone noticing, which is exactly the profile of a small model, not a large one. Retrieval (actually finding a note later) is its own separate question, and a different discussion.

Why small models are the right size for sorting

"Small" now has a working definition. The NVIDIA paper is comfortable "considering most models below 10bn parameters in size to be SLMs" as of 2025, meaning small enough to run on a consumer device with usable latency for one person ¹. That is a device-shaped definition, not a leaderboard-shaped one.

The economics follow from the size. The same authors estimate that serving a 7-billion-parameter model is "10–30× cheaper (in latency, energy consumption, and FLOPs) than a 70–175bn" model ¹. Cost that low is what turns a one-off request into a standing background job.

When each classification is nearly free and runs in milliseconds on hardware you already own, you can afford to re-tag the whole vault on a whim, or check every note the instant it is written.

The point of a small model is not that it is a weaker large model. The point is that it is small enough to run constantly, on your machine, which is the only way a sorting job on a personal vault ever makes sense.

None of this makes a small model a substitute for a large one. Ask it to weigh two arguments across fifty notes, to infer an intent you never wrote down, or to answer a question that reaches past your vault, and it will strain or invent. Those are reasoning tasks, and reasoning is where a big model earns its cost. The trick is not to pick a side. It is to notice which kind of task you are doing, sorting or thinking, and send it to the model built for that job. Most of what happens to a vault is sorting, which is why most of the work should stay small and close.

What a tiny model actually does over a vault

The models built for exactly this shipped in 2025, and they are startlingly small. Google's Gemma 3 270M (270 million parameters) is aimed squarely at "sentiment analysis, entity extraction, query routing" and "text classification and data extraction" ². Sorting is precisely the job it was designed for.

In Google's own test, an INT4-quantized copy "used just 0.75% of the battery for 25 conversations" on a Pixel 9 Pro ². A model that sips a fraction of a percent of a phone battery can run over your notes all day and never be felt.

Classification and tagging also lean on embedders, models that turn text into vectors you can group, dedupe, and label. Google's EmbeddingGemma is 308 million parameters, runs "on less than 200MB of RAM with quantization," and returns a vector in "<15ms" for a short passage on an edge chip ³.

It handles 100-plus languages and, as Google reported at its September 2025 launch, ranked as the highest open multilingual embedding model under 500M on the Massive Text Embedding Benchmark ³. Crucially for a notes vault, it "generates embeddings of documents directly on the device's hardware, helping ensure sensitive user data is secure" ³.

Apple ships the same idea at the OS level: an "approximately 3-billion-parameter" on-device model tuned for "entity extraction" and "text understanding" — and explicit that it "is not designed to be a chatbot for general world knowledge" ⁴. That last line is the whole thesis stated by the people who built the model.

How to put a small model to work on your notes

You do not need to train anything, and you do not need a lab. Putting a small model to work on your notes is mostly a matter of matching the tool to the task and keeping the sorting local, where the notes already live. The practical moves are few, and each one is a decision, not a project.

1. Split the jobs. Send open-ended reasoning to whatever big model you like; keep tagging, routing, classification, and extraction on a small local model.
2. Prefer on-device for the constant work. A background job that touches every note is where privacy and cost compound. Run it where the notes already live.
3. Use an embedder for grouping and tagging, and a tiny instruct model for "what kind of note is this" — they are different tools for different halves of sorting.
4. Keep notes as plain files. A small model can only sort what it can read; open Markdown on your own device is the most portable input a local model can get. If you are weighing tools, the ones that work fully offline are the ones a local model can pair with.
5. Stay assisted, not automated. Let the model propose tags and let yourself confirm them — the line between AI-assisted and AI-controlled is where trust in your own vault is kept or lost.

Frequently asked questions

Do I need a frontier model to organize my notes? No. Organizing notes is classification, routing, extraction, and tagging: narrow, repetitive tasks that a small language model handles well. NVIDIA's 2025 analysis argues small models are "more suitable" for exactly this kind of scoped, non-conversational work ¹. Save the frontier model for genuine reasoning across your whole vault.

What counts as a "small" language model in 2026? A common working line is under about 10 billion parameters — small enough to run on a consumer device with usable latency for one person ¹. In practice the models built for sorting are far smaller: Gemma 3 270M is 270 million parameters ², and the EmbeddingGemma embedder is 308 million ³.

Can a small model run entirely on my own device? Yes. EmbeddingGemma runs in under 200MB of RAM and generates embeddings "directly on the device's hardware" ³. Gemma 3 270M used 0.75% of a Pixel 9 Pro's battery across 25 runs ². Apple ships a roughly 3-billion-parameter model on-device for extraction and text understanding ⁴.

Is a small model accurate enough for tagging and extraction? For scoped tasks, yes. Gemma 3 270M is built to "execute tasks like text classification and data extraction with remarkable accuracy" ², and Apple's on-device model targets entity extraction directly ⁴. Accuracy on narrow, well-defined tasks is different from open-ended reasoning — sorting is the narrow kind.

Why not just use one big model for everything? Because a big model is a poor fit for a constant background job: it bills per token, needs a network round-trip, and sends your notes off-device. A 7B model is "10–30× cheaper" to serve than a 70–175B one ¹. The savings are what let sorting run continuously rather than occasionally.

Does using AI on my notes mean my notes leave my device? It depends on the path. On-device models keep the text local by design ^{3 4}. Cloud models, including bring-your-own-key setups, send the text you choose to your chosen provider, so "your notes never leave your device" is only true on the local path. Keep the constant sorting local, and you keep the exposure small.

What is the difference between sorting my notes and searching them? Sorting is classifying, routing, extracting, and tagging a note as it arrives. Searching is finding a note later. They lean on related tools but they are different jobs with different trade-offs — this piece is about the sorting side.

Files outlast the tools that made them; in Steph Ango's words, "In the fullness of time, the files you create are more important than the tools you use to create them" ⁵.

A model small enough to live on your own machine, reading notes you keep as plain files, is the version of AI that respects that order — it treats the copy of your data on your own device as the primary one, the way local-first software was always meant to ⁶. The vault stays yours, and the sorting stays quiet.

When your notes are open Markdown on your own device, as they are in MNMNOTE, the smallest, closest model is the one that can read them.