Your Frontmatter Is Content to the AI

When you paste a Markdown note into an LLM, the --- block at the top is not invisible. It is billed as tokens, parsed as Markdown, and can carry IDs you never meant to send. Before each paste, sort that header into one of three fates: signal, noise, or leak.

This post assumes you already know what a frontmatter block is — the ----delimited YAML header that tools like Jekyll, Hugo, and most note apps read for a note's title, tags, and dates. Jekyll's docs are typical: the front matter "must be the first thing in the file and must take the form of valid YAML set between triple-dashed lines."¹ If that is new, what a YAML frontmatter header is and how to add one is the prerequisite.

This post is about the other reader of that block: the model. Tokens are the unit AI vendors bill and budget against, and token cost is the new page count for your notes prices the whole paste. This one is about a single block inside it — one that is denser, more parseable, and more revealing than the prose it sits above.

The three fates: signal, noise, or leak

Your frontmatter is content to the AI, so the only useful question is what kind of content. There are three answers, and each has a different action. Decide before you paste. Ask whether a retriever filters on the header; whether it holds identifiers you would not paste on their own; and, if neither, whether it is just cost.

The decision is a short walk down two questions:

flowchart TD
  A["The header block<br/>at the top"] --> B{"Does a retriever<br/>filter on it?"}
  B -->|Yes| C["SIGNAL<br/>make it structured<br/>metadata"]
  B -->|No| D{"IDs, paths, or<br/>aliases inside?"}
  D -->|Yes| E["LEAK<br/>scrub before<br/>you paste"]
  D -->|No| F["NOISE<br/>strip it to<br/>save tokens"]

Figure: the three-fates decision. If a retrieval system filters on the header's fields, it is signal — convert it to structured metadata. If not, check whether it holds IDs, file paths, or private aliases; if it does, it is a leak, so scrub it. If neither, it is noise — strip it to save tokens.

Most casual pastes land on noise. A personal retrieval pipeline turns it into signal. Any header carrying internal identifiers is a leak until you check it. The rest of this post takes each fate in turn and gives you the one-line command to act on it.

Noise: what the block costs you in tokens

For a casual paste into a chat window, the header is pure cost — it changes nothing about the answer and inflates the bill. And it costs more at the margin than you would guess. A realistic ten-field YAML header, measured on OpenAI's current o200k_base tokenizer, ran 80 tokens — more than double the body of a short note.²

That is the number worth sitting with. In this example — a metadata-heavy header above a three-sentence note — the frontmatter was 71% of the note's tokens (80 of 113).² The tail wagged the dog: the part you wrote to help humans skim outweighed the part you actually wanted the model to read.

Structured YAML is also denser than prose. The rule of thumb for English text is roughly four characters per token; this header packed 2.95 characters per token, because keys, brackets, dates, and IDs fragment into more pieces than running sentences do.² A header is small on screen and large in the tokenizer.

Sample size one, of course. Your ratio depends on how many fields you keep and how long your note runs. But the direction is stable: a fixed header is a bigger share of a short paste, and short pastes are most of what people send. When the header is noise, the move is to remove it, and the payoff is measurable — stripping this one saved all 80 of its tokens, 71% of the paste.²

The hazard: your closing --- can look like a heading

The most surprising fate is not cost — it is misparsing. A capable chat model reads YAML fine. But the deterministic Markdown splitters inside a retrieval pipeline are not YAML-aware, and to a plain CommonMark parser the frontmatter fences are structural punctuation, not metadata.

The same three dashes mean two different things to two readers. To YAML, --- is a boundary: the spec says it "uses three dashes ("---") to separate directives from document content," which "also serves to signal the start of a document."³ To a Markdown parser that has never heard of YAML, those dashes are not a boundary — they are punctuation.

Read the CommonMark spec literally. It says a line of "three or more matching -, _, or * characters" forms a thematic break.⁴ So your opening ---, sitting at the very top with nothing above it, is a horizontal rule. That alone is harmless. The closing fence is where it bites.

CommonMark also says: "The heading is a level 1 heading if = characters are used in the setext heading underline, and a level 2 heading if - characters are used."⁴ A line of dashes directly under a line of text is not a thematic break — it is a setext underline. So a header that ends like this:

---
title: Q3 Planning
cssclass: wide-table
---

is read by a YAML-blind parser as: a thematic break, then a paragraph (title: Q3 Planning / cssclass: wide-table), then a --- that turns that paragraph into a level-2 heading. The block you meant as metadata becomes a phantom H2 titled title: Q3 Planning cssclass: wide-table.

This is exactly what a heading-aware chunker splits on. LangChain's MarkdownHeaderTextSplitter cuts a document at its headings — and it will happily cut at the phantom one, stranding your first real paragraph under a garbage heading. One developer opened a 2023 LangChain issue on precisely this class of problem, warning that with small-context embedding models "having only mostly 512 sequence length," a lot of "context is lost if correct metadata is not present in each chunk."⁵

The --- block is not auto-separated; the splitter needs to be told about it. This hazard lives in parsers and splitters, not in the chat model's comprehension — scope your worry accordingly.

Signal: when the header earns its tokens

There is one case where the header is the most valuable text in the file: a retrieval system that filters on it. If you built a personal RAG, fields like project, status, or date are not noise — they are the index. But they only work as structured metadata, not as raw YAML at the top of a chunk.

That distinction is the whole game, and the tools are explicit about it. RAGFlow's documentation describes metadata as a separate field that rides along with the retrieved text, noting that "such information will be sent to the LLM with the retrieved chunks for content generation."⁶ Its auto-metadata guide is blunter still: "If a text chunk is retrieved, its associated metadata is also passed to the LLM, providing richer contextual information about the source document to aid answer generation."⁷

The direction of travel is to extract the header into fields, not to leave it inline. RAGFlow shipped an Auto-metadata feature in v0.23.0 that "uses large language models to automatically extract and generate metadata for files."⁷

The reason inline YAML fails is the chunk budget. An embedding model like all-MiniLM-L6-v2 truncates anything past a fixed length — its card states plainly that "by default, input text longer than 256 word pieces is truncated."⁸ An 80-token header eats a third of that window before your actual sentence arrives.

So when your header is signal, do not paste it as text. Lift each field into your pipeline's metadata store, where a retriever can match on it and pass it alongside the chunk. Same information, different shape — one that a filter can read. If you want a home for the fields that should never sit inside the note at all, sidecar files are how you add metadata you own.

The five-minute version

Here is the whole method, start to finish, for a single paste. It takes about a minute per note and turns "should I strip this?" into a reflex: name the fate, then run the one command that fate calls for. Five steps, and only step one needs your judgment — the rest is mechanical.

1. Look at the header and pick the fate. Retriever filters on it? Signal. Holds IDs, file paths, or private aliases? Leak. Neither? Noise.
2. For noise, strip it with one command. This sed line deletes a leading --- block and leaves everything else untouched:

sed '1{/^---$/!q};1,/^---$/d' note.md > clean.md

3. Confirm it did nothing dangerous. Run it on a file without frontmatter — it passes the file through unchanged, because the 1{/^---$/!q} guard quits the moment line one is not ---. An awk equivalent works the same way if you prefer it:

awk 'NR==1 && $0=="---"{f=1;next} f==1 && $0=="---"{f=2;next} f==2||f==0' note.md

4. For a leak, scrub the specific keys, do not just trust the strip. Delete or blank the id, aliases, path, and any related links by hand before the text leaves your machine. Stripping the whole block covers the common case; a targeted scrub covers the fields you know are sensitive. Choosing what never belongs in a paste is its own habit — decide what not to feed the AI works through it.
5. For signal, extract instead of strip. Move the fields into your retrieval store's metadata so a filter can match on them, then embed the body alone.

One decision, one command, one honest check. That is the system.

Common mistakes

Every one of these is a fate misread. The header gets treated as invisible when it is billed, as disposable when it is the index, or as private when it is merely tidied. Four of the five below cost you tokens or retrieval quality; the third costs you a false sense of safety.

• Assuming the model ignores the header. It does not. It reads every token, bills every token, and — in a splitter — parses every dash. Invisible-to-you is not invisible-to-it.
• Stripping when you meant to extract. If you run a personal RAG, deleting the header throws away your index. Extract those fields into metadata first; strip only what a retriever will never filter on.
• Trusting a blanket strip as privacy. Removing the block reduces incidental exposure of IDs and paths. It is prompt hygiene, not a security control — not encryption, and no guarantee. Anything you then send to a third-party model still leaves your device.
• Forgetting the closing fence is the dangerous one. The opening --- is a harmless rule; the closing --- under text is a setext heading. A hand-rolled chunker that splits on --- alone will miss the second one and mangle the split.
• Over-stuffing the header to "help" the AI. A ten-field block does not make a chat model smarter; it makes the paste heavier and the parse riskier. Willison's note on Claude Skills is the counter-model — good metadata design is "very token efficient: each skill only takes up a few dozen extra tokens, with the full details only loaded in should the user request a task."⁹ Load detail on demand, not up front.

How this works in MNMNOTE

In MNMNOTE, a note is a plain Markdown file on your own device, and the frontmatter is plain text you own outright. Nothing in the header is sent anywhere by default. It sits in your file, under your control, until you choose to do something with it.

When you copy a note into an AI tool, you decide what goes: keep the header when it is signal, strip it when it is noise, scrub it when it is a leak. AI features you opt into send the text you choose to your chosen provider — so the choice of what to include is yours to make, note by note. The header follows the same rule as everything else here: it is content, and it is yours.

Frequently asked questions

Short answers to the exact questions people tend to type when a .md file meets a language model. Each one maps to one of the three fates above — cost, parsing, or exposure — and each carries a single one-line action behind it.

Should I strip YAML frontmatter before feeding a Markdown note to an LLM? It depends on the fate. Strip it if it is only cost and noise. Keep and extract it as structured metadata if a retriever needs to filter on it. Scrub it if it holds IDs, paths, or private aliases. For a casual chat paste, stripping is usually right.

Does the YAML frontmatter count as tokens when I paste a note into ChatGPT? Yes. Every character in the header is tokenized and billed. In one measured example, a ten-field header ran 80 tokens on OpenAI's o200k_base encoding — 71% of a short note's total.² Structured YAML is denser than prose, near 2.95 characters per token against the ~4 for English text.

How do I remove Markdown frontmatter programmatically? Use a one-liner that deletes a leading --- block only: sed '1{/^---$/!q};1,/^---$/d' note.md. It strips the header when the file starts with --- and passes any other file through unchanged. An awk version does the same. Test it on a header-less file to confirm it is non-destructive.

Will an LLM misread my frontmatter as a heading? A capable chat model will not; a deterministic Markdown splitter can. Under CommonMark, a --- directly beneath a line of text is a level-2 setext heading, not a thematic break.⁴ A heading-aware chunker can turn your closing fence into a phantom H2 and split there. Scope the concern to RAG splitters, not chat comprehension.

Should frontmatter live in the note or in a separate file? If the metadata is sensitive or exists mainly for tooling, a sidecar file keeps it out of the text you paste while staying yours to query. Fields a retriever filters on can live in a metadata store; fields you never want to send should not sit inside the note at all.

Does stripping frontmatter protect my private data? Only in a narrow, honest sense. It removes incidental identifiers before you paste, which lowers accidental exposure. It is prompt hygiene, not a guarantee — not encryption, and it does nothing about the note body you do send. Treat it as tidying, never as a security control.

Can I set metadata for multiple documents at once? In a retrieval tool like RAGFlow, yes — metadata is managed as structured fields on documents, not as raw YAML inside each chunk, and "its associated metadata is also passed to the LLM" alongside the retrieved text.⁷ That is the signal fate at scale: set the fields once, let the filter and the model share them.

Your frontmatter was written for a human skimming a folder. It is read, now, by a machine that bills every token and parses every dash — so decide its fate on purpose, one paste at a time.

Notes kept as plain Markdown on your own device let you see the header, count it, and strip or keep it on your terms — which is how MNMNOTE treats every note you write.