Two Tools Both Ignore Case. They Disagree About Which Letters.

Two search tools both told to ignore case will disagree, because case-insensitivity is not one algorithm. Each program consults a case table it chose independently: your locale, a compiled-in Unicode version, twenty-six ASCII letters, or a copy of Unicode frozen in 2002. One binary, one flag, one file — and the answer flips.

The demonstration takes one minute and two files. Write the tag #Ꭰone into one Markdown file and #ꭰone into another. The first is spelled with U+13A0 CHEROKEE LETTER A; the second with U+AB70 CHEROKEE SMALL LETTER A, which has been its lowercase counterpart since Unicode 8.0 ¹. Then ask four tools a plain-text note stack already contains — GNU grep, ripgrep, a Python interpreter, SQLite — whether the two tags are the same string.

They return three different answers, and one of them returns two answers by itself. This is not a bug report. Every tool here behaves exactly as its documentation says it will, and Unicode's stability policy has been keeping its own promise the whole time. The disagreement comes from somewhere quieter: each program shipped its own case table, from its own version of the standard, and every one of them calls the result the same thing.

Four tools, one pair of tags, three answers

Two Markdown files, each holding one tag. One contains #Ꭰone, spelled with U+13A0 CHEROKEE LETTER A. The other contains #ꭰone, spelled with U+AB70 CHEROKEE SMALL LETTER A. Every tool below was asked, in its own idiom, whether those two tags are the same string. They do not agree.

Run it yourself. Every line below was executed on one machine on 19 August 2026 — with each tool's version printed before its result.

printf '#Ꭰone meeting notes\n' > a.md
printf '#ꭰone meeting notes\n' > b.md

LC_ALL=C.UTF-8 /bin/grep -i -c $'#Ꭰone' b.md   # -> 1
LC_ALL=C       /bin/grep -i -c $'#Ꭰone' b.md   # -> 0
LC_ALL=POSIX   /bin/grep -i -c $'#Ꭰone' b.md   # -> 0

rg -i -c $'#Ꭰone' b.md                          # -> 1, under every locale

The headline is the first three rows. One binary, one flag, one file, and the answer flips on an environment variable. Nothing about the tool changed — only which table it was told to read.

The fourth row is the second finding. ripgrep returns the same answer under all three locales, which means it is not consulting the locale at all. Two search tools sitting in the same directory, both invoked with -i, are reading two different sources of truth.

A methods note, because the first version of this table was wrong. In the shell where it was first run, grep was a shell function wrapping ugrep 7.5.0, and the finding that fell out — "grep and ripgrep disagree" — was an artifact of the wrapper, not a property of GNU grep.

Absolute paths and printed versions are the only defensible form. The locale result survives that correction untouched, because it pits one binary against itself.

GNU grep's own manual says as much, in the entry for the flag: "Ignore case distinctions in patterns and input data, so that characters that differ only in case match each other. Although this is straightforward when letters differ in case only via lowercase-uppercase pairs, the behavior is unspecified in other situations." ² Unspecified is not a bug. It is a documented boundary, and Cherokee sits on the far side of it.

The naive fix, and the key it quietly creates

The obvious repair is to lowercase both sides before comparing. It works — and it hides the defect one layer down. str.lower() and str.casefold() both report these two tags equal, but they disagree about which string the pair collapses to. Two indexes built with two different verbs can never find each other's entries.

>>> 'Ꭰone'.lower(),    'ꭰone'.lower()       # both -> 'ꭰone'   (U+AB70)
>>> 'Ꭰone'.casefold(), 'ꭰone'.casefold()    # both -> 'Ꭰone'   (U+13A0)
>>> 'Ꭰone'.lower() == 'ꭰone'.lower()        # True
>>> 'Ꭰone'.casefold() == 'ꭰone'.casefold()  # True
>>> 'Ꭰone'.lower() == 'Ꭰone'.casefold()     # False

Read the last three lines together. Both verbs agree on every comparison you ask them to make, and both produce a canonical key the other one will never match. A tag index built with lower() and a search box that folds with casefold() will agree that any two strings are equal, and still fail to find each other's rows.

That is what a note vault with two tools in it looks like from the inside.

The direction is the tell. lower() folds down, to the small letter U+AB70. casefold() folds up, to the capital U+13A0. Python's documentation is explicit that these are different operations: "Casefolding is similar to lowercasing but more aggressive because it is intended to remove all case distinctions in a string. For example, the German lowercase letter 'ß' is equivalent to "ss". Since it is already lowercase, lower() would do nothing to 'ß'; casefold() converts it to "ss"." ³

Both entries then cite the same section of the same standard — section 3.13, Default Case Folding ⁴. Same reference, two verbs, two answers. The name of the operation is doing more work than most code gives it credit for.

Where each table comes from

Six ways to answer are these the same letter, and six different sources of truth. GNU grep reads the locale. ripgrep ignores it. Python's str uses whatever Unicode data its interpreter was built with. stringprep uses a copy frozen at Unicode 3.2.0. SQLite uses twenty-six ASCII letters. ext4 uses Unicode 12.1.0.

SQLite is the quiet one, and it says why out loud: "Note that only ASCII characters are case folded. SQLite does not attempt to do full UTF case folding due to the size of the tables required." ¹² That is the thesis of this post in a vendor's own words.

Case-insensitivity is a table. A complete table is large — so every project draws that line somewhere different.

ext4 is the other end of the same trade. Case-insensitive directories on ext4 are opt-in, and what that flag does to a note vault is its own story. The sentence after the one everybody quotes is the one that matters here: "By default, the charset adopted is the latest version of Unicode (12.1.0, by the time of this writing), encoded in the UTF-8 form." ¹¹ The filesystem pins a version too.

Count the versions on one ordinary Linux box. A case-folding ext4 directory answers from Unicode 12.1.0. The interpreter behind python3 answers from 15.0.0. Any code path through stringprep answers from 3.2.0. Three tables, one machine — and a colleague running a newer interpreter makes it four, because Seth Larson's prints 17.0.0 ⁸.

None of this is visible from the command line anyone actually types. Search-first retrieval hands you rg -i as a recipe; the flag is two characters, and behind it sits a table nobody named.

What actually changed since 2002

Unicode 3.2 assigned 234,737 codepoints; Unicode 15.0 assigns 288,767 ^{13 14}. That gap is the wrong number to be frightened by. The number that rewrites text somebody already had is 126: characters assigned in 2002 that had no case at all, and acquired one afterwards. Two scripts, two releases.

Start with the honest decomposition, because the totals invite a fair objection. Of the 3.2 figure, 137,468 codepoints are private use, 2,048 are surrogates, and 65 are controls — and those three blocks are byte-identical in every version since. Counting only graphic and format characters, the move is 95,156 in 2002 to 149,186 today.

The gap is 54,030 either way. That is why the gap is the safe number to quote and the totals are not.

The count itself has a trap. UnicodeData.txt compresses large blocks into <…, First> and <…, Last> row pairs, so a line count comes out short by 220,863 rows in the 3.2 file and 253,843 in the 15.0 one ^{13 14}. The ranges have to be expanded:

rows = [(int(f[0], 16), f[1], f[2]) for f in
        (l.split(';') for l in open('UnicodeData.txt', encoding='utf-8') if l.strip())]
total, i = 0, 0
while i < len(rows):
    cp, name, cat = rows[i]
    if name.endswith(', First>'):
        total += rows[i + 1][0] - cp + 1   # expand the range
        i += 2
    else:
        total += 1
        i += 1

Newly assigned characters are the easy case: nothing in an old note is spelled with a codepoint that did not exist when the note was written. The hard case is a character that was already there and changed underneath.

Compare the 3.2 table with the 15.0 table codepoint by codepoint. 185 already-assigned characters carry a different General Category today, and 126 of them moved from Lo — a letter with no case — into a cased category. They come from exactly two scripts: Cherokee, 85 characters that became uppercase letters, and Georgian, 41 that became lowercase ones, spanning U+10D0 to U+13F4.

Both moves have dates. U+13A0 CHEROKEE LETTER A is Lo in Unicode 7.0 and Lu in Unicode 8.0, where U+AB70 CHEROKEE SMALL LETTER A appears for the first time and the 13A0 row starts carrying a lowercase mapping ¹. Georgian follows in Unicode 11.0: U+10D0 GEORGIAN LETTER AN goes from Lo to Ll, and U+1C90 GEORGIAN MTAVRULI CAPITAL LETTER AN is introduced above it ¹⁵.

An interpreter carries the receipt, because CPython ships the old database alongside the current one:

>>> import unicodedata
>>> unicodedata.category('Ꭰ')                 # 'Lu'  — today
>>> unicodedata.ucd_3_2_0.category('Ꭰ')       # 'Lo'  — the same character, in 2002
>>> unicodedata.unidata_version               # '15.0.0' on this interpreter

Say the small number out loud: 126 characters out of 234,737 is a rounding error by count. The argument was never that this is common. The argument is that case-insensitive is not one thing, and 126 characters are enough to prove it.

The receipt: CVE-2026-17084

This is not a thought experiment. On 18 August 2026 the Python Software Foundation published CVE-2026-17084 against CPython's stringprep module and idna codec, scored 6.0 MEDIUM with integrity-only impact ^{16 17}. The cause: the module read current Unicode attributes where RFC 3454 requires the ones frozen at Unicode 3.2.0.

Bitshift reported the vulnerability; Seth Larson, the record's credited coordinator, wrote it up under the title When str.lower() is a security vulnerability in Python ¹⁶. His verdict on the offending line is one sentence long: "The str.lower() call in this function is a vulnerability!" ¹⁸

The mechanism is already on the table above. stringprep implements RFC 3454, which says a profile doing caseless comparison "SHOULD map using either appendix B.2 or appendix B.3" ¹⁹ — tables derived from Unicode 3.2 and pinned there. Nameprep, the IDNA 2003 profile, states the pin flatly: "This profile uses Unicode 3.2, as defined in [STRINGPREP] Appendix A." ²⁰

CPython's implementation reached for the interpreter's live tables instead. On an unpatched CPython 3.12.3, the divergence reproduces in three lines:

>>> 'ᎠᎠ'.encode('idna')                 # b'xn--kz9aa'  — folded via today's tables
>>> b'xn--' + 'ᎠᎠ'.encode('punycode')   # b'xn--58da'   — the RFC-conformant value
>>> import stringprep
>>> stringprep.map_table_b2('Ꭰ')        # 'ꭰ' (U+AB70)  — should have been a no-op

The expected value is derived, not asserted: U+13A0 is Lo in Unicode 3.2 and appears nowhere in CaseFolding-3.2.0.txt ²¹, so tables B.2 and B.3 do nothing to it, leaving the raw Punycode encoding.

Two systems, one domain name, two different labels. The CVE record describes exactly that: "the latest Unicode codepoint attributes were used instead of the specified Unicode 3.2.0. This behavior would cause mismatches when processing domain names using IDNA 2003 (the "idna" codec) and the in_table_b2() function of the "stringprep" module." ²²

The repair is the interesting measurement. Commit 7e109d0, authored by Larson on 18 August 2026, touches seven files and changes 3,275 lines: 1,924 additions against 1,351 deletions, of which Modules/unicodedata_db.h alone accounts for 1,564 and 1,243 ²³.

Inside Lib/stringprep.py, the frozen exception table b3_exceptions grows from 650 entries to 1,388. That is 738 codepoints explicitly pinned back to Unicode 3.2 behaviour, so that "ignore case" would mean in 2026 what it meant in 2002. The issue that tracked the work, gh-155292, was still open on 19 August 2026 ²⁴.

Two honest bounds. The CVE's own text limits the blast radius: it "only affects domain names containing characters that were not previously registered or had their Unicode attributes such as case-folding behavior updated since Unicode 3.2.0" ²⁵.

And the finder's own advice points away from the affected path entirely: "In general, you should be using the idna package (IDNA 2008) and not .encode("idna") (IDNA 2003), but sometimes you do need the older behavior." ²⁶

IDNA 2003 is old. That is the point — a table outlives the decision to use it.

The standard did its job

Unicode did not move under anyone. Case folding is guaranteed stable by formal policy, and the Consortium engineered Cherokee's fold direction specifically to keep that promise. Lowercasing carries no such guarantee. The defect is in the verb: a tool that lowercases and calls the result caseless matching has stepped off the stable path on its own.

The policy is unambiguous. "Caseless matching of Unicode strings used for identifiers is stable. Case folding stability ensures that identifiers created in different versions of Unicode can be reliably matched in a case-insensitive manner." ²⁷ Its formal statement is a promise about every future version:

"For each string S containing only assigned characters in a given Unicode version, toCasefold(toNFKC(S)) under that version is identical to toCasefold(toNFKC(S)) under any later version of Unicode." ²⁸

The Cherokee case is not an oversight the Consortium missed. It is the example the policy itself uses to show the promise being kept:

"Note: Case folding is not the same as lowercasing, and a case-folded string is not necessarily lowercase. In particular, as of Unicode 8.0, Cherokee has become a bicameral script with the introduction of lowercase Cherokee letters, but Cherokee text case folds to the existing uppercase letters. This case folding behavior for Cherokee text is precisely to guarantee continued case folding stability." ²⁹

That is why casefold() folds up to U+13A0. A fold direction was chosen against intuition, in public, so that text written in 2002 still matches text written in 2026.

The failure mode was written down too, by the authors of RFC 3454 in December 2002:

"Authors of profiles of this document need to consider the effects of changing the mapping of any currently-assigned character when updating their profiles. Adding a new mapping for a currently-assigned character, or changing an existing mapping, could cause a variance between the behavior of systems that have been updated and systems that have not been updated." ³⁰

Twenty-four years early, two spec authors described the CVE — and the title of this post — in one sentence.

Trade-offs: what to do in a vault meant to last

None of this makes a note vault unsafe. It makes it inconsistent, in a narrow and specific way: 126 characters out of 234,737, across two scripts. The honest remedy is not to fix Unicode, which is already correct. It is to pick one verb, apply it everywhere a key is written, and keep filenames boring.

Five moves, in the order they pay off.

1. Fold with casefold(), never lower(), anywhere a lookup key is stored. The two verbs agree on comparisons and disagree on keys, and only one of them is covered by the stability policy ²⁸. Mixing them across an indexer and a search box is the failure that has no error message.
2. Print the version next to the table. unicodedata.unidata_version, rg --version, SQLite's sqlite_version(), the value of LC_CTYPE. A table nobody can name is a table nobody can reproduce.
3. Do not let the database be the arbiter of identity. COLLATE NOCASE folds 26 letters and says so ¹²; every accented, Cherokee, Georgian, or Greek tag passes through it unfolded.
4. Keep filenames to lowercase ASCII. It is the same discipline that survives a vault crossing three operating systems and a vault crossing three line-ending conventions, and it is a far narrower promise than "my tools handle Unicode".
5. For domain names, use IDNA 2008. Its designers replaced the frozen table with something version-independent, which is a structural answer rather than a patch ³¹.

What this does not fix: text already written under two different keys, which needs a re-index rather than a convention, and any tool whose folding you cannot inspect.

It is worth being precise about the layer, too. Encoding decides which bytes represent a character; folding decides which characters count as the same one. A vault can be flawless UTF-8 and still disagree with itself about case.

Frequently asked questions

Eight questions people actually type into a search box when two tools stop agreeing about the same string. Every answer below reduces to the same three-part discipline: name the operation you are performing, name the Unicode version behind it, and stop calling two different operations by the same phrase.

Why is upper casing not enough for case-insensitive comparison? Because uppercasing and lowercasing are locale- and version-sensitive mappings, while caseless matching is a separate operation with its own table. Unicode's stability policy covers case folding, not case mapping ²⁷. Uppercase both sides and you get a defensible answer for ASCII pairs and an unspecified one elsewhere ².

What is the difference between lower() and casefold() in Python? casefold() is more aggressive: it removes all case distinctions rather than mapping to lowercase. Python's own example is German 'ß', which lower() leaves alone and casefold() turns into "ss" ³. On Cherokee they diverge in direction — lower() folds down to U+AB70, casefold() folds up to U+13A0.

Why doesn't my regex do Unicode case folding? Usually because the engine is reading the locale rather than a Unicode table, or the pattern is being matched byte-wise. ripgrep is explicit that it does not respect locale settings and folds with Unicode simple case folding by default ^{6 7}; GNU grep's -i is documented as unspecified outside lowercase-uppercase pairs ².

Why do my database and my language disagree about case-insensitive matching? Because they consult different tables. SQLite's COLLATE NOCASE folds the 26 ASCII uppercase letters and nothing else, by design, "due to the size of the tables required" ¹². A language runtime folds by the full Unicode database it was built against. Same query, two answers, no bug.

Does SQLite COLLATE NOCASE work with Unicode? Not beyond ASCII. 'Todo' = 'TODO' COLLATE NOCASE matches; 'Ꭰone' = 'ꭰone' COLLATE NOCASE does not, on SQLite 3.45.1. The documentation states the boundary plainly: only ASCII characters are case folded, because full UTF case folding would require larger tables ¹².

Is Unicode case folding stable across versions? Yes, by formal policy — that is the part most retellings get backwards. Case-folded, NFKC-normalized strings compare identically under any later version of Unicode ²⁸, and Cherokee's fold direction was chosen specifically to preserve that guarantee ²⁹. What is unstable is lowercasing and anything derived from it.

Is comparing and sorting Unicode filenames the same problem? Related, not identical. Whether a filesystem ignores case at all is a separate question from which table it uses when it does — ext4's case-insensitive lookup defaults to Unicode 12.1.0 ¹¹. Normalization is a third, independent operation: the same visible character can be one codepoint or two.

Should I still use .encode("idna") in Python? Prefer the idna package, which implements IDNA 2008, unless the older behaviour is specifically required ²⁶. IDNA 2008 was written to specify "a procedure, and not a table, of code points so that the algorithm can be used to determine code point sets independent of the version of Unicode that is in use" ³¹.

The version is part of the question

The thesis of this post is that caseless matching is a table. The IETF reached the same conclusion in 2010 and acted on it: RFC 5892 replaced the frozen appendix with a derivation, and told implementers not to copy the printed table at all. Specify the procedure, not the snapshot.

The instruction is explicit. The derived property value "is to be calculated in cooperation with a designated expert [RFC5226] according to the specifications in Sections 2 and 3 and not by copying the non-normative table found in Appendix B" ³². Unicode kept its promise; the IETF removed the need to trust anyone's copy of a table; the tools in between still assert that "ignore case" needs no further explanation.

It does. Two programs can both ignore case, correctly, according to their own documentation, and still hold different opinions about which letters exist.

The Python half of this story is Bitshift's report and Seth Larson's write-up ¹⁸; what it looks like from inside a note vault is the half above.

Every command in this post runs against plain files you already have — which is the argument for keeping notes in open Markdown on your own device, the way mnmnote.com does.