How AI Runs Inside Your Browser Tab

A language model can now load and run inside a single browser tab, read the text already on the page, and answer — without a server call. Two paths ship it today: the browser's own built-in model, and a model your page downloads once and runs on the GPU through WebGPU.

For most of the last three years, "AI over your notes" meant a round trip. Your text left your machine, crossed a network, was processed on rented hardware, and came back. That round trip was the whole architecture, and it was also the whole privacy problem. What changed in 2025 is that the round trip became optional. Chrome shipped a family of built-in AI APIs backed by a model that runs on the device¹, and the open-source WebLLM engine reached the point where a page can run a real model on the GPU at up to 80% of native throughput². The most private place to run AI over your notes is now the same tab your notes already live in.

The problem in concrete terms

Running AI over private text usually means uploading it. A note about your health, your salary, or an unshipped idea becomes a request body sent to a vendor's server, logged somewhere you cannot see, and governed by a policy you did not write. The privacy cost is not a setting you forgot to toggle — it is the architecture itself.

This is the tension every local-first tool inherits. Notes are cheap to keep on the device and expensive to keep there once you want a model to read them. The moment you ask "summarize this" or "what did I decide," the conventional answer ships the text out. The question is not whether the model is good. It is where the reading happens.

The naive approach, and why it fails the privacy test

The default design sends the note to a hosted model over HTTPS and renders the reply. It works, it is fast, and it quietly breaks the one promise a local-first note tool makes. The text leaves the device, and no amount of transport encryption changes that it now sits on someone else's computer.

Transport security protects the note in transit. It does nothing about the note at rest on the far end. The FTC's 2024 action against Avast — a data-sale ban plus $16.5 million in redress over a misrepresented "privacy" claim — is the standing reminder that "we take privacy seriously" is a representation, not an architecture. A hosted round trip also fails offline, meters every token, and couples your notes to a vendor's uptime. For a document you may want to read in ten years, that is three failures stacked on one privacy hole.

The architecture: how in-tab inference works

In-tab inference removes the network from the hot path. The model weights are fetched once — from a CDN or the browser itself — and cached. The page then holds the model and runs the forward pass locally: on the GPU through WebGPU, or the CPU through WebAssembly. Your notes become the prompt, and the prompt never leaves the page.

The shape is worth stating precisely, because the privacy claim depends on it. There are two data flows, and only one of them touches a server. The weights come down once. The inference — your Markdown, your question, the model's answer — stays in the tab.

flowchart TD
  A[Model weights<br/>downloaded once] --> B[Model held<br/>in the tab]
  C[Your notes<br/>on the page] --> B
  B -->|WebGPU or WASM| D[Answer rendered<br/>in the page]
  B -. never uploaded .-> E[No server<br/>reads your text]

Figure: The two data flows of in-tab inference. Model weights are downloaded once and cached; after that, the page runs the model on your notes locally through WebGPU or WebAssembly, and your note text and the answer are rendered in the page without any upload.

WebGPU is the piece that makes this practical rather than academic. It exposes the machine's GPU to JavaScript, and it is no longer a Chrome-only experiment: it shipped in Chrome and Edge at version 113, reached Firefox on Windows at version 141, and landed in Safari 26³. A page that detects WebGPU can assume a real accelerator is available on most current desktops.

The weights that arrive are quantized to fit. A model shipped for the browser is compressed toward roughly four bits per parameter — that is what the q4f16_1 label in the WebLLM example means — which is why a model of one to a few billion parameters fits inside a tab's memory budget and loads in seconds on a warm cache instead of minutes. Precision falls with the bit width: a four-bit model is smaller, faster, and slightly less sharp than its full-precision original. That compression is the quiet tax that makes the whole thing fit in a page.

Path one: the browser's built-in model

Chrome now ships a model with the browser. The Prompt API and its siblings call a local model — Gemini Nano — and, per Google's documentation, "No data is sent to Google or any third party" when the model runs⁴. The Summarizer, Translator, and Language Detector APIs reached stable in Chrome 138¹.

The appeal is that there is nothing to download in your application code. The browser manages the model, and the general Prompt API reached stable for extensions while the web version continues to roll out¹. "The Prompt API uses the Gemini Nano model in Chrome. While the API is built into Chrome, the model is downloaded separately the first time an origin uses the API"⁴ — so the first call on a given site triggers a one-time fetch, and every call after that is local. The surface is small enough to read in one screen.

// Chrome built-in AI: everything runs on-device after a one-time model download.
if ("LanguageModel" in self) {
  const status = await LanguageModel.availability();
  // "available" | "downloadable" | "downloading" | "unavailable"
  if (status !== "unavailable") {
    const session = await LanguageModel.create();
    const answer = await session.prompt(
      "Summarize the decision in this note:\n\n" + noteMarkdown
    );
    console.log(answer);
    session.destroy();
  }
}

Path two: a model you load yourself

WebLLM loads an open model into the page and runs it on WebGPU, independent of any browser feature. The project's own line is blunt: "Everything runs inside the browser with no server support and is accelerated with WebGPU"⁵. It exposes an OpenAI-compatible interface, so code written against the hosted API mostly ports by swapping the client.

This path trades a larger first-load download for control: you pick the model — Llama, Phi, Gemma, Qwen, Mistral — and you are not waiting on a browser vendor to ship a capability⁵. The academic evaluation behind the engine reports it can "retain up to 80% native performance on the same device"², which is the difference between a demo and a tool.

import { CreateMLCEngine } from "@mlc-ai/web-llm";

// Weights download once into the browser cache, then inference is local.
const engine = await CreateMLCEngine("Llama-3.2-1B-Instruct-q4f16_1-MLC");

const reply = await engine.chat.completions.create({
  messages: [
    { role: "system", content: "Answer only from the note provided." },
    { role: "user", content: "What did I decide?\n\n" + noteMarkdown },
  ],
});
console.log(reply.choices[0].message.content);

The glue between either path and your notes is trivial, because the notes are already in the page. There is no export step, no upload, no parsing a proprietary format back into text — the Markdown in the editor is the context you feed the model.

// The note is already text in the DOM — read it, don't upload it.
const noteMarkdown = document.querySelector("#editor").innerText;
const prompt = `Use only the note below.\n\n---\n${noteMarkdown}\n---\n\nQ: `;

What "nothing leaves the page" actually means

Scope the claim honestly, or it breaks on inspection. Two things move across a network in the whole lifecycle, and both are one-time and content-free: the model weights, and the application code. Your note text, your prompts, and the model's answers are the inference, and the inference runs in the tab. That is the line that holds.

Simon Willison put the confusion precisely when he loaded a model into Chrome from disk: the browser warns about "upload," but "nothing is uploaded from your browser, the model file is opened locally on your machine"⁶. The verb is misleading; the direction is inward. The same is true of a cached model — the download is the model arriving, not your notes leaving. Say "your notes are never uploaded," not "nothing ever touches the network," and the claim survives a hostile read.

Benchmark and requirements

The numbers decide whether this is a toy. In-tab inference on WebGPU retains up to 80% of native decoding throughput on the same device², and the browser's built-in path adds nothing to your bundle — but it demands real hardware. Chrome's on-device model expects "Strictly more than 4 GB of VRAM" and at least 22 GB of free disk space⁷.

What a tab-sized model is actually good at

A model small enough to run in a tab is not a shrunken frontier model — it is a different tool. It excels at bounded work over text you hand it: summarizing one note, extracting action items, tagging, rewriting a paragraph, answering a question about the document on screen. It is weak at open-ended reasoning across a whole vault.

Match the task to the size and it holds up. Ask it to draft, condense, classify, or answer from a single note in front of it, and a one-to-few-billion-parameter model does honest work. Ask it to synthesize across hundreds of files, follow a long chain of logic, or recall obscure facts it was never trained on, and it will fail in the confident way small models fail. The fit is narrow and real:

• Good fits: summarize this note, list the decisions, suggest tags, rewrite for tone, answer a question about the open document, extract dates and names.
• Poor fits: reason across an entire archive at once, long multi-step logic, questions that need world knowledge a small model never held.

The scoping is not a limitation to apologize for. It is the reason the privacy math works: the tasks that stay inside the tab are exactly the everyday ones, and those are the ones you least want to ship to a stranger's server.

The trade-offs, stated plainly

In-tab inference is not without cost — it moves the expense from a metered API bill to the user's hardware, the first-load download, and a model smaller than frontier hosted ones. A phone from 2019 will not run it, and a one-billion-parameter local model will not out-reason a hosted giant. This is horses for courses, not a clean win.

The built-in path carries a second caveat worth quoting: "The Prompt API for the web is still being developed"⁴. The stable web surface today is summarization, translation, and language detection; the general web Prompt API is progressing but not finished. Chrome also reclaims the model under storage pressure — "If the available storage space falls to less than 10 GB after the download, the model is removed from your device"⁷. Plan for the model being absent, not just present. WebLLM avoids the vendor dependency at the price of a heavier first load. Neither path is magic; both are now real.

Why notes that already live in a browser tab are the natural fit

The tab is where this trend pays off first, and notes are the cleanest case. A browser-based note app already holds your text as Markdown in the page — plain, structured, and readable by a model without an export, an upload, or a format conversion. The context an in-tab model wants is the exact thing already in front of you.

This is the adjacent step to running a model as a separate process on your machine, which we covered in chat with your notes without uploading them, and it is the same principle traced in local AI is becoming the default. In-tab inference removes even the separate process: the reader and the read live in one window. For a note tool whose whole design keeps your Markdown on your own device, that is not a bolt-on. It is the architecture arriving at its own conclusion.

Frequently asked questions

Can you run an LLM in the browser? Yes. A page can load a language model and run inference locally, either through the browser's built-in model (Chrome's Gemini Nano) or by loading an open model with WebLLM, which runs on WebGPU. The WebLLM engine reports up to 80% of native throughput on the same device².

What is Chrome's Prompt API? It is a JavaScript API that calls a local model — Gemini Nano — built into Chrome. The model is downloaded once per origin, then runs on the device, and Google's documentation states "No data is sent to Google or any third party" during use⁴. Related stable APIs cover summarizing, translating, and language detection¹.

Does in-browser AI send my data to a server? The model weights and application code are fetched once over the network. The inference itself — your text, your prompt, the answer — runs in the tab and is not uploaded. Chrome states no data is sent to Google or a third party when its on-device model runs⁴; WebLLM runs "with no server support"⁵.

What is Gemini Nano? It is the small on-device language model Chrome ships with the browser to power its built-in AI APIs. Chrome's documentation describes it as a "language model, designed to run locally on desktops and laptops"⁷. It backs the Prompt API and, in adapted forms, the other built-in APIs.

What is WebLLM? WebLLM is an open-source in-browser inference engine from the MLC project. It runs open models on WebGPU and exposes an OpenAI-compatible API, so "Everything runs inside the browser with no server support and is accelerated with WebGPU"⁵. It supports Llama, Phi, Gemma, Mistral, and Qwen models⁵.

How much VRAM do I need to run AI in the browser? For Chrome's built-in model, the documented requirement is "Strictly more than 4 GB of VRAM," with a CPU fallback of "16 GB of RAM or more and 4 CPU cores or more," plus at least 22 GB of free disk space⁷. Smaller WebLLM models can run on less, at lower quality and speed.

Which browsers support WebGPU? WebGPU shipped in Chrome and Edge at version 113, reached Firefox on Windows at version 141, and landed in Safari 26³. On most current desktops a page can assume WebGPU is available, with a CPU path as fallback.

Is in-browser AI as fast as running a model natively? Close, not equal. The WebLLM paper reports the engine can "retain up to 80% native performance on the same device"². You trade some throughput and model size for keeping the whole computation — and your text — inside the tab.

The round trip was never a law of nature. It was a stage in the hardware, and the hardware moved: the most private place to run a model over your notes is the page they were already open in.

MNMNOTE is a browser-based, local-first note app that keeps your Markdown on your own device — the same tab a model can now read from without an upload, at mnmnote.com.