Botfilter's detector is a 6-layer MiniLM model with 22.7 million parameters, fine-tuned to tell human writing from AI writing and shipped as a 23.6 MB file inside the extension. It was trained on texts from eight licensed human sources written before ChatGPT (671,247 texts) and on 26,227 texts from 15 AI models prompted to write in the same styles. On a sealed test split it had never seen, at the default Balanced setting, it flagged 94 to 100% of AI-written text, depending on the model group, and 0.2 to 1.8% of human text, depending on the source.
The model is public: the Botfilter model and its card are on Hugging Face, with the test results by source and by model group.
How a page becomes a score
Everything below happens in your browser. The model file ships inside the extension, and it runs in a background worker on your device. There is no Botfilter server, no account, and no sign-in. No page text or score ever leaves your browser. The on-device walkthrough covers what Botfilter stores, including synced settings, and how to check its network use yourself.
Normalizing the text
Before scoring, text goes through the same normalizer the model saw in training. It applies Unicode NFKC, turns curly quotes, long dashes, and ellipses into plain ASCII, drops invisible characters such as zero-width spaces, and maps Cyrillic or Greek look-alike letters to Latin when the text is mostly Latin. It then replaces links with [URL] and handles with @user, and collapses runs of spaces. It never adds or removes words, so word counts stay the same.
Before: “Thanks for sharing…” read it at https://example.com/post, @dana_k. Ｗоrth it.
After: "Thanks for sharing..." read it at [URL] @user. Worth it.This matters for two reasons. Typography is a weak and easily faked signal: a model that learned "curly quotes mean AI" would flag people whose phones insert them. And look-alike letters are a known way to slip text past detectors. After normalizing, swapping 30% of the letters in AI-written text for look-alikes changed its flag rate by exactly zero in our robustness tests.
Scoring long text in two 256-token windows
MiniLM reads at most 256 tokens at a time. Botfilter keeps the first 4,000 characters of a text, and when that is longer than one window, it scores the first 256 tokens and the last 256 tokens and averages the two. The number it averages is a logit margin: how far the model's AI score sits above its human score. A margin of 0 means the model leans neither way.
From margin to probability
A raw margin is not a probability. Botfilter divides it by a temperature, T = 1.3289, and applies the logistic function: probability = 1 / (1 + e−margin / T). The temperature was fit on the validation split by minimizing log loss, so the percentages track how often texts at each score were AI-written in that validation mix, which was about 12% AI. On a page where AI text is more or less common than that, the true share at a given percentage will differ. That probability is what appears on each mark, such as "likely AI-written · 92%".
The base model and the 23.6 MB file
The starting point is nreimers/MiniLM-L6-H384-uncased, an MIT-licensed 6-layer version of Microsoft's 12-layer MiniLM, made by keeping every second layer. It has a hidden size of 384 and 22.7 million parameters. We added a two-class head (human, AI) and fine-tuned the whole network.
We chose it for size. The detector has to ship inside a browser extension and run on every post in a feed without a server. A larger encoder would mean a much bigger download, or sending text somewhere else to be scored. Botfilter does neither.
After training, the model is exported to ONNX and its weights are stored as 8-bit integers (weight-only int8). Activations stay in floating point, so a text gets the same score whether it runs alone or in a batch with others. That brings the file to 23,626,443 bytes. The extension runs it with ONNX Runtime Web, on WebGPU when the device has a hardware GPU adapter and it is faster, and on WebAssembly otherwise.
| Check | Result |
|---|---|
| Mean margin difference, int8 against fp32 | 0.055 |
| 99th percentile margin difference | 0.340 |
| Largest difference, one text scored alone against in a batch | 0.00001 or less |
| Browser runtime against the Python model, measured in the extension tests | within 0.05 |
Source: Botfilter calibration run for minilm-l6-v5-20260928 (validation split; the calibration file ships in the extension and on the model card) and the extension's parity tests.
Where the human text came from
The hardest part of building a detector is not the model. It is finding human writing that is certainly human, that you are allowed to train on, and that looks like what people read every day. We used three rules.
- Written before ChatGPT. Human text from after November 2022 might contain AI writing. The main sources date from before then, and open web pages from before 2020. Newer human text from the news, Parliament, and food blog sources trains at half weight. Newer casual text does not train at all.
- A license that allows training, for each source. For news, that meant reading the license on every document: of 172,308 articles scanned, we kept only those with an exact CC BY 4.0 match, as recorded in each article's license metadata, and rejected CC BY-SA. Non-commercial and share-alike sources were left out. That ruled out several public AI detection datasets for training, and ruled out scraped posts from X, LinkedIn, Hacker News, YouTube, Bluesky, and Mastodon for any use, including testing.
- Casual writing, not just formal writing. A detector trained only on newspapers learns that casual writing is odd, and then flags people on Reddit. An earlier internal model, v3, showed how easy this is to miss: its false-positive rates looked low because they were measured on the same sources it trained on, but on human writing from sources it had not seen, it flagged 7 to 20%. So half of the human weight now goes to casual writing: Reddit posts, work email, GitHub discussion, and ordinary web pages.
| Source | Kind of writing | Years | License | Texts |
|---|---|---|---|---|
| Webis TL;DR corpus | Reddit posts | 2006 to 2016 | CC BY 4.0, per the dataset metadata | 274,727 |
| FineWeb | Ordinary web pages | Before 2020 | ODC-BY 1.0 | 120,661 |
| Common Pile news | News articles from open newswires | Mixed | CC BY 4.0, per article | 72,750 |
| Common Pile GitHub | Issue and pull request comments | Mixed | Permissive repository licenses | 69,938 |
| UK Hansard | Parliamentary debate | 1995 onward | Open Parliament Licence v3.0 | 60,521 |
| Foodista | Food blog posts | Mostly 2009 to 2014 | CC BY 3.0 | 37,541 |
| Enron email | Work email | 1999 to 2002 | No license; made public by FERC | 26,976 |
| Anthropic HH-RLHF | Chat prompts written by people | 2021 to 2022 | MIT | 8,133 |
Source: Botfilter v5 training data list; the model card names each source and license. Totals 671,247 human texts. Enron text had headers, addresses, greetings, and sign-offs removed; GitHub text had code, logs, and bot comments removed.
We made two cleaning choices. We stripped greetings and sign-offs from email, and bylines from news, so the model could not learn "starts with Hi Tom" as a human signal. And we split the data by author, speaker, site, repository, or conversation, so the same person never appears on both sides of the train and test line.
The casual sources carry their distributors' licenses and releases. We did not clear rights with each original author of each Reddit post or email, and we say so on the public model card. Only the trained model file is distributed; none of this text ships with Botfilter.
How the AI text was written
A detector learns whatever separates its two piles. If the AI pile is all essays and the human pile is all Reddit, it learns "essay versus Reddit". So every AI text was written to match a kind of human writing, and most were seeded by a human text from the same split.
- Mirror (11,945 texts): write a new text on the same topic and in the same kind of writing as a human example.
- Free (8,851): write a text from a situation and a style, with no human example.
- Reply (4,527): answer a human post the way a commenter would.
- Heavy rewrite (462) and expand (442): rework a human text, or turn short notes into prose. These count as AI-written.
Light polish of human text never trains as AI. A person who asks a model to fix their commas still wrote the post.
For this version, we added four styles aimed at the feeds people complain about most: upbeat LinkedIn career announcements, general LinkedIn posts, X posts, and hook-first viral X threads. That run produced 3,153 usable texts from 3,520 requests, after automatic quality checks threw out 367. Those texts were split across training, validation, and test like everything else.
| Generator | How we ran it | Texts |
|---|---|---|
| Phi-4 | Self-hosted, open weights | 4,960 |
| Qwen3 8B | Self-hosted, open weights | 3,888 |
| Qwen3 4B Instruct 2507 | Self-hosted, open weights | 3,528 |
| OLMo 2 13B | Self-hosted, open weights | 3,345 |
| Mistral 7B Instruct v0.3 | Self-hosted, open weights | 2,354 |
| SmolLM3 3B | Self-hosted, open weights | 2,148 |
| Mistral Small 24B | Self-hosted, open weights | 1,994 |
| Phi-4 mini | Self-hosted, open weights | 430 |
| Claude Sonnet 5 | Provider API | 724 |
| Claude Haiku 4.5 | Provider API | 659 |
| GPT-6 Sol | Provider API | 574 |
| GPT-5.6 Terra | Provider API | 433 |
| Claude Opus 5.5 | Provider API | 415 |
| GPT-5.5 | Provider API | 391 |
| Mistral Medium 3.5 | Provider API | 384 |
Source: Botfilter v5 training data list. 22,647 texts from open models and 3,580 from provider APIs, 26,227 in total. Every open model used is Apache-2.0 or MIT licensed.
The AI texts span the same ten kinds of writing as the human side. By count: email 5,146, Reddit-style posts 4,707, social posts 4,009, articles 3,012, web pages 2,955, developer forum comments 2,319, formal writing 1,169, blog posts 1,137, chat 1,125, and reviews 648.
Some models were kept out of training on purpose, so we could measure how the detector does on writers it has never met: Gemini 3.8 Flash, the whole Granite family (Granite 3.3 8B), and three more families added only for testing, gpt-oss-20b, Ministral 8B, and Qwen3.7 Flash.
Training, calibration, and the three sensitivities
Training is short. Each epoch draws 160,000 texts with replacement from the pool of 697,474, weighted so that human and AI text each carry half the mass, for two epochs: 320,000 draws, about 5,000 optimizer steps at a batch size of 64. It took about 8 minutes on one NVIDIA L4 GPU.
| Setting | Value |
|---|---|
| Optimizer | AdamW, learning rate 5e-5, weight decay 0.01 |
| Schedule | Cosine, with 6% linear warmup |
| Loss | Cross-entropy, no label smoothing |
| Batch and length | 64 texts, 256 tokens |
| Precision | bf16 autocast; gradient norm clipped at 1.0 |
| Short-text crops | 30% of texts over 40 words cropped to a random 25 to 80 word span |
| Long-text crops | Otherwise, texts over 220 words cropped to a random 220-word span |
| Weighting | Half the human weight on casual writing; AI weight spread across generators by the square root of their counts, with provider API models counted double |
Source: Botfilter v5 training settings, as described in this article.
Social media needed one more step. Greetings, sign-offs, emoji, hashtags, links, and @mentions are added to human and AI texts at matched rates. Without that, the model would learn that an emoji means human or a hashtag means AI, and it would be easy to fool in either direction.
How Fewer, Balanced, and More are set
Each sensitivity is a false-positive target on validation human text: Fewer aims to flag 0.5% of human texts, Balanced 1%, and More 3%. Every human source counts equally when we set the cutoffs, so Reddit posts count as much as news. Short and long texts get separate cutoffs, because a 40-word post gives the model much less to go on than a 400-word article.
| Length | Fewer | Balanced | More |
|---|---|---|---|
| 25 to 80 words, margin | 4.30 | 3.19 | 0.35 |
| 25 to 80 words, probability | 96% | 92% | 56% |
| Over 80 words, margin | 2.81 | 1.13 | −1.78 |
| Over 80 words, probability | 89% | 70% | 21% |
Source: the calibration file shipped in the extension (temperature 1.3289). Probabilities are computed from the margins and rounded.
The table shows why the cutoffs come from false-positive targets and not from a fixed probability. Human short posts reach high margins more often, so a short post needs 92% at Balanced. Long human text rarely scores high, so a long text can be marked at 70% and still flag only about 1% of human writing. At More, a long text is marked from 21%: you are asking to see more, and paying for it with more human text flagged.
| Setting | Tuned to flag, of human validation texts | Human texts flagged, by source | AI texts flagged, by model family |
|---|---|---|---|
| Fewer | about 1 in 200 | 0.1% to 0.9% | 92% to 100% |
| Balanced | about 1 in 100 | 0.2% to 1.8% | 94% to 100% |
| More | about 3 in 100 | 1.2% to 5.1% | 97% to 100% |
The tuning targets were set on validation text. The measured ranges come from the 45,019 human and 5,912 AI texts of the sealed test split below. The highest human rates at Balanced were on work email (1.6%) and web pages from before 2020 (1.8%).
Results on the sealed test split
The test split was held back from the start. It was grouped the same way as training, locked by a file hash, verified against that hash before scoring, and scored once, after the model and every cutoff were final. Before training, every training and validation text that shared 30% or more of its 8-word sequences with the test split was dropped, which removed 6,176 near-copies. The split holds 51,332 texts: 45,019 human, 5,912 AI, and 401 human texts that a model edited.
False positive rate on human text, by source
| Human text | Texts | Fewer | Balanced | More |
|---|---|---|---|---|
| News | 7,031 | 0.3% | 0.5% | 2.0% |
| Food blogs | 3,160 | 0.4% | 0.5% | 1.9% |
| UK Parliament | 8,000 | 0.3% | 0.5% | 1.5% |
| Chat prompts | 1,217 | 0.1% | 0.2% | 1.9% |
| Reddit posts, 2006 to 2016 | 8,000 | 0.2% | 0.5% | 1.2% |
| Work email | 3,605 | 0.8% | 1.6% | 4.3% |
| GitHub discussion | 6,006 | 0.6% | 1.2% | 3.6% |
| Web pages, before 2020 | 8,000 | 0.9% | 1.8% | 5.1% |
Source: Botfilter model minilm-l6-v5-20260928, sealed test split, scored once on September 28, 2026; results by source and model group are on the model card. Rates are pooled over short and long texts, so a few differ slightly from the per-length figures on the model card.
Ordinary web pages and work email are the hardest human writing for the model. Both carry a lot of templated, polite prose, and short texts from both run slightly higher: 2.0% for short web pages and 1.9% for short emails at Balanced. If a page has 50 human paragraphs, 1.8% works out to about one false mark at Balanced; Fewer halves that rate on web pages, to 0.9%. For why a low percentage still matters at scale, see what false positives mean for AI detectors.
AI text caught
| AI text | Texts | Fewer | Balanced | More |
|---|---|---|---|---|
| Eight open models (trained on) | 3,159 | 95% | 97% | 99% |
| Claude Sonnet 5, Opus 5.5, Haiku 4.5 | 322 | 94% | 95% | 98% |
| GPT-6 Sol, GPT-5.5, GPT-5.6 Terra | 268 | 94% | 96% | 99% |
| Mistral Medium 3.5 | 69 | 93% | 94% | 99% |
| Gemini 3.8 Flash (never trained on) | 147 | 92% | 94% | 97% |
| Granite 3.3 8B (family never trained on) | 1,793 | 98% | 99% | 99% |
| gpt-oss-20b, Ministral 8B, Qwen3.7 Flash (never trained on) | 154 | 100% | 100% | 100% |
Source: Botfilter model minilm-l6-v5-20260928, sealed test split, scored once on September 28, 2026; results by source and model group are on the model card. Rates are pooled over short and long texts, so a few differ slightly from the per-length figures on the model card.
Within those groups, single generators range from 92.9% (Claude Sonnet 5 and Mistral Small 24B) to 100% at Balanced. The rows marked "never trained on" matter most, because new models ship every month. Gemini 3.8 Flash was caught 94% of the time at Balanced and Granite 99%. The three extra families were caught every time, though 154 texts is a small sample: the 95% confidence interval runs from 97.6% to 100%.
Length matters more than the model that wrote the text. Long AI text (over 80 words) was caught 98.8 to 100% of the time in every kind of writing except chat, at 94.8%. Short AI text (25 to 80 words) was caught 90.8 to 96.3% of the time.
| Kind of writing | Over 80 words | 25 to 80 words |
|---|---|---|
| Social posts | 100% | 95.9% |
| 100% | 96.3% | |
| Reddit-style posts | 99.3% | 95.0% |
| Developer forums | 99.5% | 95.4% |
| Web pages | 99.7% | 95.3% |
| Formal writing | 98.9% | 95.2% |
| Chat | 94.8% | 94.4% |
| Blog posts | 98.8% | 93.5% |
| News-style articles | 99.3% | 90.8% |
Source: Botfilter model minilm-l6-v5-20260928, sealed test split, scored once on September 28, 2026; results by source and model group are on the model card. Rates are pooled over short and long texts, so a few differ slightly from the per-length figures on the model card.
Text a person and a model both worked on
We also scored 401 human texts that a model had edited in one of three ways. When a model expanded a person's notes into prose, Botfilter flagged it 94% of the time. A heavy rewrite was flagged 56% of the time, and a light polish 11%.
Robustness to small edits
We tested ten meaning-preserving edits on up to 250 validation texts in each of 18 groups: smart or plain punctuation, 4% typos, 30% look-alike letters, zero-width characters, emoji and hashtags, an added link, an added mention, a named sign-off ("Thanks, Mike"), and a placeholder sign-off ("Best regards, [Your Name]"). Apart from the placeholder, no edit raised a human group's flag rate, or lowered an AI group's, by more than 2 points. Look-alike letters and zero-width characters changed nothing.
The placeholder sign-off is not held to that limit, because a leftover "[Your Name]" is itself a common sign of generated text. Added to human text, it raised the flag rate by 2.4 to 4.4 points in five of the eight kinds of human writing tested, most in chat (4.4) and email (3.6). A person who pastes a template and forgets to fill it in is more likely to be flagged.
What changed from v4 to v5
The previous model, v4, ran in earlier development builds of the extension. Two problems found in use led to v5. A human LinkedIn post about a career milestone scored at the top of v4's scale and was flagged at every sensitivity. And borderline AI-written social posts and emails slipped through unless you chose More.
The first problem traced back to label smoothing. v4 was trained with a smoothing of 0.05, which caps how confident a model can be: its margins bunched up near ln(0.975 / 0.025), about 3.66. With many texts piled at that ceiling, no cutoff could separate the confident human post from confident AI text. We trained two v5 candidates, one with smoothing and one without. They scored almost the same on validation (AUROC 0.9984 without smoothing, 0.9983 with it, and AI recall within half a point). We shipped the one without smoothing, because it keeps room at the top of the scale.
The second problem was data. v4 trained on four formal human sources and 6,443 AI texts, with none of the upbeat LinkedIn or viral X styles. v5 added the four casual human sources, grew the AI side to 26,227 texts, and added those social styles.
| Measure | v4 | v5 |
|---|---|---|
| Human news, food blogs, Parliament, chat flagged | 0.1 to 0.4% | 0.2 to 0.5% |
| Human Reddit posts flagged | 0.7% | 0.5% |
| Human work email flagged | 1.3% | 1.6% |
| Human GitHub discussion flagged | 2.0% | 1.2% |
| Human web pages flagged | 2.2% | 1.8% |
| AI from eight open models caught | 94% | 97% |
| AI from the Claude family caught | 82% | 95% |
| AI from the GPT family caught | 65% | 96% |
| Short AI social posts caught | 82% | 96% |
| Heavy rewrites / light polish flagged | 50% / 9% | 56% / 11% |
| Validation AUROC, short / long text | 0.9729 / 0.9928 | 0.9828 / 0.9948 |
| Robustness groups over the 2-point limit | 1 | 0 |
Source: sealed test results for both models, summarized on the model card. In v4, Reddit, email, GitHub, and web text were test-only sources; in v5 they are also training sources, so their v5 rates measure unseen texts from familiar sources. AI test sets also differ between versions (for example, 286 against 659 provider API texts).
Read the human rows with that note in mind. Reddit, GitHub, and web false positives went down, and work email went up slightly, from 1.3% to 1.6%. The formal sources rose by a few tenths of a point, because the new cutoffs spread the 1% target across eight sources instead of four. The large change is on the AI side: the GPT family went from 65% to 96%, and short AI social posts from 82% to 96%.
What we would tell anyone training an AI text detector
- Date-cut the human side. Only text written before November 2022 is certainly human. Newer news, Parliament, and food blog text trained at half weight here, and newer casual text not at all.
- Test on human sources you did not train on. v3 looked fine on its own training sources and flagged 7 to 20% of new human writing.
- Match the AI side to each kind of human writing. Most of the 26,227 AI texts were seeded by a human text of the same kind, so the model cannot learn "email versus essay" instead of "human versus AI".
- Set cutoffs from false-positive targets per source. Short and long text need different cutoffs (92% and 70% at Balanced), and counting each source equally kept every source at or under 1.8%.
- Watch what label smoothing does to your margins. At 0.05 it piled v4's scores near 3.66, where no cutoff could separate a confident human post from AI text.
- Hold out whole model families. Gemini, Granite, and three more families were never trained on, which is the only way to see how the detector handles a new model.
Limits
Botfilter skips text under 25 words. It was trained and tested on English only; it still scores other languages, and its false-positive rate there is unmeasured. Short AI posts (25 to 80 words) are caught 91 to 96% of the time at Balanced. Text a person and a model both worked on is often missed. We have no licensed human X or LinkedIn posts to test on, so the false-positive rate on those sites is unmeasured; old Reddit, chat, and email are the closest casual writing we could measure. No licensed essays by non-native English writers were available either, so that gap is unmeasured too. A mark is a probability about the text, not proof of who wrote it.
If you want the short version of what Botfilter does and what the modes look like on X, LinkedIn, Gmail, and news sites, start with Introducing Botfilter. If you want to know why a 23.6 MB model can be enough, read what on-device AI means.
Botfilter is in development, and its Chrome Web Store listing is coming soon. The install section on the home page will have the Chrome link when it is live.
Attribution: Contains Parliamentary information licensed under the Open Parliament Licence v3.0. Trained on CC BY 4.0 news articles from the Common Pile news collection and their publishers, CC BY 3.0 posts by Foodista contributors, Anthropic HH-RLHF (MIT), the Webis TL;DR corpus (CC BY 4.0), the Enron email corpus (made public by FERC), Common Pile GitHub discussions from permissively licensed repositories, and FineWeb by Hugging Face (ODC-BY 1.0).