How the detector works, and how we test it

Model card

The detector runs in two tiers, both inside your browser:

• Instant: text clean-up (invisible and look-alike characters), language check, sentence splitting, and explanatory style counts. No download. These counts are shown for context and are not used in the verdict.
• Standard: a small open-licence classifier, chosen by the benchmark below, scores overlapping windows of the text. A smoother joins the window scores into passages. This tier drives the verdict.

How a verdict is made

A calibrated combiner turns the model scores into a probability for the whole text and an AI fraction (the share of words in AI-like passages). Then a fixed rule decides the outcome:

• Under 50 words: no result.
• 50 to 149 words: confidence is always low, and Mixed is not allowed.
• AI when the probability clears the threshold for the text's length and the AI fraction is at least 80%.
• Human when the AI fraction is 10% or less.
• Mixed when at least one high-confidence AI passage is 120 words or longer.
• Inconclusive otherwise, and for text that isn't English.

Cut-offs are set per length bucket so that at most 5% of the calibration human texts are flagged. The same calibration gives the AI-likeness score: the share of measured human texts of similar length that scored lower. It is shown capped at 99% either way, since the calibration texts are a sample of human writing, not all of it.

Data sources

The frozen test set mixes openly licensed human writing with AI-written text from two sources. AI stories come from StoryScope, a public research dataset whose authors generated stories with several commercial chatbots; we did not generate those ourselves. Essays, emails, blog posts, news, abstracts and humanized rewrites are samples we wrote with Claude. What the test set contains, by kind of text, is listed below; its limitations are listed with the results.

Metrics

Detected at a 5% false-positive cap

Share of AI texts flagged (as AI or Mixed) by the shipped cut-off, which is set so that at most 5% of calibration human texts are flagged. The v1 baseline is scored at its own 50% mark.

False-positive rate

Share of human texts labelled AI at the shipped threshold. A Mixed result on a human text counts as a false positive.

AUROC

How well scores separate human from AI across all thresholds, from 0.5 (coin flip) to 1.0 (perfect).

Every metric is reported with a 95% confidence interval from a bootstrap that resamples whole source documents, per domain, per AI model, per length bucket and for the humanized subset.

Benchmark results

Why leaderboard scores look higher

The model we ship, TMR, reports an AUROC of 0.99 on the RAID benchmark in its model card. On our test set it scores 0.78. Both numbers are honest; they measure different things.

• Older models. RAID's AI texts come from models released up to 2023. Our AI texts come from 2026 models, which write much more like people.
• Trained on the test. TMR was trained on RAID data, so a RAID score shows how well it learned that data. Our test texts were never used to train it, and part of the AI text was kept out of tuning the cut-offs as well.
• A strict false-positive cap. We report what is caught when at most about 5% of human texts are flagged, which is how the site behaves. A high AUROC can still mean little is caught at a cut-off that is safe for people.

Results also depend on the kind of writing. Fiction is the hardest to catch; essays, emails, blogs and news are flagged far more often, as the results by kind of text above show.

Long documents

There is no word or character limit, and every word is scored: the site never samples. The text is split into windows of 254 tokens and every window goes through the model, so a long scan takes time: about two to three minutes for 30,000 words on a typical laptop, and 15 minutes or more for a full novel. It streams partial results and can be cancelled. Above 30,000 words the text is shown in pages of 5,000 words to keep the browser responsive. Chapters are found from headings, with a menu to jump to each one.

Narrative fingerprint

For fiction, the result can include descriptive narrative features, such as dialogue share, smell imagery, narrator thematic commentary, direct reader address and time-jump markers, computed with our own word lists. Each is shown beside the human and AI base rates published in StoryScope: Investigating idiosyncrasies in AI fiction (Russell, Rajendhran, Pham et al., arXiv:2604.03136, 2026). These features are heuristics inspired by that paper, not a reimplementation of it, and they are never used in the verdict.

Limitations

• No detector is proof. A result should never be the only evidence against anyone.
• Formal, formulaic, non-native and heavily edited writing is the most likely to be flagged wrongly.
• Paraphrasing (“humanizer”) tools can hide AI text; the humanized subset of the benchmark measures how much.
• English only. Other languages get Inconclusive.
• New AI models write differently; the benchmark is re-run whenever the detector changes.
• Style counts (em dashes, stock transitions and the like) change quickly with model fashions. They are explanations only.

Changelog