Can AI detectors catch the newest models?

Can AI detectors catch the newest models? The honest answer is that nobody knows until someone measures it. A detection rate belongs to the specific model versions in a test, and a new release can write differently in ways that matter. The newest releases, including GPT-6.1 Sol, Claude Sonnet 5.5 and Gemini 4 Argon, are not in our test set, so we have no figures for them yet, and we won't guess.

A detection rate belongs to a model version

When a detector reports that it catches a certain share of AI text, that figure was measured on text from particular models, in particular genres, at a particular time. Change the model and the figure can move, even within a single study.

The clearest example we know of comes from a Becker Friedman Institute working paper by Brian Jabarian and Alex Imas, which tested detectors on AI text from GPT-4.1, Claude Opus 4, Claude Sonnet 4 and Gemini 2.0 Flash. Originality.ai separated AI from human writing well when the AI text came from Gemini 2.0 Flash, and much less well when it came from Claude Opus 4. The genres and the tool were the same; only the model differed. The authors also write that they "do not expect the results presented here to remain constant over time", and they call for regular audits with published results. The RAID benchmark, covered below, found the same weakness from another angle: detectors did worse on models they had not seen.

Genre moves the figure too. A rate measured on essays says little about fiction or academic abstracts, which is why we break our own results down by genre in our open benchmark.

This is also why we don't publish a detection rate for each model. We didn't generate most of our AI test texts. They are stories from StoryScope, a public research dataset (Russell et al., COLM 2026) whose authors generated them with five commercial models: GPT-5.4, Claude Sonnet 4.6, Gemini 3 Flash, DeepSeek V3.2 and Kimi K2.5. Each of those models appears in our test only as fiction, so a rate with one of their names on it would really be a rate for that model's stories. Our non-fiction AI samples were all written with Claude. With model and genre tangled together like this, a table of rates by model would mislead more than it informs, so we report results by genre.

Why the newest models aren't measured yet

The site shows each model family's newest release automatically. Today that list reads GPT-6.1 Sol, Claude Sonnet 5.5, Gemini 4 Argon, DeepSeek V4.1 Flash, Kimi K3 and Grok 4.7. None of these is in our test set, in any genre, so we have measured nothing at all on their text. The same goes for every Grok version.

Measuring a new model properly takes more than pasting in a few samples. You need a balanced set of its texts across genres and lengths, scored by the detector exactly as shipped, at the same cut-offs, with enough texts to give a confidence interval worth reading. Until that is done for a model, "not measured" is the only honest label, and a "not AI" result on text from that model calls for extra care.

What changes between model generations

New models don't just get better at the same things. Their habits shift, and some of those habits are the very cues detectors rely on.

Favourite words and punctuation go out of fashion

Studies by Alex Reinhart's team (PNAS, 2025) and Dmitry Kobak's team (Science Advances, 2025) measured words that one generation of models overused heavily, and the StoryScope paper (COLM 2026) notes that GPT-5.4 cut back on the em dash, long treated as a giveaway. A cue like that is a strong signal for one release and a weak one for the next. We tell that story, including what happened when this site once leaned on dashes, in em dashes don't prove AI wrote it.

Training can remove the quirks on purpose

In a 2026 study, Tuhin Chakrabarty, Jane Ginsburg and Paramveer Dhillon fine-tuned a model on individual authors' complete works. Leading detectors flagged only 3% of the fine-tuned excerpts as AI, against 97% of ordinarily prompted ones. A model trained to sound human can lose much of what detectors look for.

Some deeper habits have held up so far

Not everything fades. Reinhart's team found that grammatical differences between models and humans did not shrink as models got larger, and that instruction tuning moved output further from human writing. StoryScope found that five 2026 models clustered together in how they build stories, separately from human authors. Detectors that pick up these deeper patterns may age better than those built on word lists, but that needs checking with each release.

What research says about models a detector hasn't seen

The evidence points both ways. The RAID benchmark (ACL 2024), built from 11 models, found that detectors lost accuracy on generators they had not been trained on, as well as under different sampling settings and adversarial attacks.

On the other hand, some tools have generalised well within a study. In the same Becker Friedman Institute paper that found Originality.ai's results shifting by model, Pangram classified text from all four OpenAI, Anthropic and Google models almost perfectly. And in an ACL 2025 study, a panel of five experienced human readers caught 96.7% of articles written by OpenAI's o1-Pro reasoning model, with no false positives. Both studies are summarised in are AI detectors accurate?

Generalising to new models is possible, then. It just has to be shown, one version at a time.

How to read a claim about new models

When a detector, ours included, says it can catch the latest models, these questions are worth asking:

• Which exact model versions were tested, and when?
• Was the newest model held out from tuning, or was the detector updated with its text before testing?
• Which genres and lengths were included?
• Was a false positive rate on human writing measured at the same time?
• If the claim says "detects GPT" or "detects Claude" with no version, which version is meant?

So, can AI detectors catch the newest models?

Sometimes, partly, and provably only after testing. For our detector, we can tell you how it does by genre on the AI text in our test set, which comes from the StoryScope stories and our Claude samples, and nothing more. For newer releases, the answer today is "not measured". Every rate we publish is listed on our methodology page, which also notes that the benchmark is re-run whenever the detector changes. Until a model has been measured, treat any result on its text, from any detector, with extra caution. See also our page on detecting ChatGPT, Claude and Gemini text, and our explainer on how AI detectors work, which covers why generalising to new models is hard.

Sources

1. Dugan et al., RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors (ACL 2024), checked October 2026
2. Reinhart et al., Do LLMs write like humans? Variation in grammatical and rhetorical styles (PNAS, 2025), checked October 2026
3. Kobak et al., Delving into LLM-assisted writing in biomedical publications through excess vocabulary (Science Advances, 2025), checked October 2026
4. Russell, Rajendhran, Pham, Iyyer and Wieting, StoryScope: Investigating idiosyncrasies in AI fiction (COLM 2026), checked October 2026
5. Chakrabarty, Ginsburg and Dhillon, Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers (arXiv 2510.13939), checked October 2026
6. Jabarian and Imas, Artificial Writing and Automated Detection, BFI Working Paper 2025-116, checked October 2026
7. Russell, Karpinska and Iyyer, People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text (ACL 2025), checked October 2026