Are AI detectors accurate? What the evidence actually shows

Are AI detectors accurate? Some are accurate on the kinds of text they were tested on, and a few are now very good, but a single headline percentage almost never tells you how a tool will do on your text. Accuracy is really two numbers: how much AI writing a detector catches, and how often it wrongly flags a person. Below we explain both, look at what a regulator and independent researchers found, and list the questions that test any detector's claim.

Accuracy is two numbers, not one

Every AI detector can make two kinds of mistake. It can miss AI-written text (a false negative), or it can flag human writing as AI (a false positive). The share of AI texts it catches is the detection rate, also called the true positive rate. The share of human texts it wrongly flags is the false positive rate.

The two pull against each other. A detector turns each text into a score, and someone picks the cut-off above which the tool says "AI". Lower the cut-off and it catches more AI text, but it also flags more people. Raise it and the false accusations drop, and so do the catches. A vendor can quote a flattering "accuracy" simply by picking the cut-off, and the test set, that suit its product.

The false positive rate matters more than it looks, because of scale. Vanderbilt University did the arithmetic in August 2023, when it switched off Turnitin's AI detector. Turnitin had said at launch that its false positive rate was 1%. Vanderbilt had submitted 75,000 papers the year before, so a 1% rate could have meant around 750 papers wrongly labelled as partly AI-written.

A third measure, AUROC, sums up how well a detector's scores separate human from AI text across every possible cut-off. It runs from 0.5 (a coin flip) to 1.0 (perfect separation). It is handy for comparing detectors, but it hides the one cut-off that decides what a user actually sees.

The Workado case: 98% claimed, 53% measured

In April 2025 the US Federal Trade Commission took action against Workado, the company behind a product called AI Content Detector. Workado had promoted it as 98 percent accurate at telling AI text from human text. According to the FTC's complaint, independent testing put its accuracy on general-purpose content at 53 percent. Chris Mufarrige, director of the FTC's Bureau of Consumer Protection, said the product "did no better than a coin toss."

The complaint also pointed to a likely cause. Workado said the detector had been built from a wide range of material, including blog posts and Wikipedia entries. The FTC alleged that the model underneath had been trained or fine-tuned only to classify academic content. A detector can look excellent on the one genre it learned and fail on everything else.

The final order, approved in August 2025, bars the company from making effectiveness claims about AI detection products unless it has competent and reliable evidence to back them. That is a fair standard for any detector, this one included. The more useful question is: accurate on what, and measured by whom?

What independent studies have found

Research on AI detection has moved quickly, and the older and newer findings point in somewhat different directions. Read together, they show that results depend heavily on the tool, the genre, the length of the text and whether anyone tried to disguise it.

Early tools were unreliable, and unfair to some writers

In 2023, Stanford researchers led by Weixin Liang ran seven popular detectors on 91 TOEFL essays written by people learning English. On average the detectors flagged 61.22% of those human essays as AI-generated, while handling essays by US eighth-graders almost perfectly. The same year, Debora Weber-Wulff and colleagues tested 14 tools, including Turnitin and PlagiarismCheck, and concluded that they were neither accurate nor reliable, with a lean towards calling AI text human. We look at the non-native finding closely in a separate article.

Detectors stumble on conditions they weren't trained for

The RAID benchmark, presented at ACL 2024 by Liam Dugan and colleagues, contains more than 6 million generations from 11 models across 8 domains, with 11 kinds of adversarial attack and 4 decoding strategies. The authors found detectors were easily fooled by those attacks, and that performance dropped when sampling settings changed or when the text came from a model the detector had not been trained on.

The best commercial tools have improved a lot

Two more recent studies are more encouraging about the top of the market. In a 2025 working paper for the University of Chicago's Becker Friedman Institute, Brian Jabarian and Alex Imas tested Pangram, OriginalityAI, GPTZero and an open-source RoBERTa model. They used 1,992 passages across six everyday genres, from news and blogs to restaurant reviews and résumés, with AI text from GPT-4.1, Claude Opus 4, Claude Sonnet 4 and Gemini 2.0 Flash. On medium and long passages, Pangram had essentially zero false positives and false negatives in their sample, and it held up against a humanizer tool. OriginalityAI and GPTZero formed a second tier with partial strengths. The RoBERTa baseline was judged unsuitable for high-stakes use.

At ACL 2025, Jenna Russell, Marzena Karpinska and Mohit Iyyer compared detectors with human readers on 300 non-fiction articles. Pangram caught 98.0% of the AI articles with a 2% false positive rate, and GPTZero caught 85.3% at 0.7%. Binoculars, a zero-shot method, caught 40.0% at its strict setting. A panel of five people who often use AI for their own writing, voting together, got only one of the 300 articles wrong.

Later work, including a 2026 study of long academic papers and a 2026 test against a commercial humanizer, is covered tool by tool in how accurate is Pangram and how accurate is GPTZero.

So the honest summary is this: some detectors are now accurate under the conditions researchers tested, and many are not. Short texts, unfamiliar genres, newer models and deliberate rewriting are where the errors gather.

Where our own detector fits

Our detector is tested on a frozen set of human and AI texts, scored exactly the way your browser scores them. Its cut-offs are set per length band and capped so that few human texts are flagged, and the outcome is Human, Mixed, AI or Inconclusive rather than a bare percentage. How it is tested is set out on our methodology page.

What it offers is a free first check: no account, no word limit, every word scanned rather than a sample, and your text never leaves your browser. The report adds a sentence map and an AI-likeness score, so you can see which passages read as most AI-like. Like any detector, it can be wrong in both directions, so treat a result as a reason to look closer, not as proof.

How to judge any detector's accuracy claim

Whether you're choosing a tool or looking at a result someone has shown you, these questions separate solid claims from marketing:

• Are both rates given? A detection rate without a false positive rate, or the reverse, is half an answer.
• Which genres and lengths? Essays, fiction, emails and abstracts behave differently, and short texts are harder for every tool.
• Which model versions? A rate measured on one model says little about the next one. We explain why in a piece on the newest models.
• Was the test data kept apart? Thresholds tuned on the same texts used for testing will look better than they are.
• Who ran the test, and can you see the data? Independent studies and open benchmarks are worth more than a vendor's own figure.
• How big was the sample? A rate measured on 40 texts can swing by many points.

So, are AI detectors accurate enough to use?

For a first look, yes, as long as you know the tool's error rates and treat the result as one piece of evidence among several. For a grade, a job or a publishing contract, no detector score should stand alone. Drafts, version history, the writer's other work and a direct conversation tell you things a classifier can't. If you are on the receiving end of a flag, our guide to being wrongly flagged by an AI detector walks through what to do next.

Sources

1. FTC: FTC Order Requires Workado to Back Up Artificial Intelligence Detection Claims (28 April 2025), checked October 2026
2. FTC: FTC Approves Final Order against Workado, LLC (28 August 2025), checked October 2026
3. Jabarian and Imas, Artificial Writing and Automated Detection, BFI Working Paper 2025-116 (August 2025), checked October 2026
4. Russell, Karpinska and Iyyer, People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text (ACL 2025), checked October 2026
5. Dugan et al., RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors (ACL 2024), checked October 2026
6. Liang et al., GPT detectors are biased against non-native English writers (Patterns, 2023), checked October 2026
7. Weber-Wulff et al., Testing of Detection Tools for AI-Generated Text (International Journal for Educational Integrity, 2023), checked October 2026
8. Vanderbilt University: Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector (16 August 2023), checked October 2026