Are AI detectors biased against non-native English writers?

There is good evidence that AI detectors flag non-native English writing more often than writing by native speakers. A 2023 Stanford study found that seven popular detectors labelled, on average, more than 60% of essays by English learners as AI-generated. The authors traced it to plain, predictable wording, which many detectors read as a sign of AI. Below we go through the study, why it happens, and what writers and teachers can do about it.

What the Stanford study found

The study, by Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu and James Zou, was published in the journal Patterns in 2023. The design was simple. The researchers took 91 essays written by people preparing for the TOEFL English exam, collected from a Chinese education forum, and 88 essays by US eighth-graders from a public dataset. All of them were written by people. They then ran both sets through seven widely used detectors: Originality.AI, Quil.org, Sapling, OpenAI's classifier, Crossplag, GPTZero and ZeroGPT.

The detectors handled the eighth-graders' essays almost perfectly. The TOEFL essays were another story:

• On average, the detectors flagged 61.22% of the TOEFL essays as AI-generated.
• 18 of the 91 essays (19.78%) were flagged by all seven detectors.
• 89 of the 91 (97.80%) were flagged by at least one.

In other words, a student learning English who ran an essay through several detectors would almost certainly have been told by at least one of them that it looked machine-written.

Why plain and non-native English gets flagged

The authors traced the problem to a measure called perplexity. Roughly, perplexity is how surprised a language model is by each word in a text. Language models write by choosing likely words, so their text tends to have low perplexity. Many detectors use low perplexity as a sign of AI.

People writing in a second language often have a smaller working vocabulary and lean on common words and familiar sentence patterns. That makes their writing more predictable to a language model, which is exactly what those detectors are looking for. The essays flagged by all seven detectors had significantly lower perplexity than the rest.

The researchers then tested the idea directly, in both directions:

• They asked ChatGPT to improve the word choices in the TOEFL essays to sound more like a native speaker. The average false positive rate fell from 61.22% to 11.77%.
• They asked ChatGPT to simplify the word choices in the eighth-graders' essays. The average false positive rate rose from 5.19% to 56.65%.
• They also generated college admission essays with ChatGPT, then asked it to rewrite them in more literary language. The best detection rate among the seven tools fell from 100% to 13%.

Put together, the experiments show those detectors were partly measuring vocabulary range. Plain writing looked like AI. Showy writing looked human, even when a machine wrote it.

These were 2023 tools, and the authors' explanation centred on perplexity. Many current detectors, ours included, are trained classifiers, which learn their cues from labelled examples instead of measuring perplexity directly. That doesn't rule the problem out, because plainness and predictability can still be among the cues a classifier learns.

If English is your second language

A flag on your writing is not evidence that you did anything wrong, and the research above is on your side. A few practical steps help:

• Write in a tool with version history, such as Google Docs or Word with OneDrive, and keep it switched on.
• Keep your notes, outlines and sources. Evidence of your process is the strongest answer to a detector score.
• If you are flagged, it is reasonable to mention that English is not your first language and to point to the Stanford study. It is relevant context.
• Be careful about "fixing" your writing with AI tools to avoid detection. In the study, AI rewriting lowered the scores, but using AI that way may itself break your institution's rules.

Our guide to being wrongly flagged by an AI detector covers the full process, step by step.

If you teach or review non-native writers

Simple vocabulary and predictable sentence patterns are what you would expect from someone writing in a second language. A detector may read those same features as signs of AI. Vanderbilt University listed this bias among its reasons for switching off Turnitin's AI detector in 2023.

Compare a flagged piece with the student's own earlier work, especially anything written in class. Ask about their process before drawing conclusions. Treat any detector result as one piece of evidence, and a weaker one for this group than for others. Our page for teachers has more on using a detector responsibly.

What we do about it

We test our detector on plain English too: the test set includes Simple English Wikipedia as a stand-in for plain or non-native writing, and how we test is set out on our methodology page. Cut-offs are set per length band and capped so that few human texts are flagged. The verdict includes an Inconclusive outcome rather than forcing a call, and the AI-likeness score compares your text with the human writing we tested, so you can see how unusual it really is. The detector works in English only. It is free, needs no account and keeps your text in your browser, so a student can check their own work without handing it to anyone. Like any detector, it can be wrong in both directions: never use a result as the only evidence against anyone, and be especially careful with plain or non-native English.

Sources

1. Liang, Yuksekgonul, Mao, Wu and Zou, GPT detectors are biased against non-native English writers (Patterns, 2023), arXiv abstract, checked October 2026
2. Liang et al., full text (HTML version) with the experiment results, checked October 2026
3. Vanderbilt University: Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector (16 August 2023), checked October 2026