AI 101 — What is an AI detector?
An AI detector is software that reads a piece of writing and returns a guess about who produced it — a person or a language model — usually as a score between 0 and 1. The guess is a probability wearing the costume of a verdict, and that gap is where nearly all of the trouble with these tools lives.
There are three different things people mean when they say "AI detector," and mixing them up is the root of most bad arguments about them.
The first is a statistical scorer. It does not look for a signature; it measures style. Language models pick likely next words, so their sentences are smooth, evenly paced and low in surprise — a property called low perplexity. Human writing is lumpier: odd word choices, uneven sentence lengths, a sudden three-word sentence after a forty-word one. Turnitin, the plagiarism-checking company used by thousands of schools, describes its own detector this way: it splits a submission into segments of a few hundred words, scores each sentence from 0 to 1, and averages the scores, using classifiers trained on the difference between machine-like and human-like word probabilities, per Turnitin's own technical explainer. No part of that process checks whether an AI was actually in the room.
The second is a trained classifier: a language model fine-tuned on pairs of human and AI answers to the same prompts, which then judges new text. This was OpenAI's approach. Its classifier, launched in January 2023, caught 26 percent of AI-written text while wrongly flagging human-written text 9 percent of the time on the company's own challenge set, OpenAI reported at launch. The company withdrew the tool that July, citing its low accuracy.
The third is provenance: a watermark or metadata tag inserted by the model that generated the text. Nothing is inferred from the words — the signal is planted at generation time and read back later, which is why it works when the vendor cooperates and fails after someone retypes the passage.
Why it matters right now. Teachers, editors and hiring managers are being handed these scores as evidence, and the scores do not carry the weight the workflows put on them. The most careful study of the failure mode came from a Stanford team, which ran seven widely used detectors over 91 essays written by non-native English speakers for the TOEFL exam and 88 essays by US eighth-graders. The detectors were near-perfect on the US essays. On the non-native ones they flagged more than half as AI-generated, an average false-positive rate of 61.22 percent, the authors reported in the journal Patterns. Their explanation was blunt: writers with a narrower vocabulary produce more predictable text, and predictable text is what the detectors were built to punish.
That is not a bug anyone can patch out with a bigger model. A separate team showed in 2023 that recursively paraphrasing AI text drops detection rates sharply while barely hurting the writing, and proved that the best possible detector is bounded by how distinguishable human and machine text are in the first place — a gap that shrinks as models learn to imitate us, per their paper in Transactions on Machine Learning Research. Detection is a moving target that the target controls.
Meanwhile the policy world has started mandating labels instead. Pennsylvania's AI advertising disclosure law takes effect December 21, with fines capped at $3,000, as we covered when it passed — a labeling regime, not a detection one, because lawmakers have noticed that the second thing does not reliably work.
The mental model. Think of a detector as a smoke alarm, not a witness. It measures a proxy — word predictability — and reports it with a confidence number that looks like evidence but is only a threshold someone chose. Change the threshold and the same essay flips from "100 percent AI" to "mostly human." Nothing about the text changed. So the useful question is never "what score did I get," it is "which tool, which version, what threshold, and what is its measured false-positive rate on writing like mine." Almost no institution that deploys these tools can answer that last part.
The analogy. Picture a border officer who decides nationality by ear. A native speaker sails through. Someone who learned the language well but in a narrow register — a textbook, a single workplace, one region's idioms — sounds "off," and gets pulled aside. The officer is not lying about what they heard; they are answering a different question than the one they were asked, and the cost lands on the person with the accent. Then add the second half: anyone can practice an accent. A few passes through a paraphrasing tool and the AI text walks through the same gate, which is exactly the asymmetry the research found.
Common misconceptions. "A high score proves it" — no, it reports a probability, and the one vendor that published its numbers withdrew its tool at 26 percent recall. "Detectors are too unreliable to matter" — also wrong: they are unreliable in a specific direction, over-flagging plain, formal, non-native writing, which is the direction that costs someone a semester or a job. "Short answers are fine" — the opposite; OpenAI said its classifier was very unreliable below 1,000 characters. "Watermarking solves it" — watermarking is the stronger approach, but it covers only text from a model that agreed to embed the mark, and it degrades under editing.
Where to learn more. Read OpenAI's launch post for the honest version of the numbers, including the limitation list, and the Patterns study for how the bias shows up in practice. Then, if a detector is ever pointed at your writing, ask the four questions above in writing. An institution that cannot answer them is not applying evidence — it is applying a vibe with a percentage attached.
Has an AI detector ever flagged your writing — and did anyone let you appeal it? Write to us via our contact page.
Related reading: AI 101 — What is AI watermarking? · AI 101 — What is AI slop?
