When Princeton student Edward Tian released GPTZero in early January 2023, he explained it with two terms: perplexity and burstiness (NPR). Those words have followed AI detection ever since, in vendor explainers, news coverage, and student forums. They are usually defined in one line each, which leaves out the parts that matter most: what the measures assume, and where they break down.

What perplexity measures

Perplexity is a standard quantity from language modeling. When a language model reads text, it assigns a probability to each word (technically, each token) given the words before it. Perplexity summarizes those probabilities across the passage. Low perplexity means the model found the text predictable. High perplexity means the model was frequently surprised.

It became a detection signal because of how language models generate text. Models tend to choose high-probability words, which is part of why their output reads smoothly. Text produced that way tends to look predictable to a similar model, so its perplexity is low.

Human writing is often less predictable: an unusual word chosen for feel, a personal idiom, a sentence that turns somewhere unexpected. A model reading that text is surprised more often, so perplexity is higher. The detection logic follows: unusually low perplexity is treated as a hint of machine generation.

One qualification matters a lot. Perplexity is always measured relative to a particular reference model. The same passage can look more or less predictable depending on which model does the measuring, and a detector's reference model may not match the model, if any, that produced the text.

What burstiness measures

Burstiness is a looser idea. In detection discussions it usually refers to variation in sentence length and structure across a passage, and there is no single standard formula. One simple way to quantify it is the coefficient of variation of sentence lengths: the standard deviation divided by the mean. A paragraph that mixes a six-word sentence with a forty-word one scores high. A paragraph of similar-length sentences scores low.

The intuition is that human prose varies because ideas arrive in different sizes: a claim in a few words, an explanation that runs long, a short aside. Tian's original framing was that people write in bursts while language models are more consistent.

Why early detectors used them

They are cheap to compute. A perplexity-style score needs only a reference language model, not a large labeled set of confirmed human and AI texts. Burstiness needs little more than sentence boundaries.
They are easy to explain. A named measure is easier to describe to a student or a review panel than an opaque classifier score.
They tell an intuitive story. Early chatbot output was widely described as smooth and even, which is exactly what these measures are designed to pick up.

Where the measures break down

Both measures describe style, not origin. Anything that makes human prose more predictable or more even can push it toward an "AI-like" score: a narrow working vocabulary, standard technical terminology, formal academic conventions, or heavy editing.

Second-language writers. The clearest evidence is a 2023 Stanford study published in Patterns. Seven detectors available at the time labeled, on average, 61% of 91 human-written TOEFL essays as AI-generated, while essays by US eighth-graders were classified far more accurately. When the researchers enriched the essays' vocabulary, the false-positive rate dropped sharply, and they tied the effect to perplexity (Liang et al., 2023). More in what the Stanford study found.

Famous and widely copied text. Language models see some texts many times during training, so those texts become very predictable to them. Pangram, an AI-detection company that uses a different approach, has argued that this is why perplexity-based detectors can classify the Declaration of Independence as AI-generated (Pangram, 2025). That is a vendor's critique of a competing method, but the point follows directly from how perplexity is defined.

Mismatched models. Because perplexity depends on the reference model, a detector measuring with one model may misjudge text from another. As text generators change, a fixed reference can drift out of step.

Detection is hard in general, not only for these measures. OpenAI's own AI text classifier, a trained classifier rather than a perplexity tool, was withdrawn in July 2023 because of its low rate of accuracy. In OpenAI's evaluation at launch, it labeled 26% of AI-written text as "likely AI-written" and mislabeled human-written text as AI-written 9% of the time (OpenAI).

What detectors rely on now

Many commercial detectors now rely on classifiers trained on labeled examples rather than on perplexity and burstiness alone, and most do not publish their full methods. GPTZero, for example, says it no longer relies on perplexity and burstiness for its main detection and now uses a deep-learning model, with those measures as only some of its indicators (GPTZero). Turnitin does not publish the details of its model, but its guidance says the score can misidentify text, shows no percentage for scores between 1% and 19% because of more false positives in that range, and says the score should not be the sole basis for adverse action against a student (Turnitin guides).

Trained classifiers have limits of their own. The RAID benchmark (ACL 2024) found that a range of open-source and commercial detectors were easily fooled by adversarial attacks and by generative models they had not seen (Dugan et al., 2024).

Why this matters if you are flagged

Knowing what these measures do clarifies what a flag can and cannot show. A passage with low perplexity and even rhythm looks statistically like some machine output. That is not a finding about who wrote it. Process evidence, such as version history, outlines, notes, and earlier drafts, speaks to authorship directly in a way no score does. If you are dealing with a flag now, see why AI detectors can flag your own writing.

How HumaText's own check relates

HumaText's free writing-pattern check uses a related but simpler set of local signals: sentence-length variation, average sentence length, vocabulary variety, density of stock phrases, and contraction rate. It does not compute perplexity. Its score is an uncalibrated pattern estimate, not a probability, not authorship detection, and not a prediction of what any other detector will report. Use it as a prompt to reread a passage. Details are on the writing-pattern check page.

If your course or workplace permits AI-assisted editing and a passage reads stiff, HumaText can also propose a phrasing pass on a short section. Review every change against what you meant to say.

Sources

"A college student created an app that can tell whether AI wrote an essay," NPR (January 2023) — npr.org
"What is perplexity and burstiness for AI detection?" GPTZero — gptzero.me
Liang et al. (2023), "GPT detectors are biased against non-native English writers," Patterns — arxiv.org/abs/2304.02819
"Why perplexity and burstiness fail to detect AI," Pangram (2025) — pangram.com
"New AI classifier for indicating AI-written text," OpenAI (2023, with July 2023 update) — openai.com
Turnitin guides, AI writing detection — guides.turnitin.com
Dugan et al. (2024), "RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors," ACL 2024 — arxiv.org/abs/2405.07940

What Perplexity and Burstiness Mean in AI Detection, and Where They Break Down