Starts Nov 23Live Cohort 3 of Vibe Coding & AI Agents: 3 weeks, 9 live hands-on sessionsCohort 3 (Nov 23) →
ReleaseInsight · On RadarMIT News→ original

AI-Text Detector Taught to Keep Learning on the Fly: 93% of Disguised Texts Caught vs. 27%

AI-text detectors can be fooled by "humanizers", programs that rewrite machine-generated text to read as human. Researchers at MIT and Cornell Tech let a deployed detector keep learning from users' fresh, unlabeled texts. It caught more than 93% of the disguised texts, while a commercial detector caught about 27%. For anti-fraud and moderation teams, the signal is clear: a model trained once needs regular updates, like an anti-spam filter.

AI-processed from MIT News; edited by Hamidun News
AI-Text Detector Taught to Keep Learning on the Fly: 93% of Disguised Texts Caught vs. 27%
Source: MIT News. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceMIT News ↗
Editor-in-Chief
Natural drift in writing style85% accuracy vs. 73% for conventional methods
Key metric #2The test-time adaptation method does not require retraining the model from scratch
Key metric #3The authors warn that text from future LLMs will be even more natural

A teacher suspects that a student's term paper was written by a neural network and runs it through an AI-text detector. The catch: the student may have passed the draft through a "humanizer", a program that rewrites machine-generated text so it looks human, and an ordinary detector will miss that disguise. Researchers at MIT and Cornell Tech propose not freezing the detector after release, but letting it keep learning from new texts while it is in use.

The approach is called test-time adaptation (TTA), meaning adaptation during deployment. In a test on disguised texts, a commercial detector found only about 27% of AI-generated texts, while the TTA version found more than 93%. If you build anti-fraud, moderation or content-verification systems, this speaks directly to your problem: an adversary will eventually get around a model that was trained once.

Why detectors break

Typically, a developer collects texts from the internet that are known to be written by a human or by an AI, and trains a model to tell them apart. After release, the model stays unchanged. From there, problems arrive from two directions.

The first is natural drift. People change topics, turns of phrase and analogies. An author writing in 2030 uses different imagery than an author in 2010. A detector trained on older texts may mistake a new human style for a machine one and wrongly accuse a real author.

The second is an arms race. Humanizers quickly appear around any popular detector. The company adds their output to its training data, new humanizers appear, and the cycle repeats. According to lead author Kevin Ren, it can go on forever.

What the authors proposed

With TTA, the model adjusts to incoming data after release. For a detector, that data is the texts users submit. These texts have no labels: nobody knows whether a human or a neural network wrote them. But they reflect how people and AI write right now, and that is exactly what the detector has to recognize. Once it has accumulated enough examples, the model finds on its own the patterns that separate the two types of text. It does not need to be retrained from scratch.

To test this, the authors assembled datasets with a known number of human and machine texts, but did not show the model the labels. They used language models to generate some of the examples specifically to evade the detector. After adaptation, the model was evaluated on data with known answers.

What they found

  • In the scenario with disguised AI texts, existing methods found about 27%, while the model with TTA found more than 93%.
  • In an experiment simulating the natural change of human language over time, the detector with TTA reached 85% accuracy vs. 73% for traditional methods.

The second task turned out to be harder: shifts in human writing are slow and subtle, and they are harder to spot than the deliberate tricks of humanizers. The company Pangram provided API credits for access to the detectors.

Limitations and practical takeaways

The authors do not promise a win. The more naturally future LLMs write, the harder it becomes to tell their text from human writing. It is also getting harder to find verified human-written texts for training. Co-author Manish Raghavan acknowledges that it is unclear whether the problem is solvable at all. He frames the question more broadly: how should we think about science, writing and art if it is impossible to tell a human from an AI?

For practitioners, the takeaways are as follows. First, a detector trained once degrades over time: the data changes, and the adversary adapts. Metrics on a static test set describe reality poorly. Second, an unlabeled stream of user texts is a resource, not junk: it can be used to tune the model without expensive manual labeling. Third, this is not only about schools and newsrooms. The method may be useful against phishing emails and SEO fraud, where the adversary is adaptive by definition.

The result should not be read as "detectors are now reliable". A detector's verdict remains a probabilistic estimate, so in high-stakes cases, such as a scientific publication, a short-story contest or hiring, it must be checked by other means. The work will be presented at the NeurIPS conference. The authors plan to look for other areas where TTA can help and to build more convenient benchmarks for dynamic evaluation of detectors.

Story

Key Questions Answered

Why detectors break?

Typically, a developer collects texts from the internet that are known to be written by a human or by an AI, and trains a model to tell them apart. After release, the model stays unchanged. From there, problems arrive from two directions. The first is natural drift. People change topics, turns of phrase and analogies. An author writing in 2030 uses different imagery than an author in 2010. A detector trained on older texts may mistake a new human style for a machine one and wrongly accuse a real author.

What the authors proposed?

With TTA, the model adjusts to incoming data after release. For a detector, that data is the texts users submit. These texts have no labels: nobody knows whether a human or a neural network wrote them. But they reflect how people and AI write right now, and that is exactly what the detector has to recognize. Once it has accumulated enough examples, the model finds on its own the patterns that separate the two types of text. It does not need to be retrained from scratch.

What they found?

In the scenario with disguised AI texts, existing methods found about 27%, while the model with TTA found more than 93%. In an experiment simulating the natural change of human language over time, the detector with TTA reached 85% accuracy vs. 73% for traditional methods. The second task turned out to be harder: shifts in human writing are slow and subtle, and they are harder to spot than the deliberate tricks of humanizers. The company Pangram provided API credits for access to the detectors.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…