OpenAI Releases a Free CSAM Detector: It Flags 48% Fewer Images for the Same Findings
CSAM, or child sexual abuse material, is something any platform that hosts user images has to find and remove. OpenAI has released a classifier that detects such images, including AI-generated ones, and is giving it for free to vetted organizations through its API. In testing, it flagged 48% fewer images than the company's previous safeguards while finding almost as many violations. Moderators see less of the most harmful content.
AI-processed from OpenAI Blog; edited by Hamidun News
CSAM, or child sexual abuse material, is material depicting the sexual abuse of children. Any service where users upload images or generate them with a neural network sooner or later encounters it. The problem is not only legal. Human moderators are forced to look at these images, and a poor filter buries them in false positives.
On November 16, 2026, OpenAI introduced a set of measures, the main one being a new image classifier. It flags 48% fewer images than the company's previous safeguards while finding nearly the same number of violations. To put it concretely: if the old filter sent moderators 1,000 images, the new one would send about 520, and the number of real violations among them would be the same. The classifier is available free of charge to vetted organizations.
What the classifier does and who gets it
The model analyzes images and looks for visual signs of abuse. It targets new images the system has not seen before. This matters because classic hash databases only catch files that are already known. The classifier does not return a yes-or-no verdict. It returns scores, and reviewers use them to decide which images to look at first.
OpenAI calls it the first widely available CSAM classifier that separately indicates whether material may have been generated by AI. The model was trained and evaluated with the participation of NCMEC, the U.S. National Center for Missing & Exploited Children, which runs the CyberTipline, the single channel for reporting child exploitation online.
Vetted organizations will get access through the OpenAI API, free of charge. To spread the tool, the company brought in the Tech Coalition and ROOST. Organizations apply through a form. The model comes with a model card.
According to OpenAI itself, on user-uploaded images the new classifier would flag 48% fewer images than the current safeguards and find roughly the same number of violations. Among the flagged images, real cases turned up about twice as often. Early-access partners report their own figures:
- fal: on production images checked by humans, moderators confirmed 72% of high-confidence hits as new CSAM;
- invideo AI: one in four flagged cases ended in a ban;
- SafetyKit: on drawings and animation sexualizing children, the model found 90% of cases.
Warnings in ChatGPT
The second part concerns the chatbot itself. Requests for CSAM were already prohibited in ChatGPT, and the models are trained to refuse them. Now warnings are being added that interrupt attempts to search for such material. They explain the consequences and direct the person to specialists who help stop harmful behavior. The text states directly that sexualizing children causes harm, even when the material is AI-generated.
Next comes a ladder of measures. The conversation is deleted, and access to ChatGPT is suspended for 24 hours. For repeated violations, restrictions get longer, and account closure and reporting to authorities are possible. The approach was developed with Protect Children, the Lucy Faithfull Foundation, and academic advisers from the University of Kent and the University of Tasmania.
What this means for developers
OpenAI published a practical guide to CSAM: how to prevent abuse, detect content, and take action. It was prepared with the participation of the Tech Coalition, Thorn, NCMEC, and the Internet Watch Foundation. It describes:
- product usage rules;
- ways users report problems;
- detection tools;
- the process for reviewing and forwarding reports;
- safety identifiers that help link abuse to a specific user;
- support for the moderators themselves.
The guide is meant to be used together with the Moderation API, the ROOST Coop review console, and specialized services such as Thorn Safer. Separately, OpenAI, Anthropic, and Google commissioned recommendations on how AI systems should respond when users disclose violence. The Tech Coalition published them. According to the company, after the guide was published, more developers began connecting child safety tools and registering to send reports to NCMEC.
OpenAI offers law enforcement investigating such cases training and AI tools. The first tool helps analyze long conversations and responses to legal requests, find connections, and build timelines with linked sources. Access depends on approval from the agency.
Limitations
All figures come from tests run by OpenAI and its early partners. The article contains no independent verification. The samples and methodology are not disclosed in detail; they should be checked in the model card. Access is limited to vetted organizations, so a small startup cannot connect the model without its application being approved.
The practical takeaway for product teams: if you offer image uploads or image generation, plan the chain in advance, covering the filter, the review queue, reporting, and care for moderators. The guide and the free model make that path considerably cheaper.
- Nov 22, 2026Microsoft's Agent Hand-Wrote Neural Network Weights and Hit 100% Accuracy on 13 Tasks
- Nov 22, 2026LangChain, a Side Project Turned $3 Billion Startup in Four Years, Raises $300 Million
- Nov 22, 2026Android work profile closes real-IP leak that Happ and v2rayNG don't catch
- Nov 22, 2026Anthropic Goes Public on Nasdaq: Shares Rise from $83 to $111, but Can Safety Hold Up?
Key Questions Answered
What the classifier does and who gets it?
The model analyzes images and looks for visual signs of abuse. It targets new images the system has not seen before. This matters because classic hash databases only catch files that are already known. The classifier does not return a yes-or-no verdict. It returns scores, and reviewers use them to decide which images to look at first. OpenAI calls it the first widely available CSAM classifier that separately indicates whether material may have been generated by AI.
What are the key details on Warnings in ChatGPT?
The second part concerns the chatbot itself. Requests for CSAM were already prohibited in ChatGPT, and the models are trained to refuse them. Now warnings are being added that interrupt attempts to search for such material. They explain the consequences and direct the person to specialists who help stop harmful behavior. The text states directly that sexualizing children causes harm, even when the material is AI-generated. Next comes a ladder of measures.
What this means for developers?
OpenAI published a practical guide to CSAM: how to prevent abuse, detect content, and take action. It was prepared with the participation of the Tech Coalition, Thorn, NCMEC, and the Internet Watch Foundation. It describes: product usage rules; ways users report problems; detection tools; the process for reviewing and forwarding reports; safety identifiers that help link abuse to a specific user; support for the moderators themselves.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.