Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableOpenAI Blog→ original

OpenAI Releases a Free CSAM Detector: It Flags 48% Fewer Images for the Same Findings

CSAM, or child sexual abuse material, is something any platform that hosts user images has to find and remove. OpenAI has released a classifier that detects such images, including AI-generated ones, and is giving it for free to vetted organizations through its API. In testing, it flagged 48% fewer images than the company's previous safeguards while finding almost as many violations. Moderators see less of the most harmful content.

AI-processed from OpenAI Blog; edited by Hamidun News
OpenAI Releases a Free CSAM Detector: It Flags 48% Fewer Images for the Same Findings
Source: OpenAI Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceOpenAI Blog ↗
Editor-in-Chief
Key metric #1The share of real violations among flagged images roughly doubled
Falmoderators confirmed 72% of high-confidence hits as new CSAM
SafetyKitthe model found 90% of drawings and animation that sexualize children

CSAM, or child sexual abuse material, is material depicting the sexual abuse of children. Any service where users upload images or generate them with a neural network sooner or later encounters it. The problem is not only legal. Human moderators are forced to look at these images, and a poor filter buries them in false positives.

On November 16, 2026, OpenAI introduced a set of measures, the main one being a new image classifier. It flags 48% fewer images than the company's previous safeguards while finding nearly the same number of violations. To put it concretely: if the old filter sent moderators 1,000 images, the new one would send about 520, and the number of real violations among them would be the same. The classifier is available free of charge to vetted organizations.

What the classifier does and who gets it

The model analyzes images and looks for visual signs of abuse. It targets new images the system has not seen before. This matters because classic hash databases only catch files that are already known. The classifier does not return a yes-or-no verdict. It returns scores, and reviewers use them to decide which images to look at first.

OpenAI calls it the first widely available CSAM classifier that separately indicates whether material may have been generated by AI. The model was trained and evaluated with the participation of NCMEC, the U.S. National Center for Missing & Exploited Children, which runs the CyberTipline, the single channel for reporting child exploitation online.

Vetted organizations will get access through the OpenAI API, free of charge. To spread the tool, the company brought in the Tech Coalition and ROOST. Organizations apply through a form. The model comes with a model card.

According to OpenAI itself, on user-uploaded images the new classifier would flag 48% fewer images than the current safeguards and find roughly the same number of violations. Among the flagged images, real cases turned up about twice as often. Early-access partners report their own figures:

  • fal: on production images checked by humans, moderators confirmed 72% of high-confidence hits as new CSAM;
  • invideo AI: one in four flagged cases ended in a ban;
  • SafetyKit: on drawings and animation sexualizing children, the model found 90% of cases.

Warnings in ChatGPT

The second part concerns the chatbot itself. Requests for CSAM were already prohibited in ChatGPT, and the models are trained to refuse them. Now warnings are being added that interrupt attempts to search for such material. They explain the consequences and direct the person to specialists who help stop harmful behavior. The text states directly that sexualizing children causes harm, even when the material is AI-generated.

Next comes a ladder of measures. The conversation is deleted, and access to ChatGPT is suspended for 24 hours. For repeated violations, restrictions get longer, and account closure and reporting to authorities are possible. The approach was developed with Protect Children, the Lucy Faithfull Foundation, and academic advisers from the University of Kent and the University of Tasmania.

What this means for developers

OpenAI published a practical guide to CSAM: how to prevent abuse, detect content, and take action. It was prepared with the participation of the Tech Coalition, Thorn, NCMEC, and the Internet Watch Foundation. It describes:

  • product usage rules;
  • ways users report problems;
  • detection tools;
  • the process for reviewing and forwarding reports;
  • safety identifiers that help link abuse to a specific user;
  • support for the moderators themselves.

The guide is meant to be used together with the Moderation API, the ROOST Coop review console, and specialized services such as Thorn Safer. Separately, OpenAI, Anthropic, and Google commissioned recommendations on how AI systems should respond when users disclose violence. The Tech Coalition published them. According to the company, after the guide was published, more developers began connecting child safety tools and registering to send reports to NCMEC.

OpenAI offers law enforcement investigating such cases training and AI tools. The first tool helps analyze long conversations and responses to legal requests, find connections, and build timelines with linked sources. Access depends on approval from the agency.

Limitations

All figures come from tests run by OpenAI and its early partners. The article contains no independent verification. The samples and methodology are not disclosed in detail; they should be checked in the model card. Access is limited to vetted organizations, so a small startup cannot connect the model without its application being approved.

The practical takeaway for product teams: if you offer image uploads or image generation, plan the chain in advance, covering the filter, the review queue, reporting, and care for moderators. The guide and the free model make that path considerably cheaper.

Story

Key Questions Answered

What the classifier does and who gets it?

The model analyzes images and looks for visual signs of abuse. It targets new images the system has not seen before. This matters because classic hash databases only catch files that are already known. The classifier does not return a yes-or-no verdict. It returns scores, and reviewers use them to decide which images to look at first. OpenAI calls it the first widely available CSAM classifier that separately indicates whether material may have been generated by AI.

What are the key details on Warnings in ChatGPT?

The second part concerns the chatbot itself. Requests for CSAM were already prohibited in ChatGPT, and the models are trained to refuse them. Now warnings are being added that interrupt attempts to search for such material. They explain the consequences and direct the person to specialists who help stop harmful behavior. The text states directly that sexualizing children causes harm, even when the material is AI-generated. Next comes a ladder of measures.

What this means for developers?

OpenAI published a practical guide to CSAM: how to prevent abuse, detect content, and take action. It was prepared with the participation of the Tech Coalition, Thorn, NCMEC, and the Internet Watch Foundation. It describes: product usage rules; ways users report problems; detection tools; the process for reviewing and forwarding reports; safety identifiers that help link abuse to a specific user; support for the moderators themselves.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…