Meta pauses work with Mercor after leak of AI model training data

Mercor, a leading data supplier to the AI industry, is at the center of a serious security incident. Several of the largest AI labs have opened internal investigations, and Meta, one of the company's key clients, has announced it is suspending cooperation. Confidential data on AI model training methods may have been put at risk: information that technology companies guard closely as their main competitive asset.

Mercor is a platform that connects AI companies with thousands of data labeling and annotation specialists around the world. Labeled data, meaning carefully selected texts, dialogues and images tagged with quality labels, underpins the training of modern language models. Without quality labeling, neither GPT-4, nor Claude, nor Llama would exist. Mercor served leading players in the market and over several years became one of the most prominent vendors in this segment.

The key question is what exactly may have been exposed. This is not simply a leak of a customer database or personal data. Annotator instructions, data categories and preference schemes (the RLHF labels by which models learn to give desired answers) all indirectly reveal a given company's methodological decisions. Developing such processes costs hundreds of millions of dollars and requires years of accumulated expertise. In terms of value, compromising this data is comparable to a source code leak.

Meta reacted quickly and preventively, suspending work with Mercor until the full picture of the incident is clear. This is standard protocol when a supply chain compromise is suspected: continuing to hand sensitive data to a supplier whose security status is unknown is an unjustified risk. That is all the more so given that Meta is investing tens of billions in its own AI systems, including the open Llama model family and the Meta AI assistant.

Other AI labs that worked with Mercor are also running their own checks. It has not yet been established what exactly was compromised, to what extent, or whether the incident resulted from an external cyberattack or an internal security error. Neither Mercor nor the companies involved have disclosed technical details of what happened.

The case exposes a systemic vulnerability in the AI industry. Mass outsourcing of data labeling means that dozens, if not hundreds, of intermediary companies take part in the production chain of every major AI model. Each of them gets access to fragments of its clients' confidential methodology. Meanwhile, there are no unified industry security standards for such vendors: no mandatory audits, no encryption requirements, no incident notification protocols.

For Mercor, this is a reputational crisis. The company's business rests entirely on the trust of AI labs, and that trust is now in question. Even if the investigation shows that the actual damage was limited, the incident itself and Meta's public reaction will change the company's negotiating position in the market.

This incident should accelerate the discussion of mandatory security standards for data providers. Training methodology is the key competitive weapon in the AI race. Treating data vendors as ordinary contractors is no longer possible: the level of vetting and oversight must match the level of access to confidential information.

The scale of the incident and the full list of affected companies have not yet been disclosed. Details of the investigation will become known in the near future.

*Meta has been designated an extremist organization and is banned in Russia.