Canada demands a safety plan from OpenAI after incident involving a teen shooter

Canadian government has issued OpenAI a strict demand: the company must present a detailed security plan after it became known that a teenager from British Columbia used ChatGPT to rehearse violence scenarios — and OpenAI did not deem it necessary to notify law enforcement. The case, as reported by Bloomberg, puts a question before the entire artificial intelligence industry that no one yet has a convenient answer to: where does user privacy end and the obligation to prevent real threats begin?

The details of the incident have not been fully disclosed, but the following is known. A teenager from British Columbia systematically conducted dialogues with ChatGPT in which he rehearsed scenarios involving violence. OpenAI's moderation systems apparently detected this activity — however, the company decided not to pass the information to the Canadian police. Subsequently, the teenager turned out to be connected to an actual shooting incident. Canadian authorities viewed OpenAI's silence as a serious failure and are now insisting that the company propose "concrete" measures to prevent a recurrence.

To understand the scale of the problem, it is worth recalling the context. This is far from the first case where ChatGPT ends up at the center of a scandal related to minor safety. In 2024 in the US, the mother of a teenager sued Character.

AI after her son took his own life — according to her, the chatbot encouraged his suicidal intentions. In the same year, several school districts in America and Europe restricted students' access to generative AI tools. OpenAI has repeatedly updated its security policy, introduced age restrictions and content filters.

But the Canadian incident shows that technical filters are not enough — the issue comes down to a fundamental decision: should an AI company act as an informant to law enforcement?

Legally, the situation is extremely ambiguous. In most jurisdictions, the obligation to report potential threats falls on specific categories of professionals — doctors, psychologists, teachers. Technology companies are generally not on this list. Social networks like Meta and Google have long since established protocols for interaction with police, especially in cases involving child exploitation. But generative AI is fundamentally different territory. A user conducts a private dialogue with a model, and the content of this dialogue is formally protected by privacy policy. OpenAI probably found itself facing a classic dilemma: violate user privacy for the sake of potential safety or comply with data protection obligations. The company chose the latter — and now faces the consequences.

The position of the Canadian government, however, is also not without complications. The demand for "concrete steps" sounds resolute, but what exactly does it mean? If Canada wants OpenAI to automatically pass data to police about users whose requests contain descriptions of violence, this creates a precedent for mass surveillance. Millions of people daily discuss with ChatGPT plots from books, films, video games in which violence appears. Distinguishing a creative request from a real threat is a task that even live psychologists do not always manage, let alone algorithms. If we are talking about manual moderation of suspicious cases, the question of scale arises: ChatGPT processes hundreds of millions of requests per day.

For OpenAI, this incident creates a reputational and strategic risk extending far beyond Canada. The company is actively expanding its presence in international markets, and every government will now ask the same question: what safety guarantees can you provide? The European AI Act already sets strict requirements for high-risk systems. The Canadian precedent may push other countries to develop special protocols obligating AI companies to cooperate with law enforcement in certain situations. This, in turn, will require OpenAI, Anthropic, Google and other players to create entire divisions dealing with compliance in dozens of jurisdictions simultaneously.

There is also a deeper question that this case poses to society. Generative AI models have become a space for self-expression for millions of people — especially young people — sometimes the most candid space available. Teenagers tell chatbots things they dare not tell parents or friends. If AI companies begin passing these conversations to police, trust in the technology will collapse. If they do not — they risk becoming accomplices to tragedies. A golden mean here may not exist, and that is precisely why the Canadian incident will become a turning point for the entire industry. The response that OpenAI presents to Ottawa will set the standard — or show that no standard exists yet.