Fireworks AI Breach Leaks Notion AI Chat Transcripts: What Teams Should Do

You use a convenient AI assistant in a work tool and assume your data lives with the vendor. In practice, there is often another contractor between you and the model, and behind it a cloud provider. This is exactly the chain that failed in the Notion and Fireworks AI story.

What happened

Notion said that Fireworks AI, an AI infrastructure provider, leaked transcripts of Notion AI chats from a small number of workspaces. According to the company, on October 23 an attacker gained access to Fireworks' development environment through a third-party cloud provider. Resource metadata, payment information, model weights, and datasets belonging to some customers were compromised.

This is not the first warning sign. Back in early October, Fireworks had already faced a similar incident: hackers reached environment variables containing API keys and tokens for a limited number of accounts. In other words, one provider suffered two leaks within a month.

Notion's timeline looks like this:

• On November 2, Fireworks notified Notion of the exposure of a dataset that, according to Fireworks, contained only synthetic test data.
• As a precaution, Notion blocked data transfers to Fireworks, deleted everything from the provider's systems, and by November 3 stopped model training processes.
• On November 8, Fireworks confirmed that customer data, including that of Notion users, had in fact been accessed.

The initial conclusion that "it's only synthetic data" turned out to be wrong. Fortunately, Notion did not rely on it and cut off the data flow in advance.

Why chats were there at all

Fireworks powers some open-weight models inside Notion AI. It also analyzes saved conversations for the Insights feature, which lets administrators see how teams use AI. The transcripts that leaked were specifically those processed for Insights. The feature is now disabled.

According to Notion, the chats did not indicate who conducted them or which workspace they came from. The transcripts also contained no tool call results. However, the text of the messages remained, and people paste anything into a chat with an assistant: code snippets, access keys, passwords, pieces of client correspondence. Stripping metadata does not help if a secret sits right in the message text.

Under the contract, Fireworks was not supposed to store Notion's data at all. The contract prohibited it, yet the data was still found. Paper guarantees without technical verification work poorly.

What to do right now

The scale is small: fewer than 1% of users are affected. Notion advises workspace owners to open the "Security incident review" section in the desktop web version and share with members the link through which they can download their transcripts. Those affected should change passwords, API keys, and any credentials they entered into the affected chats. If neither the user nor the workspace owner received a notification, no action is required for now.

For those building AI products, the takeaway is broader than a single incident:

• Map your subcontractors. Find out whose servers prompts and responses pass through, where they are stored, and why. Analytics on top of conversations is a separate leak channel.
• Keep a kill switch. Notion was able to quickly block data transfers. Make sure you can likewise disconnect a provider without breaking your product.
• Don't take "the data is synthetic" on faith. Ask for confirmation and assume the worst case until it is ruled out.
• Don't store more than needed. If conversations are only needed for analytics, shorten the retention period and scrub obvious secrets before sending them to a contractor.
• Educate users and yourself. Keys and passwords do not belong in an assistant chat. If they were already pasted, they need to be rotated.

Limitations and context

It is unknown how much data the attackers saw and whether they used it. All that is known is that access to customer data has been confirmed. Other Fireworks customers may also have been affected. Notion speaks only about its own users, while the scale of the incident at the provider itself remains opaque.

There is a separate irony in the fact that in May Notion launched a developer platform with a Workers feature: customers write their own logic and run it in a secure sandbox, without relying on external infrastructure. The more AI agents and automations live inside a product, the more important it is to know where your perimeter ends and someone else's begins.