CRA-Bench and CRA-Net: Catching Harm That Builds Up Over the Course of an LLM Conversation

In July 2026, a team of researchers posted a paper to arXiv titled "Stateful Guardrails for Multi-Turn LLM Systems." It introduces CRA, a framework that tracks risk accumulation across an entire conversation with a language model rather than in a single message. The team also released CRA-Bench, a benchmark of 1200 labeled sessions.

Why conventional guardrails miss the harm

Most LLM safety systems evaluate each request-response pair in isolation, so they miss attacks that are assembled gradually, turn by turn. The authors call this Conversational Risk Accumulation (CRA) and identify three forms it takes: the conversation drifts smoothly away from a harmless topic, a prohibited instruction is assembled piece by piece from separate messages, and sensitive data accumulates through repeated disclosures.

"Individually harmless turns add up to harm," is how the authors put

the core of the problem in the arXiv abstract.

What the CRA Framework tracks

The CRA Framework works at the session level and monitors three signals along the conversation's trajectory. The first is semantic drift from the session's "anchor": how far the conversation has moved from its original topic. The second is an information accumulation graph built from extracted entities and weighted by their sensitivity. The third is a "compliance gradient": a signal of the model's growing willingness to fulfill increasingly risky requests.

Key facts from the paper:

• Term: Conversational Risk Accumulation (CRA) — intent drift, piecewise assembly of prohibited instructions, accumulation of sensitive information
• Three signals — semantic drift, information accumulation graph, compliance gradient
• Two scoring methods — unsupervised convex fusion and CRA-Net DA, a trainable model with family-adversarial objectives
• CRA-Bench v0.1 — 1200 sessions of 8 turns each, three threat families, with topic-matched "benign twins"
• Extended set — 2000 sessions, five families (adding persona priming and context stuffing)

For scoring, the authors propose two approaches. The first is unsupervised convex fusion, a linear combination of the signals that requires no training and makes it easy to see how much each signal contributes. The second is CRA-Net DA, a compact trainable trajectory model trained with family-adversarial objectives, so that conversation length and topic diversity do not stand in for a real threat signal.

How effectiveness was measured

The authors released three versions of the benchmark at once. CRA-Bench v0.1 contains 1200 eight-turn sessions across three threat families, and each harmful session is paired with a harmless "twin" on the same topic. CRA-Bench v0.2 uses the same scenarios, paraphrased by a different LLM to remove template artifacts. The extended set raises the total to 2000 sessions and five families.

For evaluation, the authors propose a "trajectory" protocol with session-level splits. It includes Trajectory AUROC, a turns-to-detection metric (how many turns it takes the system to spot an attack), calibrated false-positive rates, bootstrap confidence intervals, and a leave-one-family-out stress test, in which one threat family is excluded from training entirely to check generalization.

What this means

LLM safety is shifting from checking individual messages to analyzing the full trajectory of a conversation. Multi-turn attacks, in which harm is assembled from pieces that look harmless, become a distinct, measurable class of threats, with a benchmark of 2000+ sessions and metrics for catching them.

Frequently asked questions

What is Conversational Risk Accumulation?

It is the buildup of risk over the course of an LLM conversation, when messages that are harmless on their own gradually add up to harm: intent drifts, a prohibited instruction is assembled in parts, and sensitive data accumulates through repeated disclosures.

What does the CRA-Bench benchmark include?

CRA-Bench v0.1 consists of 1200 sessions of eight turns each across three threat families, with topic-matched harmless "twins." Version v0.2 adds variants paraphrased by a different model, and the extended set covers 2000 sessions and five threat families.

How does CRA-Net differ from a simple filter?

CRA-Net DA is a compact trainable model that analyzes the entire session trajectory rather than a single message. It is trained with family-adversarial objectives so that it does not confuse conversation length and topic breadth with a real threat.