Who Wins the Instruction Conflict: Position, Chat Role, or Just a Label?

Who

Wins the Instruction Conflict: Position, Chat Role, or Just a Label?

When an AI agent works on a real task, it receives commands from several places at once: from the operator via the system prompt, from a repository's config file, directly from the user, from the output of a tool it called, and even from notes saved in memory from a previous session. These commands regularly contradict each other, and the agent can only act on one. A team of researchers from LNMIIT (India) and Virginia Tech set out to figure out what actually determines the winner in such a dispute — and found that the industry's standard "instruction hierarchy" (system outranks user, user outranks tool) works far less reliably than the simple order in which lines appear in the text.

Why Earlier Tests Were Misleading

The problem is that an instruction's source simultaneously changes three things at once: its position in the text (earlier or later), its role in the chat (system or user), and its label (how the text is signed). If a model ignores the system prompt, it's unclear why — did it arrive too early, too late, or was it simply signed differently? Most prior tests compared only one pair of instructions at a time and crammed them into a single message, which doesn't reflect how agents actually work, where conflicts arise across multiple messages and more than two sources.

To tease these three factors apart, the authors built 1,080 scenarios covering five task types — tool calls, file writes, step ordering, response-format selection, and task refusal — each with up to five mutually exclusive possible answers. Instructions were distributed across five "slots": the operator and the repo's config file (both living in the system message, but under different labels), the user, a tool's output, and a memory note. Seven separate experiments varied only how the text was delivered, leaving the instruction text itself untouched.

What the Tests Revealed

The team tested 14 open models — Qwen 3 (0.6 to 14 billion parameters), Qwen 3.5 (0.8 to 122 billion), and gpt-oss — each with reasoning mode on and off, for 26 model-mode combinations in total. They collected nearly 1.9 million responses, all scored by code against clear-cut criteria, with no AI judge involved.

Three main findings stood out:

• Position within a message decides almost everything. When two conflicting lines sit in the same message, the first one wins 72–95% of resolved responses. Once the lines are split across separate messages, that first-mover edge drops to 23–63%, and in 16 of the 26 model-mode combinations, the later message wins more often.
• Model families behave in opposite ways. In Qwen 3 and Qwen 3.5, a line from the operator sent first in the system message loses to later text from the user, a tool, or memory in nearly every case — winning only 6–41% of the time. Gpt-oss, by contrast, was trained to rank the system above the user and follows the system message 63–76% of the time, even when it arrives before everything else.
• For tool-calling and file tasks, the label matters more than the role. The exact same operator text wins less often when sent as a genuine system message than when the identical text is simply labeled as coming from the user — a 6–49% gap across all 26 combinations. The likely reason: in all three chat formats, the definitions of available tools are written directly into the system/developer message, so the operator's command gets buried among technical details, whereas a clearly labeled user message stands out on its own.

The team also checked whether the outcome of a three-to-five-way instruction conflict could be predicted purely from pairwise comparison results. The forecast improves noticeably if the model knows exactly where in the sequence the actual task sits, but even that holds up unevenly — for Qwen 3, this approach only helped in 3 of 10 combinations. And almost never — at most 2.92% of the time — did a model even mention noticing that the instructions conflicted at all.

What This Means for Agent Developers

If you're building an agent and counting on the system prompt to automatically outrank a later user message, that assumption may not hold for Qwen models — message order turns out to matter more than the declared rank. For tool-calling tasks, it's worth checking separately whether an important operator instruction gets "drowned out" among tool definitions in the system message — sometimes placing it as clearly labeled text is more reliable than trusting the chat role alone. And most importantly: instruction hierarchies need to be tested not just in pairs, but with three- and five-way conflicts among real sources — a model's behavior in a complex case can't be reliably inferred from simple pairwise tests.