AI and Disinformation: How LLM Agents Erode Trust

Researchers have published an analysis on arXiv of how AI agents and humans jointly undermine trust in information chains by exploiting trust mechanisms in public communication.

What is adversarial epistemology

In modern information spaces, most claims are never checked directly. Instead, we rely on trust "scaffolds": chains of evidence, institutional certifications, references to experts and inferences drawn from earlier inferences. We trust an article because a peer-reviewed journal published it; we trust the journal because colleagues cite it; we trust those colleagues because they are affiliated with a respected university.

The authors of the ASE framework argue that dishonest agents deliberately exploit exactly this trust infrastructure. In their view, the notions of "filter bubbles" and "echo chambers" do not describe the main threat, since they concern unintended consequences of the information environment. ASE studies intentional action: the strategic exploitation of what normally makes claims trustworthy.

How exactly agents distort information

The researchers catalog the key manipulation tools available to agents in public communication:

• Distortion: deliberately altering facts as they pass along the chain
• Omission: intentionally leaving out critical context or alternative accounts
• Fabrication: creating false references, nonexistent evidence and forged attributions
• Strategic underspecification: wording that deliberately allows several interpretations, all of which benefit the agent
• Exploitation of authority: applying pressure through a source's apparent authority without real grounds

The cumulative effect of long chains is especially dangerous: each link may introduce a small bias, which by the end becomes a systemic lie, even though every individual step looks legitimate.

Why do language models change the rules of the game?

LLMs are being built into the same trust chains as humans: they generate academic texts, news digests, legal documents and institutional reports. A language model's output is outwardly indistinguishable from a verified human statement. The authors note that language systems have specific incentives to distort: pressure from reinforcement learning training (RLHF), operators' goals and the need to appear authoritative.

"What needs explaining is not how information becomes distorted by

itself, but how communicative agents exploit the obligations and entitlements that normally make claims reliable," the authors write.

The result is "mixed" chains: some claims in them are verified by humans, others are generated by models, and an auditor will find it extremely hard to draw the line without specialized tooling.

What the researchers propose

For their analysis, the authors use inferential semantics, an approach in which a claim's meaning is determined by its role in chains of inference rather than by its literal content alone. On this basis they build the concept of epistemic networks: a graph of claims, sources and the logical transitions between them. The researchers stress that ASE's aim is not to explain why information goes wrong, but to formalize strategies of deliberate deception through seemingly legitimate channels.

Such a graph makes it possible to identify "trust breach points", the specific places in a chain where a distortion was introduced, and to analyze the violation formally. The authors also propose audit mechanisms: methods for restoring the auditability of inferential chains and detecting systemic manipulation.

What this means

As LLM agents become full participants in public discourse, generating news, scientific reviews and legal documents, the need for formal methods of verifying trust will grow. For AI product developers, this is a concrete task: build verification into pipelines not only for facts but for the chains of logical transitions themselves. The work offers a theoretical foundation for such tools and leaves open a practical question: how to build information systems that withstand strategic abuse.

How do AI and humans create disinformation together?

According to the researchers, AI agents working alongside humans exploit trust mechanisms in public communication. This is the first systematic description of the phenomenon in the context of systems where large language models and humans act together.