How Personal Data Anonymization Affects LLM Agents: Hivetrace Dataclean Experiment

Personal data cleanup before sending to an LLM agent usually looks like an obvious security step, but the cost of this solution is not always clear. In a Habr AI article, the authors tested how much the work of a banking agent changes if instead of real full names and other identifiers, it receives masks or aliases.

Where the LLM-agent conflict arises

LLM agents increasingly work with user scenarios where personal data is indispensable: banking requests, support, insurance, documents, transaction history. At this point, any team faces a choice between two risks. First — pass sensitive data to the model as is and face privacy, compliance, and internal security questions. Second — clean the input but lose part of the context on which the agent bases its logic, entity relationship discovery, and action accuracy.

"How much does an agent degrade if instead of 'Ivanov

Ivan' it sees 'PERSON_1' or 'XXXXXXXX'?"

In practice, this is not an academic debate but an engineering task. If the system stops understanding that the same client appears in multiple places in a request, it affects not just text quality but business logic: case routing, credential verification, status interpretation, step sequence validation. Therefore, the question is not a general thesis "anonymization is useful" or "anonymization gets in the way," but a measurable answer for a specific class of agents and a specific cleaning method.

How the hypothesis was tested

To test the hypothesis, the authors deployed a minimalist banking agent and integrated Hivetrace Dataclean with it. They then sent the agent 102 synthetic requests in three input data variants: unmodified, masked, and with aliases. This design is useful because it removes noise from real user histories and provides a controlled comparison. In each case, the agent solves the same task but sees different forms of identifiers, names, and other personal attributes.

The evaluation method is particularly important. The authors used DeepEval in LLM-as-a-judge mode — that is, they compared not subjective impressions from a pair of responses but tried to formalize quality through a single validation loop. For quick applied research, this is a reasonable approach: it doesn't make conclusions "absolute truth," but allows you to see where degradation is immediately noticeable and where the agent retains usefulness even after sensitive fields are cleaned.

Three data modes

The essence of the experiment is not just to check whether the agent "works or doesn't work" after anonymization, but to compare different levels of context loss. This is especially important for LLM systems that rely not only on the meaning of words but on stable relationships between entities within a single dialog or document. Full masking and pseudonymization look similar to a model only at first glance: in a real task, these are different signals.

• Clean data — baseline scenario with no distortions.
• Mask like XXXXXXXX — maximum privacy, minimum semantics.
• Aliases like PERSON_1 — hide identity but preserve relationships.
• 102 synthetic requests per mode — opportunity for honest comparison.

From such a test, teams usually get not a universal ban or permission but a map of tradeoffs. If the agent only needs the user's general intent, hard masking may prove acceptable. If logic depends on the same object repeating across multiple fragments, aliases often look like a more practical option. This is precisely why such checks are especially useful before deploying agents in banking, fintech, medicine, and any process where errors on personal data quickly become expensive.

What this means

The main takeaway of the article is that the anonymization question for LLM agents cannot be solved at the intuition level. Before production, you need to separately measure how a specific cleaning method affects a specific scenario: query understanding, preservation of relationships between entities, and overall response quality. For teams building AI agents in sensitive domains, this is no longer an option but a fundamental part of the architecture and test loop.