ID-TAP finds the right persona for an LLM agent in ten pairwise comparisons

Personalizing an LLM agent runs into a simple problem: users don't want to fill out questionnaires or explain exactly how they want to be answered. They are, however, willing to pick the better of two responses. The authors of a new paper built an entire framework, ID-TAP, on that signal. Within a fixed budget of ten pairwise-comparison rounds, it finds a suitable persona in a large discrete set and accounts for the fact that preferences depend on the query itself. On seven user-aware generation benchmarks, ID-TAP achieved the best aggregate result among the evaluated baseline methods.

The task: choosing a persona with almost no feedback

Here, a persona means the style and stance in which the agent answers: tone, level of detail, manner of presentation. There can be many candidates, and trying them all in a live conversation is unrealistic. Every round demands the person's attention, so the budget is tight: ten comparisons.

The authors frame this as a dueling bandits problem. The agent shows two responses generated under different personas, receives a binary choice, and updates its picture of the user's preferences. The difficulty is that a good persona for a question about a borscht recipe may be a bad one for a question about a tax deduction. A static "user profile" doesn't work here; the evaluation has to depend on the query.

How ID-TAP decides who to pit against the current leader

The approach rests on information-directed sampling (IDS): choose an action not by maximum expected reward, but by balancing expected benefit against the amount of information the action provides. In the dueling setting, this becomes choosing a "challenger" to face the current leader.

ID-TAP has two working components:

• a query-dependent neural utility model: it shares statistical strength across different queries while keeping separate estimates for each of them;
• a lightweight explorer built on the Bradley–Terry model: it updates quickly and scores hypothetical challengers, that is, it estimates what the next comparison would yield.

A third element handles knowledge transfer. Comparisons collected on some queries are passed to related queries through a warm start weighted by similarity. A new query doesn't start from zero if the system has already seen similar ones.

What the evaluation showed

The evaluation ran on seven generation benchmarks in which the response has to take the user into account. Preferences were simulated in two ways: from reference answers and with a judge model. This is a standard technique for online learning from comparisons, because running live people for every experiment is expensive. The downside is obvious too: a simulator is still a simulator, and real human behavior may differ.

Key parameters of the experiment:

• signal: a binary comparison of two responses;
• budget: ten comparison rounds;
• space: a large discrete set of personas;
• test suite: seven user-conditioned benchmarks;
• result: the best aggregate metric among the evaluated baseline methods.

The authors state their conclusion cautiously: choosing a challenger by informativeness works as a scheme for adapting a persona at inference time. The paper makes no sweeping claims of superiority over the entire market, and that is an honest framing.

Where this is useful in practice

Test-time personalization is appealing because it doesn't require fine-tuning the model. The base LLM stays frozen; the only thing that changes is which persona it adopts for a given query. For product teams, this is a cheap path: adaptation happens within the session, and user data doesn't have to be turned into a training set.

The most direct application is assistants and agents whose users are already accustomed to choosing between two response options. "This one is better" buttons have long been common in chat interfaces, and ID-TAP offers a way to use them not just for collecting labels but for immediate adjustment.

Three things are worth checking before deployment. First, how closely preference simulators match real people in your domain. Second, how the method behaves when a user contradicts themselves: pairwise choices are noisy. Third, what it costs to generate two responses in every round, since ten comparisons mean at least twenty generations per query. If those questions are settled, the scheme looks like a workable candidate for personalization without fine-tuning.