Video Generation With an Expression Gallery Beats Baseline Models on Identity Similarity

Identity-preserving video generation models usually take a single portrait as input and must keep the face recognizable throughout the clip. The authors of preprint arXiv:2610.11023 argue that this setup has a structural flaw. One photo captures the face in a single configuration, and the reference does not determine how that face will look when laughing, shouting or surprised. The researchers propose replacing the single portrait with a compact gallery of different expressions. They extended the Stand-In model to handle such sets and tested the result in three modes: weak, strong and extreme expressions. The approach performed best on extreme expressions.

The problem: every face changes in its own way

When an expression changes, similarity to the neutral photo does not change in a universal way. For one person, a smile barely affects their features; for another, it changes them noticeably. A model that has seen only a neutral portrait must therefore guess how that face will look in motion. It guesses from average statistical patterns in the training data, and the identity gets blurred.

The same quirk distorts quality evaluation. The authors measured how similarity in face recognition systems depends on expression intensity, using controlled photos and videos from the MEAD dataset. They found that even for real images of the same person, similarity to the neutral reference drops under strong expressions. A metric that compares a generated frame with a single portrait penalizes the model for the natural variability of the face, not for losing identity.

What the authors propose

The solution works on two levels: evaluation and generation.

For evaluation, the authors assembled a compact but expressive reference gallery that covers different facial configurations. Similarity is computed as a match against the gallery rather than against a single photo, which makes it more robust to expression. Results are reported separately for weak, strong and extreme expressions, so the degradation as intensity grows is visible rather than lost in an average.

For generation, the authors extended Stand-In so that the model accepts sets of expressions. They also built a data preparation pipeline. It extracts face frames from training videos that are both consistent in identity and diverse in expression. If the user has only one portrait, a pretrained face reenactment model (that is, an expression-transfer model) synthesizes the missing expressions.

Key elements of the work:

• basis for studying how similarity depends on expression: controlled MEAD photos and videos;
• three evaluation modes: weak, strong and extreme expressions;
• two types of reference sets: real ones and ones synthesized from a single portrait;
• base model: Stand-In conditioned on a set of references;
• result: both variants outperform the tested baseline models on identity similarity in all three modes, with the largest gain on extreme expressions.

The abstract gives no numerical values for the gain, and the comparison was run on the authors' own controlled benchmark. There is no independent confirmation on external datasets yet, so the size of the effect cannot be judged from the abstract alone.

Practical takeaways

The variant with synthesized references matters in its own right. It means the improvement does not require users to upload a dozen photos. One portrait is enough, and the reenactment model fills in the rest. For products built around avatars, virtual presenters and personalized advertising, this is a cheap way to improve recognizability in emotional scenes, where current generators break most often.

Teams evaluating their own models can adopt the idea of breaking results down by expression intensity. An average similarity metric hides failures on extreme expressions, and those are exactly the ones viewers notice. Comparing against a gallery instead of a single frame also reduces the risk of false positives.

There is an obvious risk as well. The better a model holds a face across any expression, the easier it becomes to create convincing deepfakes of specific people from a single photo. For platforms, this is a reason to plan consent checks and labeling of synthetic video in advance, before such methods move from preprints into products.