Distilling ECG into PPG Memorized Identity: Fixed-Effects More Than Doubled the State Signal

People buy wearables to see how a specific person's cardiovascular state changes: heart rate is higher today, recovery was better yesterday. But preprint arXiv:2610.10662 shows that standard ECG-to-PPG distillation teaches the model something else. The per-recording mean feature vector, which the authors call the trait, makes up 40–59% of the frozen ECG teacher's target signal. In other words, almost half of what the student tries to reproduce describes who the person is, not what is happening to them. The authors' fixed-effects distillation removes that component, and state agreement more than doubles.

Where identity hides

ECG has long served as the "gold standard" for training models that work from photoplethysmography (PPG) alone, meaning the optical sensor in a watch or ring. The teacher looks at the electrocardiogram, the student looks at PPG, and the student fits its embeddings to the teacher's. Quality is usually measured with cosine similarity, and that is where the trap lies. A constant predictor that outputs the same vector regardless of input scores 0.793 on this metric. A high cosine proves little: much of the similarity comes from the model simply landing on the "midpoint" of the target distribution.

Worse, models trained on a shared pool of recordings memorize this trait but do not transfer it to new recordings. That is the worst of both worlds: the student spends capacity on identity, and that gives it no useful information about unfamiliar recordings. The authors tested the relationship across 34 runs. The pattern is direct: the more identity the student memorizes, the less state it learns.

How fixed-effects distillation works

The idea comes from econometrics and panel data: if you care about variation within a unit, you have to factor out the unit's average level. Here that works as follows: the per-recording mean is subtracted from both the student's prediction and the teacher's target, and then an ordinary distillation loss is computed. The trait cancels out exactly, because by definition it is constant within a recording. The alternative, a pooled anchor, keeps this component and serves as the control.

According to the authors:

• The trait makes up 40–59% of the frozen ECG teacher's target signal.
• A constant predictor scores 0.793 on raw cosine alignment.
• The experiment covers 34 runs, in which identity memorization is negatively associated with learned state.
• State agreement more than doubles.
• Within-person labels improve, while age and sex stay at the same level.
• The effect reproduces on two backbones and two additional databases.

The age and sex result matters separately. Both are static characteristics of a person, and if the method simply improved every metric across the board, this one would have gone up too. It did not, which is consistent with the claim that identity has been removed from the representation while the dynamics remain. In the authors' words, conditioning on the recording turns distillation toward the within-person changes that wearables are meant to track.

What to check in your own pipeline

For teams building PPG models for watches, rings, and medical patches, the takeaway is practical. First, do not rely on raw cosine or a similar alignment metric without a baseline: compute the constant predictor's score and subtract it. Second, evaluate the model on within-person tasks, that is, on changes relative to a person's own recording, and not only on cross-subject labels such as age and sex. Third, if you distill from ECG or another "expensive" signal, try per-recording centering as a cheap loss modification: it does not require a new teacher.

The limitations are also visible. This is a preprint, and the claimed "more than double" applies to the state-agreement metric the authors chose. Centering requires the training batch to contain whole recordings or sufficiently large segments of them; otherwise the mean will be noisy. For tasks that need absolute identification (biometrics, age estimation), the method, by the authors' results, will not help, and is not meant to. But for monitoring dynamics, it looks like a neat, inexpensive correction to what is actually being optimized.