Data Rewriting Beats DP-SGD on the Privacy-Quality Trade-Off

Teams that train models on medical, financial, and other sensitive text usually pick from two bad options: fine-tune directly and risk leakage, or turn on differential privacy and accept a loss in quality. A new paper from Hugging Face Daily Papers proposes a third path and tests it in a controlled experiment. The authors compared data-level and training-level interventions on synthetic medical records. Synthetic data generation, in which a model rewrites the records and replaces unique personal details, gave the best balance of capability and privacy.

How leakage was measured: "canaries" in medical records

To make the comparison fair, the researchers built a set of synthetic medical records and planted "canaries" in it. These are unique fragments, known in advance, that should not appear in normal data. If the trained model reproduces them, or shows abnormally high confidence in them, it has memorized training examples. This approach gives an empirical estimate of privacy: not a theoretical guarantee, but an observed risk. Model quality is measured separately, using capability metrics.

The comparison covered three families of approaches:

• direct fine-tuning on the original records;
• DP-SGD, that is, training with differentially private stochastic gradient descent;
• training on synthetic data derived from the original records, first through naive paraphrasing, then through privacy-aware rewriting.

Where the usual choice breaks down

Direct fine-tuning, as expected, led on capability and, just as expected, produced the most leakage. A model that sees the data as is uses it best and also memorizes it best.

DP-SGD goes the other way. Leakage drops noticeably, but the price is high: capabilities fall substantially. For practical tasks this is an awkward trade, especially when the model has to handle medical terminology and fine detail. That is why DP-SGD often stays an academic benchmark and reaches products less often.

Synthetic data lands in the middle, and on fairly favorable terms. The key factor is how the task is phrased for the rewriter model. If you simply ask it to paraphrase a record, some unique details survive the paraphrase and can leak later. If you add an instruction that accounts for uniqueness and replaces precisely the details that identify a specific case, protection improves. The authors show that private rewriting improves protection over naive paraphrasing across different rewriter models. The effect is not tied to any one model.

A stronger rewriter means lower risk, but the relationship isn't linear

Another observation concerns the model that does the rewriting. The more capable it is, the lower the leakage risk, on average, for a model trained on its synthetic output. That makes sense: a strong model better understands which details are unique and replaces them more carefully, while preserving the useful structure of the text.

But the relationship is not monotonic. Held-out fit does not rise strictly with the rewriter's strength. And additional inference-time compute yields an unstable gain. You can't simply drop a powerful reasoning model into the pipeline and expect privacy and utility to improve together. The configuration will have to be tested on your data.

What teams with sensitive data should do

The practical takeaways for anyone preparing corporate or medical datasets:

• Direct fine-tuning on raw records is justified only where leakage is not critical or the data is already anonymized.
• If you need a formal guarantee, DP-SGD remains an option, but you will have to budget for the loss in quality up front.
• For most applied cases, it makes sense to start with a "rewrite, then train" pipeline with an explicit instruction to replace unique details rather than just paraphrase.
• Choose the rewriter model based on the results of a canary audit, not on size or rankings. Check both leakage and quality on a held-out set.

The study itself is limited to synthetic medical records, and transferring the results to real clinical text will require separate validation. But the authors also offer a reproducible evaluation framework against which future models and methods can be compared. For a field where privacy is more often discussed in terms of declarations, this is a useful tool: it lets researchers measure leakage the way quality has long been measured.