FormulaSPIN: нейросеть пишет формулы для таблиц по описанию и обгоняет проприетарные модели

In July 2026, researchers published the FormulaSPIN framework on arXiv — a self-play training system that generates spreadsheet formulas from natural-language descriptions and achieves 74.9% exact matches and 87.1% execution accuracy on the NL2FORMULA benchmark, outperforming both the classic supervised approach and top proprietary models.

How the model learns from itself

FormulaSPIN trains without a single new line of labeled data — the model plays against itself. According to the arXiv abstract, training is framed as a two-player game: the main player learns to prefer ground-truth formulas over those produced by its previous version. Feedback from formula execution sorts responses by "granularity," building an adaptive curriculum: the model first masters semantic correctness, then stylistic polish of the notation.

The key advantage of the task is binary executability: a formula either runs and produces a result, or it doesn't. This gives an implicit control signal that separates meaning-level errors from variants that are simply written differently but still valid.

Why standard SPIN breaks down here

Standard SPIN doesn't work for formula generation because it penalizes any answer that doesn't match the reference literally, in the same way every time. According to the authors, the same formula, equivalent in execution, is punished as "incorrect" in one example and serves as the reference in another — this causes the training gradients to contradict each other, and the model fails to converge.

FormulaSPIN removes this contradiction through execution-based verification. If two differently worded expressions produce the same result, the system counts them as valid rather than as an error. The authors additionally introduced ExecVote — a semantic-level voting mechanism that natively handles cases where a single task has multiple correct formulations.

What the measurements showed

FormulaSPIN set a new best result on several benchmarks. On NL2FORMULA — 74.9% exact matches and 87.1% correct by execution, comparable to models trained on additional preference annotations, and higher than both traditional SFT and leading proprietary models.

• 74.9% exact match on the NL2FORMULA benchmark
• 87.1% execution accuracy
• Training without a single additional labeled example
• ExecVote — semantic-level voting for multiple valid formulas
• Outperforms both classic SFT and frontier proprietary models

"These results highlight the potential of self-play for data-scarce tasks and open a path toward extending it beyond executable domains," the

FormulaSPIN paper's abstract on arXiv states.

What it means

Writing formulas remains a barrier for hundreds of millions of spreadsheet users, and there is little labeled data to train such models. FormulaSPIN shows that a model can improve itself by using execution as a free source of verification — without expensive manual labeling. The approach applies to any task where the result can be executed and checked: code, SQL queries, calculations.

Frequently Asked Questions

What is FormulaSPIN?

FormulaSPIN is a self-play training framework for generating spreadsheet formulas from natural-language text. The model improves itself iteratively, without additional labeled data, using formula executability checks as a source of feedback.

How does FormulaSPIN differ from regular fine-tuning?

Classic supervised fine-tuning (SFT) quickly hits a ceiling set by limited labeling. FormulaSPIN trains as a two-player game and separates meaning-level errors from valid notation variants through execution, so it outperforms SFT and requires no new data.

What results did FormulaSPIN achieve?

On the NL2FORMULA benchmark, the framework reached 74.9% exact matches and 87.1% execution accuracy — a level matching models trained on additional preference annotations, and higher than top proprietary models.