FormulaSPIN: A Neural Network That Writes Spreadsheet Formulas From Descriptions and Beats Proprietary Models

In July 2026, researchers published FormulaSPIN on arXiv, a self-play training framework that generates spreadsheet formulas from natural-language descriptions. It reaches 74.9% exact match and 87.1% execution accuracy on the NL2FORMULA benchmark, ahead of both the classic supervised approach and top proprietary models.

How the model learns from itself

FormulaSPIN trains without a single new line of labeled data: the model plays against itself. According to the arXiv abstract, training is framed as a two-player game. The main player learns to prefer ground-truth formulas over those produced by its own previous version. Feedback from executing the formulas sorts the answers by "granularity" and builds an adaptive curriculum: the model first masters semantic correctness, then stylistic polish.

The key advantage of this task is binary executability: a formula either runs and returns a result, or it does not. That provides an implicit signal that separates semantic errors from formulas that are merely written differently but are still valid.

Why standard SPIN breaks here

Standard SPIN does not work for formula generation because it penalizes every answer that does not literally match the reference. According to the authors, the same formula, execution-equivalent, is punished as "wrong" in one example and serves as the reference in another. As a result, the training gradients contradict each other and the model fails to converge.

FormulaSPIN resolves this contradiction through execution checks. If two expressions that look different produce the same result, the system counts both as valid rather than treating one as an error. The authors also added ExecVote, a voting mechanism at the semantic level that handles the case where a single task has several correct formulations.

What the measurements showed

FormulaSPIN set a new best result on several benchmarks. On NL2FORMULA it reached 74.9% exact match and 87.1% execution accuracy, comparable to models trained on additional preference annotations and higher than both traditional SFT and frontier proprietary models.

• 74.9% exact match on the NL2FORMULA benchmark
• 87.1% execution accuracy
• Training without a single additional labeled example
• ExecVote — semantic-level voting for multiple valid formulas
• Outperforms both classic SFT and frontier proprietary models

"These results highlight the potential of self-play for data-scarce tasks and open the way to transferring it beyond executable domains," the

FormulaSPIN abstract on arXiv says.

What this means

Writing formulas remains a barrier for hundreds of millions of spreadsheet users, and labeled data for training such models is scarce. FormulaSPIN shows that a model can improve itself by using execution as a free source of verification, without expensive manual annotation. The approach applies to any task where the output can be executed and checked: code, SQL queries, calculations.

Frequently asked questions

What is FormulaSPIN?

FormulaSPIN is a self-play training framework for generating spreadsheet formulas from natural-language text. The model improves itself iteratively, without additional labeled data, using formula executability checks as its source of feedback.

How does FormulaSPIN differ from ordinary fine-tuning?

Classic supervised fine-tuning (SFT) quickly hits a ceiling set by limited annotations. FormulaSPIN trains as a two-player game and separates semantic errors from valid notation variants through execution, so it outperforms SFT and needs no new data.

What results did FormulaSPIN achieve?

On the NL2FORMULA benchmark, the framework reached 74.9% exact match and 87.1% execution accuracy. That is on par with models trained on additional preference annotations and higher than top proprietary models.