MULTIFINGROUND: LLMs Recognize Accounting Concepts With 99% Accuracy but Miss Facts

Language models almost always understand which line item of a financial statement is being discussed, but they stumble when they have to collect all the numbers tied to it. The authors of the MULTIFINGROUND benchmark ran 15 models through FY2025 annual disclosures from 90 issuers in English, French and Greek. Eight reference models, both API and open-weight, identify the accounting concept with 97.3–99.4% accuracy. Yet in 26.6–41.9% of correctly recognized groups, they miss at least one target fact.

How the benchmark works

Comparing financial statements across countries relies on two skills. First, a model has to understand that different wordings in different languages describe the same accounting concept. Second, it has to find all the numbers that express that concept: in the primary statements, the notes and the explanatory text. Existing tests assess these skills separately, so it is hard to tell whether models manage both on the same material.

MULTIFINGROUND closes this gap. The reports were prepared independently and are not translations of one another, so a model cannot peek at a parallel text. The authors aligned the documents through a shared accounting taxonomy and built fact-level annotations that trace back to the source. The benchmark tests four tasks:

• identifying a financial fact;
• recognizing a concept;
• exhaustively selecting all facts that belong to a concept;
• distinguishing concepts without predefined candidates.

What the numbers show

Key parameters and results of the study:

• 90 issuers, FY2025 reports;
• 3 languages: English, French, Greek;
• 15 models tested;
• 97.3–99.4% concept-recognition accuracy for the eight reference models;
• 26.6–41.9% of correctly recognized groups with at least one missed fact.

The paper's title, Recognition Is Not Grounding, describes the result precisely. Concept recognition is essentially classification: the model sees the context and picks a label. Exhaustive extraction works differently. The model has to go through the entire document, make a decision on every number and lose nothing. A model can correctly understand that the topic is, say, revenue, and still fail to collect every instance of it in the document. The authors do not explain the causes in the abstract. But the gap is visible only because both skills were measured on the same data.

Two other findings matter just as much. How well models distinguish concepts without candidate hints varies widely between models, which means the open-ended setup filters out more models than choosing from a ready-made list. And the finance-specialized models in the sample delivered only a limited gain. Fine-tuning on industry texts does not close the completeness problem.

What to check in your pipeline

If you are building data extraction from financial statements, for example to screen issuers or compare companies from different jurisdictions, concept-classification accuracy is misleading. A 99% figure looks convincing, but it does not tell you how many facts from a group made it into the final table. The metric should be fact-level recall, or better still, the share of groups where all target numbers were found.

It is also worth testing without a candidate list: in real tasks, the taxonomy is rarely handed to the model ready-made. It helps to compute results separately by language, especially if some are lower-resource, like Greek. Buying a specialized finance model in hopes of better completeness is not advisable for now: on this data, the gain is small. A more reliable approach is to add a second pass, a programmatic reconciliation of extracted numbers against the taxonomy where one exists, and manual review for critical line items. A model that names the concept correctly does not guarantee that you got all the numbers behind it.