FinMMEval 2026 Task 2: A Five-Language Financial QA Benchmark, With the Top 4 Systems Within 1%

FinMMEval 2026 Task 2, a competition on short-form financial question answering over multilingual data, has announced its results. In July 2026, the organizers published a final leaderboard of 12 systems, in which the top four are separated by less than one percentage point on ROUGE-1 F1 across a set of 256 questions.

What the benchmark tests

FinMMEval 2026 Task 2 measures how accurately AI systems answer financial questions using documents in five languages at once. Each test item pairs an English-language question with financial reports and news in English, Chinese, Japanese, Spanish, and Greek. The system must return a single short answer in JSONL format.

• The final set contains 256 questions, split evenly between the easy and expert levels
• Each level has 4 question templates, instantiated across 32 groups of corporate reports
• Source-data languages: English, Chinese, Japanese, Spanish, Greek
• The ranking metric is macro-averaged ROUGE-1 F1, computed per item
• The final leaderboard includes 12 systems

How the systems were evaluated

The organizers kept the reference answers hidden while submissions were open. They ranked participants by macro-averaged item-level ROUGE-1 F1 against their own reference answers. The spread at the top turned out to be minimal: the top 4 systems are separated by less than one percentage point on ROUGE-1 F1. In practice, the leaders are running neck and neck, and at this margin the choice of architecture comes down to fractions of a percent.

"The strongest systems are tightly clustered: the top 4 are separated by less than one percentage point in ROUGE-1 F1," the organizers of

FinMMEval 2026 say in their report, published on arXiv.

What participants used

The participating teams built their solutions around handling sources in multiple languages, rather than around a single model. According to the system papers published alongside the results, participants documented retrieval-augmented generation (RAG), processing of cross-lingual evidence, structured prompting, answer compression, and validation strategies. The combination of 32 groups of corporate reports and four question templates per level forced systems to pull a fact out of a Chinese or Greek document and fit it into a short, precise English answer.

Why this matters for financial AI

FinMMEval 2026 Task 2 shows where language models currently hit their limit in finance: not in generating text, but in precisely extracting a fact from financial reporting in an unfamiliar language. The tight leaderboard, with the top 4 within 1%, signals that multilingual financial QA is no longer a problem for one strong model and is turning into pipeline engineering: retrieval, cross-lingual matching, and compression. For banks and funds that work with reports in several languages, this is the nearest quality benchmark available.

What this means

The benchmark marks a maturing field in multilingual financial question answering: the leaders are already nearly indistinguishable on the metric, and further progress will be measured in fractions of a percent and in the quality of source handling, not in leaps by a single model.

Frequently asked questions

What is FinMMEval 2026 Task 2?

It is a competitive benchmark for short-form financial question answering: the system receives an English question and a set of financial reports and news in five languages, and returns one compact answer. The final set contains 256 questions.

Which languages does the data cover?

The questions are written in English, while the accompanying financial documents and news are in English, Chinese, Japanese, Spanish, and Greek. The system's task is to extract a fact from the multilingual sources and give a precise answer.

How were the systems evaluated and ranked?

Reference answers were hidden during the submission period, and the final ranking of 12 systems was built on macro-averaged item-level ROUGE-1 F1. The top 4 participants are separated by less than one percentage point on this metric.