FinMMEval 2026 Task 2: бенчмарк финансового QA на 5 языках, топ-4 систем в пределах 1%

FinMMEval2026 Task2 — a competition on short-form financial question-answering over multilingual data — has wrapped up: in July 2026 the organizers published a final leaderboard of 12 systems, where the top four are separated by less than one percentage point on the ROUGE-1 F1 metric across a set of 256 questions.

What the benchmark tests

FinMMEval2026 Task2 evaluates how accurately AI systems answer financial questions by drawing on documents in five languages at once. Each test item pairs an English-language question with financial reports and news in English, Chinese, Japanese, Spanish, and Greek, and the system must return a single concise answer in JSONL format.

• The final set is 256 questions, split evenly between the easy and expert tiers
• Each tier has 4 question templates, expanded across 32 groups of corporate reports
• Source data languages: English, Chinese, Japanese, Spanish, Greek
• Ranking metric: macro-averaged ROUGE-1 F1 per item
• The final leaderboard includes 12 systems

How the systems were scored

The organizers withheld the reference answers during the submission period and ranked participants by macro-averaged item-level ROUGE-1 F1 against their own reference answers. The spread at the top turned out to be minimal: the top-4 systems are separated by less than one percentage point on ROUGE-1 F1 — meaning the leaders are effectively neck and neck, and the choice of architecture at this margin comes down to fractions of a percent.

"The strongest systems are tightly clustered: the top four are separated by less than one percentage point on ROUGE-1 F1," the

FinMMEval2026 organizers' report, published on arXiv, states.

What participants used

Participating teams built their solutions around handling multilingual sources rather than a single model. According to the system papers published alongside the results, participants documented retrieval-augmented generation (RAG), cross-lingual evidence processing, structured prompting, answer compression, and validation strategies. It was precisely the set of 32 groups of corporate reports and four question templates per tier that forced systems to pull a fact from a document in Chinese or Greek and pack the answer into a short, precise formulation in English.

Why this matters for financial AI

FinMMEval2026 Task2 shows where the boundary of language models in finance lies today: not in text generation, but in accurately extracting a fact from reporting in an unfamiliar language. The tight leaderboard — top 4 within 1% — is a signal that multilingual financial QA is ceasing to be a task for a single strong model and is turning into pipeline engineering: retrieval, cross-lingual matching, compression. For banks and funds working with reporting in multiple languages, this is the closest current benchmark for quality.

What it means

The benchmark captures the maturity of multilingual financial question-answering: the leaders are already nearly indistinguishable by the metric, and further progress will be measured in fractions of a percent and the quality of source handling, rather than leaps by a single model.

Frequently asked questions

What is FinMMEval2026 Task2?

It is a competitive benchmark for short-form financial question-answering: a system receives a question in English along with a selection of financial reports and news in five languages, and returns a single condensed answer. The final set contains 256 questions.

What languages is the data in?

The questions are formulated in English, while the accompanying financial documents and news are in English, Chinese, Japanese, Spanish, and Greek. The system's task is to extract a fact from multilingual sources and give a precise answer.

How were the systems evaluated and ranked?

The reference answers were withheld during the submission period, and the final leaderboard of 12 systems was built on macro-averaged item-level ROUGE-1 F1. The top 4 participants are separated by less than one percentage point on this metric.