A test for AI analysts: the leader scored 68 out of 100, but only 52.5 on forecasting

Investor's Last Exam is a new benchmark, that is, a set of tasks for comparing models. It was built by researchers at the Antai College of Economics and Management at Shanghai Jiao Tong University, together with experienced investors and fund managers. The task is simple: give an AI agent the job of an analyst who studies a company and decides where to put money, then check whether it can do the work without looking into the future. The best system, GPT-6 Astra, scored 68.1 out of 100. In forecasting, however, it got just 52.5. In other words, models have become noticeably better at reading reports and writing up conclusions than at predicting numbers.

How to lock an agent in the past

The main problem with financial tests is leakage from the future. Data is usually restricted with a filter on a page's publication date. The authors show that this doesn't work. A page can keep its old date even though its content was updated later. In the Tesla task (number O74), the GPT-6 Astra agent sent a query dated before the end of the quarter and still received a source from July 2 with actual vehicle deliveries for the second quarter.

So the authors built a corpus that stores every version of a document and overwrites nothing. As of October 15, 2026, it holds 3 430 336 versions and 1.52 TB of material. The sources fall into nine categories:

• financial reports; • structured reporting tables; • quarterly investor calls; • investor materials; • issuer events; • analyst forecasts; • research reports; • institutional investor positions; • news.

There is also market data on prices, returns and volatility. The corpus covers the US, mainland China and Hong Kong markets.

Every version has a timestamp, a time zone and verification. If the date cannot be established reliably, the document does not make it into the tasks. For questionable records, modern models first estimated the dates and then rechecked them. The restriction applies to the whole chain: search, reading, downloading and cache all check availability in the same way. Tabular data is frozen after the query, and later edits do not change it. The agent runs its calculations in a sandbox with no network access.

What exactly is tested, and who does the grading

The tasks are built around real analyst deliverables: five categories, twenty subtypes, plus comprehensive reports and trading tasks that forecast price, returns or volatility. The test has 86 tasks in total. Each is scored on three dimensions: research quality, forecast accuracy and hypothetical investment outcome.

Every requirement in the grading criteria is tied to a source, a calculation or an assumption through a graph of financial dependencies. Experts write the question and the judgments, agents gather the evidence, several models check the drafts, and humans approve the final criteria. The reference answers and criteria are hidden from the agents. The paper uses GPT-6 Astra as the grader.

The examples show where models break. In the Microsoft task (O58), cloud revenue cannot be added as a fourth segment, because it is already included in the other three. In the AEP task (N27), growth in operating cash flow does not mean the company needs less outside funding once investments and adjustments are taken into account. Refinancing has a similar trap: the sum of cash, cash flow and an undrawn credit line looks sufficient until you read the loan covenants.

Where the models go wrong

The researchers tested 11 systems. Claude Opus 5.5, with 66.9, follows GPT-6 Astra, with 68.1. Astra's breakdown by skill:

• search and work with evidence — 78.9; • valuation and investment decision — 77.6; • forecasts and model building — 52.5.

The ranking changes when you look at outcomes rather than analysis. DeepSeek V4.1 Flash came closest to the actual forecast results. Kimi K3, despite a lower research score, delivered more hypothetical returns than Opus. The overall score does not tell you which model will better guess the market.

On errors, the authors write that models can balance cash flows but are poor at separating real operating improvement from temporary cash movements. They produce coherent explanations and recommendations, but there is no guarantee that the assumptions, accounting states and final figures add up to a verifiable model. Models also have trouble submitting answers and recovering from errors. Some of the discrepancies come from vague wording in the criteria rather than from model weakness.

The main conclusion: the weak point has shifted from information access and arithmetic to calibrating assumptions and keeping the model consistent. For teams building financial agents, this means evaluating not only the final report but also the chain "source — assumption — calculation — decision." And cutting off the future will have to happen at the system level, not with a date filter in the query.