EinsiaAI Launches Quant Arena: Half of 9 LLMs Lose Money on China's A-Share Market

EinsiaAI has published Quant Arena, a benchmark that tests whether large models can work as quantitative researchers on China's stock market. The test covers 9 models from 6 vendors. Each model wrote a strategy, ran a backtest, refined its code and finally submitted a single finished version. That version was then run against pre-sealed market data. Only the GPT-series models and Opus 5.5 turned a profit; the rest lost money. The best result belongs to GPT-6 Astra: about 10% annualized.

An exam where you can't fit the answers

Most coding and agent benchmarks measure whether a program runs and whether tests pass. That is not enough for quantitative research. Code can work flawlessly and still describe a strategy that looks great on historical data and falls apart on new data. The classic trap here is overfitting to the backtest: the longer a model tunes parameters on a known period, the better the return curve looks and the less it is worth.

Quant Arena sidesteps this trap through the design of the test itself. The model sees the history and can change its logic through as many iterations as it likes, but the final version is evaluated on data the model has never seen. This measures not the ability to write a strategy, but the ability not to fool yourself.

Test conditions, based on the known information:

• 9 models from 6 vendors on a single platform;
• market: Chinese A-shares;
• cycle: write a strategy, backtest, fix the code, submit the final version;
• final evaluation: pre-sealed price data;
• profitable: the GPT series and Opus 5.5; the rest lose money;
• leader: GPT-6 Astra, about 10% annualized.

Why A-shares make a tough examiner

The market chosen for this test is a difficult one. A-share is known for a high share of retail investors, daily price-change limits and the T+1 rule: a security you buy cannot be sold the same day. A strategy carried over from the US market, recalled from training data, can easily break on this microstructure. Instead of reciting a textbook on momentum and mean reversion, the model has to account for exactly how this market trades, and do so in code.

That half the participants failed to make money looks natural. A language model is good at writing plausible code and a plausible rationale. The market, however, pays only for strategies with a statistical edge, and that edge does not come from fluent text. It is also telling that only two series came out ahead. The ability to honestly assess one's own backtest appears to depend on the specific model more than on its general coding level.

The scale of the success deserves a look too. About 10% annualized for the best model is the result of a modest fund, not a money-printing machine. For now, the benchmark shows the gap between models, not a readiness to hand them capital.

What to take away if you build financial agents

First: evaluate the agent on its final result on held-out data, not on the polish of a backtest it optimized itself. If you are building an internal quant assistant, keep the holdout set out of the model's reach and do not show it even indirectly.

Second: don't choose a model for financial tasks based on general coding rankings. Quant Arena offers a separate axis of comparison: how resistant a model is to self-deception in the "write, check, fix" loop.

Third: teams working with Chinese markets need checks for local specifics. Price limits and T+1 should be built into the simulator from the start, otherwise even a strong model will optimize a strategy for rules that do not exist on the real exchange.

And one last point. Benchmarks of this type go stale quickly once their data ends up in training sets. That is why the sealed holdout matters more here than the rankings themselves: it is what makes the comparison of models meaningful.