Axion 1.0-35B scores 52.2% on HLE as its creators release agent training code

If your agent needs to find information across multiple sources, compare it and calculate an answer, simply connecting a search tool is not enough. The model also needs to learn how to use what it finds. Axion 1.0 offers an open recipe for that training. For developers, it is an opportunity to examine the data preparation process and test the approach on their own tasks.

The project has a striking figure to its name: Axion 1.0-35B scored 52.2% on Humanity’s Last Exam with tools. But for a team building a product, another question is more useful: which parts of that result can be reproduced, and how much will they cost in its own use case?

What exactly was released

According to the project repository, the smaller model has 35 billion parameters, with 3 billion active during computation. The repository includes code for data generation and filtering, instructions, training and evaluation. However, the README directs users to a community reproduction of the weights and explicitly states that it is not an official Axion checkpoint. The promise of “download the authors’ model and reproduce the result” still requires separate verification.

Training consists of two stages using prepared examples, followed by a reinforcement learning stage. First, the model solves tasks using information provided in the problem statement, then learns to retrieve missing information with tools. In the final stage, it receives rewards for the outcome and verifiable intermediate progress. This sequence is described in the authors’ paper.

The practical appeal is the ability to examine the process piece by piece. A team may, for example, find the method for assembling verifiable tasks more useful than the model itself. This is particularly relevant if an agent already knows how to call tools but regularly selects the wrong data for a calculation.

What this looks like in a real task

Imagine a procurement assistant. It needs to choose the supplier offering the lowest total order cost while meeting a specified delivery deadline. It must find offers, check availability, convert prices to a common currency and add shipping. A well-written answer is worthless if the agent has compared different product configurations.

To train such an assistant, you could prepare a table of offers in advance and compute the correct choice programmatically. Then give the model only the question and access to approved sources. Save the successful solution together with its sequence of actions. This illustrates how the idea of verifiable tasks could be applied to a product; the authors do not report having a ready-made procurement agent.

Checks should cover more than the final total. Was the correct configuration selected? Do the delivery dates meet the requirements? Was shipping included? These intermediate checks help identify exactly where the chain breaks. If the correct answer was reached by chance, the example is better left out as a demonstration of reliable performance.

You can start without training your own model. Assemble a small set of typical requests, record the sources and correct answers, and run your current agent through them. This step alone will reveal what it lacks: search, calculation or an understanding of the requirements. It then becomes easier to decide whether a new dataset is needed.

Why 52.2% is not enough to choose a model

In the evaluation description, the authors report the best result from an individual run, rather than an average. Access to certain websites is restricted for the main HLE evaluation. Some competitors’ results come from external reports, so the comparison conditions are not fully aligned. Matching GPT-5.5 in the table cannot be taken to mean matching it across all real-world tasks.

Nor do 3 billion active parameters mean the model can be hosted with the same memory as a conventional model of that size. All the weights, the context and the chosen deployment method must be taken into account. The cost of a successful task also includes search, code execution and retries.

For a pilot, I would compare Axion with the current solution using identical tools and limits. I would measure the share of correct answers, execution time and cost per completed task. I would separately test cases involving an unavailable source or contradictory data. If the advantage holds up there, there would be grounds to discuss deployment. For now, the release’s main practical contribution is an agent training process that is available to study.