Epoch AI: Recent Models Struggled to Match a Human-Invented Algorithmic Idea

Epoch AI has published InnovationEval, and its headline speaks for itself: recent AI models struggled to match an algorithmic innovation devised by a human. The post went up on 9 October 2026. On Hacker News it collected 86 points and 71 comments, so the community latched onto the topic right away. A caveat up front: the material that reached our editorial team contains only a headline, a link and discussion counters. It has no tables of results for specific models, and we are not guessing at them.

What we know about the publication

Epoch AI has been running independent evaluations of model capabilities for years, so its findings tend to get read more closely than lab press releases. The name InnovationEval signals that the test targets the ability to invent. The models are not solving a memorized task. They have to arrive at an idea that a human researcher proposed earlier. The wording of the headline ("struggled to match") suggests the result came in weaker than expected.

Here is what can be stated based on the available data:

• Source: Epoch AI, page publications/innovationeval.
• Publication date on the aggregator: 9 October 2026.
• Main conclusion: recent models could not reliably reproduce a human algorithmic innovation.
• Hacker News reaction: 86 points, 71 comments.
• Not disclosed in our source: the list of models, the number of attempts, the scoring criteria and specific percentages.

Why this kind of test matters

Most popular benchmarks measure how well a model solves problems with a known answer: olympiad math, bug fixes in repositories, competitive programming. Progress there has been fast in recent years, and the charts often hit a ceiling. A test for invention works differently. It asks whether a model can propose a new technique, rather than assemble a good solution from what is already in its training data.

This ties directly to the industry's central promise: AI agents that accelerate AI research itself. If a model cannot reproduce even an idea a human has already found, talk of automating scientific work should be kept within narrower bounds. The reverse also holds: a single test does not settle the question. The result depends on how the task is set up, on hints, on the reasoning budget and on which innovation was chosen as the reference.

How to read the result

There are three things to look for in the primary source. First, how the authors separate invention from recall: if the idea could have ended up in the training set, the test measures memory, not novelty. Second, how many attempts each model got and how the best answer was selected. Third, what counted as success: an exact match with the human solution or a functionally equivalent approach. These details determine how loud the conclusion can be.

The level of discussion on Hacker News (71 comments for 86 points) suggests that readers have questions specifically about the methodology. That is normal for work of this kind: measuring "inventiveness" is far harder than measuring accuracy on a test with an answer key.

What to do in practice

Teams building products on coding and research agents can usefully split tasks into two classes. Routine engineering, refactoring, porting known algorithms and parameter tuning fall within what models do reliably. Setting out new algorithmic approaches remains an area where a human still sets the direction, while the model acts as a fast executor and a generator of variants to test.

Before revising any plans, open the original at epoch.ai and check three parameters: the list of models tested, the number of runs and the scoring criterion. If you are buying AI tools for R&D, ask the vendor for results on tasks that require novelty, not only on standard benchmarks. And keep in mind that the next release may shift the picture: evaluations like this go out of date quickly.