KernelEvo: A Russian Framework That Uses AI to Automate GPU Kernel Creation

Writing fast GPU kernels has always been the preserve of a chosen few. A narrow circle of engineers who can juggle memory models, access patterns and the constraints of specific hardware backends has set the pace of high-performance computing. Russia's AIRI Institute wants to break that vicious circle with KernelEvo, a framework that turns the painful process of hand-optimizing GPU kernels into an automated search.

The problem KernelEvo addresses is familiar to anyone who has tried to squeeze the maximum out of a graphics accelerator. The classic kernel development cycle goes roughly like this: an engineer writes code, runs it, hits a compilation error or unexpected runtime behavior, goes back to the code, rewrites it and checks again. This iterative process can stretch over days or weeks, and the result depends directly on the developer's skill. Meanwhile, the gain from a well-optimized custom kernel over a generic implementation can be enormous, sometimes a multiple-fold speedup.

The "Computational Intelligence" team at AIRI proposed a fundamentally different approach. Instead of relying on human intuition and expertise, KernelEvo builds an automated search cycle. The framework takes source code as input and independently looks for efficient implementations in CUDA and Triton, the two main platforms for GPU programming. The key word here is "looks": the system does not simply generate a single code variant but methodically explores the space of possible solutions, checking each one for correctness and performance.

Technically, the approach relies on large language models inside the optimization loop. The model generates kernel variants, the system compiles and tests them, and feedback from the results goes back to the model for the next iteration. In essence, this is the same loop a human engineer goes through, but run automatically and with a much higher search speed. The developers say a single optimization task takes on the order of a million tokens. Converted into the cost of API calls to modern language models, that comes to quite acceptable sums, especially compared with paying for the working time of a highly qualified CUDA engineer.

The context in which KernelEvo appears matters. The industry is in a real boom of demand for optimized GPU computing. Training and inference of large neural networks require ever more compute, and hardware accelerators are expensive. Every percent of optimization at the kernel level translates into real savings, whether in model training time, cloud infrastructure costs or data-center power consumption. At the same time, the shortage of specialists who can write efficient low-level GPU code remains one of the industry's main bottlenecks. Automating this process is not just a convenience but a strategic necessity.

KernelEvo fits into a broader trend that has been gaining momentum over the past year and a half. Several research groups around the world are working on tools that let language models optimize low-level code. Google is actively developing similar approaches for its TPUs, and NVIDIA is investing in automating CUDA kernel optimization. However, most of these solutions remain closed and tied to specific ecosystems. An open framework from a Russian institute is a notable development, since it extends access to such technology beyond large corporations.

Of course, automatic kernel generation will not replace experienced engineers entirely. Complex architectural decisions, non-standard hardware configurations and fundamentally new algorithms still require human understanding. But tools like KernelEvo can already take over routine optimization, which makes up a significant part of a GPU programmer's work. That shifts the engineer's role from writing code to defining tasks and validating results, a shift we see in nearly every field where generative AI arrives.

AIRI's KernelEvo is one more sign that the future of high-performance computing will be shaped not only by the power of the hardware but by the intelligence of the software tools that use it. The framework is still at an early stage, but the approach itself, automatically searching for optimal implementations with language models, looks like a direction that will only gain strength.