Fudan and Huawei Xiaoyi Teach an AI Agent to Write Operators for Ascend NPUs: 95.5% Success Rate

A team from Fudan University, working with Huawei Xiaoyi, has pushed the share of successfully generated operators for Ascend NPUs to 95.5%. This is a coding agent that writes low-level code for a hardware platform language models know poorly, then compiles, runs, and reworks it on its own until it produces a working result.

Where Speed Gets Lost After Deployment

By the time a model is ported to Ascend, the computational logic — layers, attention, normalizations — is already fixed. What comes next is work users typically never see: deciding exactly how operators map onto the hardware. The same attention computation can be implemented in countless ways. Implementations differ in how data is split into blocks, how that data moves between memory tiers, how work is distributed across compute units, and where synchronization points sit. Runtime can vary dramatically as a result.

Traditionally, this tuning is done by engineers who know the chip architecture deeply. There are few such engineers, and modern models have a lot of operators. So optimizing for a specific NPU becomes a bottleneck for the entire deployment pipeline.

An Agent on a Platform It Never Studied

Large language models can generate code. Coding agents go further: they call tools, assemble a project, run tests, and revise based on the results. For popular platforms, this loop works well because training data is full of examples.

Ascend is harder. Most open-source code targets other stacks, so the model has less knowledge of the finer points of NPU programming. The researchers framed the question this way: what should an agent do if it has to compile, debug, and optimize code on a platform with an insufficient knowledge base? The answer from the Fudan and Huawei Xiaoyi team: teach it to rely on feedback from the platform itself — that is, on the results of compilation and execution.

The project's bottom-line metric is a 95.5% success rate in operator generation.

What We Know About the Project

• Participants: Fudan University and the Huawei Xiaoyi team.
• Target platform: Ascend NPU.
• System type: a coding agent with tool calling, compilation, execution, and iterative refinement.
• The problem: making efficient use of hardware once the model's computational logic is already fixed.
• What affects speed: block tiling, data movement, parallelism, and synchronization.
• Headline result: 95.5% successful operator generation.

Why This Matters for the Ascend Ecosystem

For Chinese accelerators, the core problem has long stopped being just about silicon. Competitiveness hinges on the software layer: operator libraries, compilers, and tuning tools. If an agent can take over a significant share of the manual work of writing and debugging operators, the barrier to entry for Ascend drops. New architectures and nonstandard attention blocks could be supported faster, without waiting for a hardware specialist to become available.

The approach also generalizes. Any accelerator maker runs into the same issue: models barely know their stack. A scheme where an agent learns from compiler and profiler feedback can carry over to other NPUs and GPUs.

What Teams Working With Ascend Should Watch

The practical takeaway is simple: build automation around a "generate, compile, run, fix" loop. The sharper the feedback signals, the better the agent handles an unfamiliar platform.

Questions remain that will determine the method's real-world value. How fast do the generated operators run compared to hand-written implementations? How robust is the approach across new types of computation? Will it be available outside Huawei's and the university's labs? For now, the 95.5% figure shows that agent-driven operator generation for Ascend has already moved beyond a mere demonstration.