Fudan and Huawei Xiaoyi Teach an AI Agent to Write Ascend NPU Operators: 95.5% Success Rate

A team from Fudan University and Huawei Xiaoyi has raised the share of successfully generated operators for the Ascend NPU to 95.5%. The system is a coding agent that writes low-level code for a hardware platform that language models know poorly. It compiles the code, runs it and reworks it until it works.

Where speed is lost after deployment

When a model is ported to Ascend, its computational logic is already defined: layers, attention, normalizations. What follows is work that users usually never see. Engineers must decide exactly how the operators map onto the hardware. The same attention computation can be implemented in many ways. The implementations differ in how data is split into blocks, how it moves between memory levels, how work is distributed across compute units and where synchronization points sit. Execution time can vary significantly as a result.

Traditionally, engineers who know the chip architecture well do this tuning. There are few of them, and a modern model contains many operators. Optimizing for a specific NPU therefore becomes a bottleneck for the entire deployment cycle.

An agent on a platform it was not trained on

Large language models can generate code. Coding agents go further: they call tools, build the project, run tests and make fixes based on the results. This loop works well on common platforms because the training data contains many examples.

Ascend is harder. Most open-source code is written for other stacks, so the model has less knowledge of the finer points of NPU programming. The researchers framed the question this way: what should an agent do when it must compile, debug and optimize code on a platform with an insufficient knowledge base? The answer from the Fudan and Huawei Xiaoyi team was to teach it to rely on feedback from the platform itself, meaning the results of compilation and execution.

The project's final metric is a 95.5% success rate in operator generation.

What we know about the project

• Participants: Fudan University and the Huawei Xiaoyi team.
• Target platform: Ascend NPU.
• System type: a coding agent that calls tools, compiles, runs and iteratively refines code.
• Problem: using the hardware efficiently after the model's computational logic is already fixed.
• What affects speed: block partitioning, data movement, parallelism and synchronization.
• Main result: 95.5% success rate in operator generation.

Why this matters for the Ascend ecosystem

For Chinese accelerators, the main problem has long been more than silicon. Competitiveness hinges on the software layer: operator libraries, compilers and tuning tools. If an agent can take over a significant share of the manual work of writing and debugging operators, the barrier to entry for Ascend drops. New architectures and non-standard attention blocks could be supported faster, without waiting for a hardware specialist to become available.

The approach also applies more broadly. Every accelerator vendor faces the same problem: models know its stack poorly. A scheme in which an agent learns from compiler and profiler output carries over to other NPUs and GPUs.

What teams working with Ascend should watch

The practical takeaway is simple: build automation around a "generate, compile, run, fix" loop. The more precise the feedback signals, the better the agent handles an unfamiliar platform.

Several open questions will determine the method's real value. We need to know how fast the generated operators run compared with hand-written implementations. We need to check how robust the approach is on new types of computation. We need to find out whether it will be available outside the labs of Huawei and the university. For now, the 95.5% figure suggests that agent-based operator generation for Ascend has already moved beyond a demo.