NVIDIA's New CompileIQ Finds Hidden GPU Kernel Performance by Tuning Compiler Parameters

NVIDIA has unveiled CompileIQ, a system that automatically selects optimal compiler parameters for GPU kernels. It is aimed at the last mile of performance optimization, when standard methods (quantization, kernel fusion, algorithm optimization) have already been exhausted.

When manual optimization hits a wall

Imagine a scenario: developers have spent weeks optimizing LLM inference on a GPU. They tuned batch sizes, quantized the model to FP8, added flash attention, merged micro-kernels into a single kernel and rechecked memory usage. The profiler says: "Nothing left to optimize." But CompileIQ finds another 5-10% speedup simply by changing compiler flags.

Why is that possible? The NVCC compiler (CUDA) has hundreds of parameters: inlining levels, caching strategies, register management, warp scheduling. Their combinations yield millions of variants. Checking them by hand would take months. Any single flag can radically change code performance on a specific GPU architecture.

How CompileIQ finds speedups

The system uses machine learning to search for optimal parameters automatically:

• Search space — the system generates combinations of compiler flags, starting with typical ones and moving on to exotic ones
• Profiling — each variant is compiled, loaded onto a GPU and tested on a real workload
• Model training — an ML algorithm identifies correlations: which flags affect speed for a given type of code
• Adaptation — parameters are tuned to a specific architecture (H100, L100, RTX4090)
• Validation — the final configuration is checked across several workloads for stability

The result: instead of manually trying hundreds of combinations, the system finds a near-optimum in hours of compute.

Why this saves millions

In the era of large language models, every percent of performance is real savings. On cloud GPU clusters, an H100 instance costs almost twice as much as an A100. If CompileIQ delivers a 5-10% speedup, a company can save millions of dollars on infrastructure simply by not buying additional GPUs. For a startup with 100 GPUs, that could mean a difference of millions per year. For companies deploying private models (Llama, Mistral, Code Llama), every speedup directly improves latency for end users, which is critical in production.

"Compiler-level optimization is the last performance frontier that most developers ignore because it is too complex.

CompileIQ changes that."

What this means

CompileIQ reflects a new trend in AI: machine learning is being used to optimize machine learning itself. Developers no longer need to spend months experimenting with compiler flags: give CompileIQ a profiler, and the system will find the hidden speedup automatically. This lowers the barrier to entry for teams without deep experience in low-level GPU optimization and makes this critical area more accessible.