CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution

A systems paper from the FlashInfer/TVM lineage (Zihao Ye and Tianqi Chen among the authors; NVIDIA and CMU), arguing that GPU kernel agents and GPU programming languages have been improved separately while the gap between them is where expert kernels are lost. Kernel agents treat the compiler as a fixed black box returning errors, pass/fail, and one latency number; the DSLs an agent might write are built for humans — tile-level DSLs (Triton, Helion, TileLang, cuTile) hide the warp specialization, barrier choreography, and memory-tier placement that separate expert kernels from merely correct ones, while low-level DSLs (CuTe) expose that control through a layout algebra that makes agent errors likely and hard to localize. CAKE co-designs both sides: agents author Cake IR, a typed, hardware-explicit schedule representation, and the compiler harness answers with localized diagnostics — and is itself a target of evolution. Targets NVIDIA Ampere through Blackwell; most evidence is B200.

The representation bet

Cake IR records how the machine is driven — named warp roles, declared memory regions and pipelines, explicit barriers gating producer–consumer handoffs — while lowering derives the mechanical consequences (barrier addresses, phase bits, TMEM offsets, descriptor encodings). The deliberate anti-position is on layout: rather than an algebraic layout calculus, the agent writes concrete commitments (an SMEM view offset, a swizzle tag, a TMA coordinate) and the compiler checks their mutual consistency and hardware legality. Because schedule decisions are explicit and typed, analyses run before compilation and tie findings to the responsible resource, role, or stage — the harness’s contract is a seven-category table of pre-compile gates (safety, conformance, data consistency, schedule semantics), an execution gate (numerical validation against an external reference), and non-blocking reports and hints (cost model, optimization guidance). Notably, the IR itself was not designed top-down: it was bootstrapped by agents mining a corpus of production CUDA kernels for recurring schedule patterns, biased by human hardware expertise toward the Blackwell model, validated against eight design principles, and grown port by port (Appendix A).

The evolution loop

The harness is maintained primarily by agents under human merge gates. Two coupled paths: agents mine production kernels and hardware documentation for missing capabilities (new instruction forms, resource types, synchronization idioms) and propose IR extensions; and recurring failures are distilled upward — an opaque runtime crash becomes a verifier rule, a repeated illegal lowering becomes a static check, a systematic cost-model misprediction becomes a calibration target. Changes are test-gated across the kernel corpus because primitives and analyses must evolve together. The agent workflow itself is conventional (generate structurally distinct candidates, filter by verifier and cost model before spending GPU time, evaluate against the oracle, route evidence to candidate/verifier/model/vocabulary); all agent tasks use one model (GPT-5.6-sol at xhigh) precisely so results attribute to the environment, not model capability.

What the evidence shows

The headline experiment is unusually well-controlled for this literature: matched three-run clean starts on one workload (Flash-KMeans assign, a compute-bound BF16 GEMM-and-reduction at one fixed shape, B200), same model, scaffold, task statement, oracle, and 80M-token budget, with low-level reference implementations withheld (enforced in isolated environments, audited afterward). The treatment arm writes Cake IR, the control writes CUDA C++/PTX directly. Result: Cake IR crosses the tuned FlashML Triton baseline by ~55M tokens and reaches a median best of 1.144× (plateauing 3/3 runs, 1.89 h median active evolve time); direct CUDA/PTX ends at 0.928× (0/3 plateaus, 3.73 h). The attainment curves show the CUDA arm climbing more slowly while spending nearly twice the active evolve time — consistent with the design claim that cheap pre-compile analysis filters candidates before they consume GPU time, though the paper does not decompose the time difference.

Around that controlled core sit production results of varying evidential weight, with external receipts as upstream FlashInfer PRs: agent-generated Kimi Delta Attention prefill at 2.05× geometric mean over official FlashKDA (black-box baseline only, bitwise-correct, validated in Kimi-K3 serving under SGLang); TinyGEMM at 18–23% kernel-time reduction across 35 shapes; an Alpha-MoE W8A8 Hopper-to-Blackwell megakernel rewrite (the paper carefully splits its 6.2×/4.0× API-level wins from the 1.2× GPU-span remeasurement — most of the API gain is launch fusion, not kernel speed). Known-kernel reproduction against TensorRT-LLM, CUTLASS, DeepGEMM, FlashAttention-4, and FlashInfer: ten of eleven fixed comparisons meet or exceed the reference (strongest: MQA indexers at ~1.27×), the eleventh at 96.5% — with the honest gloss that above-parity entries reflect search during porting, not transcription fidelity. Section 6 treats generalization as a separate stage with its own objective: shape-bucketed portfolios behind dispatchers, with the shape domain declared before tuning so the dispatcher cannot be tuned on the evaluation set. Dispatcher-inclusive geometric means on GB200: 1.42×/2.12×/1.80× for KNN build/search and KMeans across 112/198/124 shapes.

Against the library

On liu2026‘s what-evolves axis this is a locus the shelf lacked measured evidence for: not the model (fixed), not the agent scaffold (fixed), not the artifact population (novikov2025‘s locus, which CAKE cites as complementary — AlphaEvolve evolves programs under a fixed scorer; CAKE evolves the environment the search runs in), but the domain compiler between agent and hardware. The consensus loop of the Self-evolving agents shelf reappears intact at this new locus: evidence before proposal (failures distilled into rules), bounded test-gated deltas, the gate outside the loop (corpus tests plus human merge gates) — the zhang2026 protocol shape, but with a controlled matched experiment that harness’s evaluation lacked. Against the lattner2025k analysis of the Python-eDSL landscape, CAKE is a third position: Lattner’s series argued tile DSLs trade performance ceiling for productivity; CAKE argues the trade is an artifact of designing for humans, and an agent audience changes the optimal point — full hardware explicitness, no layout algebra, verification instead of abstraction. And the paper closes the loop with this week’s other ingestions: its design principles explicitly invoke Tao’s proof-abundance observation (generation shifting the bottleneck to verification and understanding, cited to his Mastodon thread) — tao2026‘s digestion problem, imported as a compiler design requirement (inspectable, reusable artifacts).

Assessment

• Durable: the co-design thesis (the environment is a design variable, not a fixture); the matched clean-start methodology — fixed model and budget, implementation-hidden, audited — as the way to attribute gains to environments; generalization-as-separate-stage with declared shape domains as leakage discipline; upstream PRs as externally checkable receipts; failure-to-rule distillation as the transferable evolution mechanism.
• Era-bound: Blackwell/B200 specifics (TMEM, warp-specialized pipelines), the GPT-5.6-sol dependency, token-budget accounting, and the current DSL field (Triton/CuTe/Gluon) it positions against.
• Caveats: the controlled claim rests on one workload at one shape with three runs per arm — strong internal validity, narrow scope; the comparison bundles representation and harness feedback, so their individual contributions are not separable (the paper’s own framing — “the environment” — concedes this); NVIDIA authors evaluate against NVIDIA-ecosystem baselines throughout, and the verifier and cost model are characterized by contract rather than internals, so soundness coverage is asserted, not shown (Appendix C admits false positives and negatives occur); portability of the whole approach is explicitly unmeasured beyond NVIDIA. The disclosure quality is notably high — GPU-span vs API-level splits, LOC-comparison disclaimers, internal-metric denominators flagged — which earns the claims the benefit of precise doubt rather than blanket skepticism.