What about Triton and Python eDSLs? (Democratizing AI Compute, Part 7)
Part 7 examines an intermediate point between graph compilers and CUDA C++: Python embedded domain-specific languages that let developers write custom kernels in familiar syntax while compiling a restricted sublanguage for the accelerator. Triton is the main case. Lattner credits it with materially improving GPU programmability, then argues that it does not deliver either semantic transparency or performance portability.
Reusing syntax is not reusing semantics
The post explains an eDSL through decorated functions. Rather than execute the function as Python, the system captures its syntax tree and translates the supported constructs. This avoids designing a parser and lets users remain in Python’s package and editor ecosystem. The cost is a semantic trap: ordinary features such as dynamic containers, exceptions, recursion, or library calls may be absent even though the code looks like Python. Errors can emerge from compiler internals, and ordinary Python debuggers may not cross the staged boundary.
This is a durable account of eDSL ergonomics. Familiar surface syntax lowers entry cost but can also create false expectations. The relevant quality is not whether a language is embedded; it is how clearly the supported subset, staging model, diagnostics, and generated code are exposed.
Triton’s chosen level of control
Triton’s block-centric model is the positive technical contribution. It replaces explicit coordination of individual GPU threads with operations over blocks and arrays. The examples show concise indexing, masked loads, and a dot-product operation that can use tensor cores. This level is lower and more programmable than a fixed graph compiler, yet higher than CUDA’s thread and memory machinery. Integration with PyTorch makes it useful for researchers and specialized training kernels.
Lattner argues that the middle layer leaks in four ways. First, a compiler may trail an expert CUDA kernel in peak efficiency. Second, code tuned for one NVIDIA generation may require restructuring for another because pipelining and asynchronous transfers remain visible. Third, the SIMT-centered model does not naturally generalize to structurally different accelerators. Fourth, the staged compiler interrupts debuggers and profilers that CUDA developers use to understand performance. These are distinct claims: portability of source, portability of performance, architectural generality, and observability should not be collapsed into one score.
Evidence limits and fast-aging comparisons
The post’s strongest material is architectural explanation and the explicit wishlist used to evaluate Triton. Its quantitative and ecosystem conclusions are weaker. The claim that a roughly 20% loss on H100 is common is not backed by a defined workload set, tuning budget, or benchmark table. The conversion of that ratio into a billion-dollar cloud-cost example assumes performance maps directly to total cost. Statements that Triton is not useful for inference, cannot use particular tools, and makes AMD portability impractical are sweeping and receive no systematic comparison in the article.
The alternatives section is deliberately provisional. At publication, Lattner describes Pallas from an outsider’s perspective and says CUTLASS Python and cuTile had been announced but were not downloadable. Conclusions about their openness, typing, adoption, and performance are therefore early impressions rather than evaluations. The Hacker News submission attracted no live discussion; its only child was deleted, so it offers no counterevidence.
Assessment
- Durable: the distinction between Python appearance and staged eDSL semantics; block-level programming as a useful compromise; and the need to evaluate portability separately across syntax, speed, hardware models, and tooling.
- Era-bound: Triton’s governance and hardware coverage, the H100 performance estimate, DeepSeek’s implementation choices, and the pre-release descriptions of NVIDIA’s Python systems.
- Caveats: the essay does not run common kernels across CUDA, Triton, Pallas, or vendor backends. Several absolute conclusions extend far beyond the examples and links it provides.
- Commercial stake: Mojo is Modular’s alternative language and MAX its AI platform. Emphasizing Python eDSLs’ semantic surprises, performance ceiling, and proliferation makes a direct case for Modular’s own language-and-stack design, even while the article fairly acknowledges Triton’s productivity.