Triton paper

The middle ground

CUDA exposes threads and leaves coalescing, shared memory and register pressure to you. A tensor DSL hides those decisions and with them the ability to write anything the DSL did not anticipate. The paper’s claim is that a program written over blocks, with the compiler choosing how each block is distributed over threads and memory, gets most of the performance with far less ceremony.

What the paper actually built

A tile-level IR and the passes that make it fast on a GPU: layout assignment for distributed tensors, conversion between layouts where an operand needs a different distribution, shared-memory allocation and swizzling, and instruction selection over the resulting tiles. The matrix multiplication case study is the argument, with block sizes and a small number of configuration parameters reaching the range of a hand-tuned library.

What it does not claim

Not that tile-level is always enough: the paper is explicit that some kernels need control the model does not offer. Not distributed execution, not anything about non-NVIDIA targets. The 2019 implementation is also not the Triton you will install; today’s compiler is an MLIR pipeline with TTIR and TTGIR, and the paper’s IR is its ancestor.

What to read

The IR and layout sections, then the matrix multiplication section with the tutorial open beside it. Skip the comparison against other DSLs on a first pass; the comparison that matters is against your own hand-written kernel.

Afterwards, take the Triton matmul tutorial and annotate every decision the compiler made for you: the output tile per program, num_warps, num_stages, and each place a layout conversion appears.

Things that break

Writing Triton as if it were CUDA, with one program per output element and index arithmetic per lane. It compiles and it is slow, because the model’s advantage is per-tile work and the compiler cannot recover a tile you never expressed. The tell is a kernel whose time scales with elements rather than tiles, and TTGIR full of layout conversions.