TorchInductor design docs

The pipeline in three moves

TorchDynamo captures a graph from Python frames and guards the assumptions it made. AOTAutograd traces forward and backward through that graph. TorchInductor takes the resulting graph of ATen operations, schedules it, and emits either Triton or C++.

The scheduler is the part to study. It builds a loop-level IR of the graph, decides which nodes fuse into one kernel, tiles reductions, picks layouts, then hands each kernel to a code generator. The Triton it emits is ordinary Triton, which means you can read it and compare it with what you would have written.

Guards decide what compiles

A guard is a condition on the traced assumptions: shapes, dtypes, device, and sometimes values. A call that fails a guard recompiles, and a construct Dynamo cannot trace becomes a graph break and runs eagerly. Most reports of the compiler making something slower are one of those two, and the logs say which.

What these docs are and are not

They are design notes and debugging guides, not a specification. Read the torch.compiler overview for the pipeline, the design post for the IR and the symbolic-shape problem, and the Dynamo deep dive for guards and graph breaks. Use TORCH_LOGS and the debug directory to see the graph and the generated kernel. Skip the provenance page until you are debugging one kernel.

Afterwards you should be able to compile a small model, find the Triton generated for one operation, and say which scheduling decision produced it.

Things that break

Reading Inductor as a general-purpose compiler. It compiles the graphs Dynamo can capture, with the fusion and layout decisions its scheduler makes, and it falls back to eager at every boundary it cannot cross. The tell is a speedup measured on a model whose hot loop is a graph break, or a benchmark that improves when you turn the compiler off.