TVM paper

TVM joins graph-level operator fusion with schedule-level loop transformations in one compilation stack.

Two levels, one stack

The claim is that deep learning compilation needs optimisation at two levels and that they meet. A graph IR sees the whole dataflow and can fuse operators. A tensor-expression layer separates an operator’s computation from its schedule. Fusion at the graph level deletes intermediate tensors; the schedule then decides how the fused operator is looped, tiled, vectorised and parallelised.

The learned cost model

The part to read closely is the search. A schedule is a point in a large space, and running every candidate on hardware is too slow, so the paper simulates candidates, extracts features, and fits a model to measured runtimes. The model ranks candidates; it does not invent schedules. That distinction is the one people lose when they say TVM learns to optimise.

What it does not claim

The graph IR is not a frontend and not a semantics for arbitrary Python. The search space is whatever the schedule language can express, which is why later work replaced hand-written schedules with sampled ones and kept the learned model. The evaluation is against vendor libraries on whole networks, which is the right comparison and a moving target.

How to read it

Read the graph-level section and the cost-model section, and read the operator-level section with a matrix multiplication in front of you. Skim the hardware backends you do not target. Then read the architecture documentation for the current split between the graph dialect and the tensor dialect, because the names have moved since 2018.

Afterwards you should be able to say which of the two levels a given optimisation belongs to, and why a fusion decision made without the schedule that will run it can lose.

Things that break

Treating the levels as independent: fuse aggressively at the graph level, then schedule each fused operator without noticing that the fusion created a reduction over a large intermediate. The tell is a fused kernel that reads more bytes than the unfused pair, which the cost model would have ranked lower had it been asked about both levels at once.

Sources