Roofline paper

What the paper claims

The roofline is one inequality: attainable FLOP/s is min(peak_flops, bandwidth * arithmetic_intensity), where intensity is flops divided by bytes moved from DRAM. The bound itself is arithmetic. The claim is that two machine constants and one kernel constant explain most of the spread across kernels, and that the distance to the nearer ceiling names the next optimisation.

Do the arithmetic yourself

The paper is unreadable until you have computed an intensity by hand. Fill in the three lines below from the traffic each kernel actually causes, before you read the case studies:

square f32 matmul:     2N^3 flops / 12N^2 bytes = N/6
f32 vector add:        ?            ?              ?
f32 row-wise softmax:  ?            ?              ?

The matmul number is algebra, not a measurement, and it moves with N, which is why a 512 matmul is hungrier per flop than a 4096 one. If your three numbers do not separate a matmul from a vector add by two orders of magnitude, redo them.

Two ceilings, both stateful

Both ceilings are measurements at a dtype and a clock; a plot that omits either is not reproducible. An f16 tensor-core ceiling and an f32 CUDA-core ceiling differ by a large factor on the same die, and a boost clock is not a sustained clock. Write f32, 1.4 GHz sustained beside the number, not peak.

What to read and what to skip

Read the model section, then the case study where a kernel is lifted from one ceiling to another. The CACM article is the short version; the 2008 technical report carries the derivations and the multicore case studies. Skip the per-architecture detail on hardware you will never touch.

Afterwards you should be able to take one kernel of your own, state its intensity, place it against the ridge, and say what would move it.

Things that break

The plot is drawn against datasheet peaks, so every kernel lands at a fraction of a ceiling the part cannot hold and the picture recommends nothing. The other failure is ideal traffic: uncoalesced loads move several times the bytes the shape suggests, so the true intensity is lower and the dot belongs further left. The tell is that the bytes counted by a profiler do not match the denominator you wrote down.