Fusion benchmark

Two paths on one rig

Build both from the same source with a switch, FUSED=0 and FUSED=1, same shape, same buffers, same reference check, and run them interleaved in one session with warmup discarded and at least twenty samples each. The baseline is the unfused chain at the same optimization level. If the fusion pass runs on both binaries, the baseline is fused and the difference you report is measurement noise.

For whole-process comparisons hyperfine is enough, with --warmup and --runs, and --export-csv to keep the rows. For a kernel inside a harness, the harness’s own loop is the one that can interleave per sample.

Report bytes beside time

Two numbers per path: the analytic traffic for the shape, and the measured bytes per second. Google Benchmark produces the second if the kernel reports what it moved with state.SetBytesProcessed(...), which adds a bytes_per_second counter next to the time.

For f32 matmul plus bias plus relu at M by N, the unfused chain moves 20 bytes per output element and the fused path 4, with the A and B traffic shared by both. So the fusion removes at most 16 * M * N bytes. The measured difference says how many of those bytes the memory system actually charged for.

fused     512x512  bytes=16*M*N  <median>  <spread>  <speedup>
unfused   512x512  bytes=20*M*N  <median>  <spread>

Expected shape

The advantage grows with M * N and saturates once the unfused intermediate stops fitting in the last-level cache. Below that size the intermediate stays on chip, the extra traversals cost cache bandwidth rather than DRAM, and the speedup shrinks toward the spread.

Things that break

Bytes reported without a level. “Removes 64 MB of traffic” is a claim about DRAM written by someone who did not measure DRAM. State the level, or state that the number is an upper bound computed from the shape. The second failure is a baseline that was never unfused: a pipeline flag left on, or the fusion applied by a later pass to both binaries, giving a ratio near one that gets published as “fusion does not help here”.