Compiler versus hand-tuned gap

The definition

Headroom is (hand - generated) / hand, both medians, each with its spread, and it is a percentage of the ceiling rather than of the machine. Absolute GFLOP/s belongs to the rig; the fraction is the part that survives a change of host.

One binary, two kernels

Put both kernels in one binary and select with an argument, so the compiler version, -march, -mabi and -mrvv-vector-bits are identical by construction. Interleave the two in one session, discard warmup, keep at least twenty samples, and check both against the same reference before timing anything.

matmul_generated   512x512x512  <median>  <spread>  20
matmul_intrinsics  512x512x512  <median>  <spread>  20
headroom                        <headroom>          20

A <headroom> whose bands overlap is not a gap. The honest sentence is that the two paths are indistinguishable at this sample count, and the next step is more samples or a quieter rig.

Where the gap comes from

The percentage sizes the problem; the disassembly names it. Four causes account for most of it, and each has a signature in the generated loop: no register tile, so the load to MAC ratio is near one; a vsetvli inside the body, so vtype is re-derived per iteration; no unroll, so there is one branch per MAC group; and a materialized intermediate, so the epilogue is a second loop over memory rather than a few instructions before the store.

Expected shape: the gap is widest on large square shapes, where a missing tile costs the most, and narrowest on thin matrices, where both kernels are limited by the same traffic. A flat curve across shapes usually means the two paths are not actually different, so check the build flags before plotting.

Things that break

The two kernels differ in a compile flag. One was built with -mrvv-vector-bits=zvl and the other was not, or one at -O2 and the other at -O3, so the percentage contains a flag change and the results file does not say so. Print the compile command with the table. The second failure is arithmetic rather than scheduling: if the hand kernel contracts into FMA and the generated one does not, part of the gap is one rounding per MAC, and no amount of tiling will close it.