Remaining 2x story

Decompose the gap

The gap between generated and hand-written code is a total, and a total is not an explanation. Split it into terms you can measure: instruction selection and scheduling, register allocation and LMUL choice, tail handling, epilogue fusion, layout, and the overhead of the harness itself. Each term gets an experiment that changes it while holding the others fixed.

generated, current schedule        <GFLOP/s>
same schedule, hand intrinsics     <GFLOP/s>
same intrinsics, LMUL swept        <GFLOP/s>
hand kernel, best LMUL and tile    <GFLOP/s>

The differences are the terms. If the intrinsic version at the same schedule recovers most of the gap, the loss is in codegen, not in the schedule. If it does not, the schedule or the tiling is the larger term, and no amount of instruction selection closes it. The roofline bounds the exercise: a kernel already at the applicable ceiling has no gap left to attribute.

What would close it

The writeup has to name a change and a gain, not a direction. “Improve register allocation” is a direction. “Hold four accumulator rows at LMUL=2 and peel the K loop so the tail does not force a narrower LMUL” is a change, and it comes with a predicted gain and a test.

Part of the gap may be out of reach in the remaining time. Saying so, with the term and its size, is a stronger result than claiming the whole gap is one pass away. The judgment the post demonstrates is knowing which term is worth buying.

The gap is whatever the table measured. If the plan calls the remainder a 2x, the decomposition has to show where the factor comes from; if the measurement says otherwise, the measurement wins.

Things that break

Adding independent percentages that overlap produces a decomposition larger than the gap. Comparing the generated and hand-written kernels at different shapes measures the shape. And attributing the gap to “LLVM” without an experiment is the same intuition the roofline node warned against, applied to the compiler.