Softmax benchmark

Shapes, not a shape

Sweep columns over 128, 256, 512, 1024, 2048 and 4096, with rows sized so each shape moves about 64 MB. Report nanoseconds per element and GB/s beside the speedup, because softmax is a memory operation with an exponential attached: three reads and one write per row when it must be streamed, one read and one write when it stays resident.

What the curve should look like

At 128 columns the kernel is mostly call overhead and the ratio sits inside the noise. The ratio rises while the row stays resident, peaks near the column count where the register file still holds the row and its temporaries, and falls once the row must be streamed a second time for normalisation. That knee is the result. One row length is a point on the curve, and the point you pick decides the claim you are making.

Both paths, one session

Build the scalar and vector softmax with the same flags, run them interleaved with twenty samples each, and keep the median and the spread per shape. The scalar baseline calls the same libm exponential, because a baseline with a cheaper approximation compares two functions rather than two implementations. Put the rig line under the table: QEMU CPU string, VLEN, LLVM commit, sample count.

A second column

The speedup table says how much faster. It does not say whether the answer is right, so keep a column for the maximum relative error against the scalar reference at the same shape and fail the run when it exceeds the tolerance. A fast wrong row is worse than a slow one, and the error column is what makes the table a result instead of a stopwatch reading.

Things that break

A column count that divides evenly by VLMAX at every group size, such as 1024 at VLEN=256, so the tail path never runs and a bug in it never shows. A scalar baseline built at -O0. A timer that includes the allocation. A ratio with no band, when the band is what tells a reader whether the knee is real.