Reproduce benchmark script

One entry point

#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
mkdir -p results
: "${QEMU_CPU:=rv64,v=true,vlen=256,elen=64,vext_spec=v1.0}"
commit=$(git rev-parse --short HEAD 2>/dev/null || echo nogit)
make -B bin/matmul_scalar bin/matmul_rvv
{
  printf '# %s\n' "$commit"
  qemu-riscv64 -cpu "$QEMU_CPU" ./bin/matmul_scalar --reps 20
  qemu-riscv64 -cpu "$QEMU_CPU" ./bin/matmul_rvv --reps 20
} > "results/$commit.tsv"
awk -f scripts/table.awk "results/$commit.tsv" > results/table.md
cat results/table.md

Every line earns its place. set -eu turns a missing binary into a failed run instead of a short table. cd makes the script work from any directory. make -B rebuilds both paths under one set of flags. A single QEMU_CPU variable stops the two invocations from drifting apart, which is the commonest way a comparison stops being one. The .tsv is the artifact; awk keeps the table out of human hands.

make bench already exists from the baseline work. This script is the layer above it: two paths, one rig, one artifact, one table. It prints the table and writes it; printing alone leaves the artifact in a scrollback buffer, which is not something a reader can diff.

The stranger test

Clone into a fresh directory and run ./scripts/repro.sh. Compare what it prints with the README table. Any step you needed and did not write down is a missing line, and the fix belongs in the script rather than in more README prose. If their numbers differ by more than the spacing between two local runs, the script is pinning something it should not: an absolute path, a cached object file, or an environment variable that only you export.

Things that break

A shallow clone or a missing git makes commit fall back to nogit, so two runs overwrite each other and the artifact loses its provenance. Plain make instead of make -B silently benchmarks yesterday’s binary. And awk reading a tab-separated file whose field count drifted produces a table with shifted columns and no error, which is worse than a crash: it still looks like a result.