Fused matmul+bias+relu

Two kinds of evidence

The claim is that nothing between the matmul and the epilogue is materialized, and it needs evidence in the IR and in the traffic. Either one alone can be true while the other is false.

IR

After bufferization, a whole-tensor intermediate is an allocation of output size plus a copy that fills it. Count both:

mlir-opt --one-shot-bufferize="bufferize-function-boundaries" fused.mlir \
  | grep -c 'memref.alloc'
mlir-opt --one-shot-bufferize="bufferize-function-boundaries" fused.mlir \
  | grep -c 'memref.copy'

The flag spelling has moved between releases; confirm it against mlir-opt --help | Select-String bufferize on your build, and print the pipeline beside the counts, because a count without a pipeline is not reproducible. What should remain is the boundary allocation for the output and, at most, a tile-sized buffer.

Traffic

Count bytes per output element. Fused: 4 written. Unfused: the matmul writes 4, the bias pass reads and writes 8, the relu pass reads and writes 8, for 20. The difference, 16 bytes per element, times M * N, is the traffic removed at the DRAM level if the intermediate would not have stayed in cache. Report it as an upper bound, and put the measured bytes_per_second beside it.

fused     512x512  allocs=1 copies=0  <median>  <spread>
unfused   512x512  allocs=2 copies=0  <median>  <spread>

Things that break

The intermediate leaves the IR and returns as a copy: bufferization decided a buffer was not writable, or that a read after write needed a distinct destination, and memref.copy performs the traversal you deleted. The allocation count does not move, so a check that only counts alloc reports success. The second failure is a flat timing curve, which is a real result: the intermediate was cached at that shape, so report two shapes before concluding the fusion was pointless.