I'm writing a short series of tutorials on FlashAttention: from theory to efficient CUDA kernels. Part 1 is the theoretical foundation. It walks through a modern algebraic formalism showing that FlashAttention is an associative operation, which lets you treat it as a regular reduction on the GPU and apply all the same scheduling optimizations. Some recent MLSys and CVPR papers lean on this framing, and I find it much more powerful than the original. Overview: Safe softmax, Welford's vari...