glossary/tooling/cutlass
CUTLASS
also CuTe
NVIDIA's C++ template library for building GEMM and related kernels that hit tensor core peak, and CuTe, its layer for describing the layouts and tilings that make that possible. It is how the fastest matmuls, attention kernels and MoE grouped GEMMs on Hopper and Blackwell are written, exposing the asynchronous copy engines and warp-group instructions that Triton abstracts away. The entry cost is high; the ceiling is the hardware's.
wgmma and TMA
The Hopper instructions for warp-group matmul and bulk tensor copies that CUTLASS exposes and peak GEMMs depend on.
Related
- glossary/
- GEMM
General matrix multiply, the operation the projections and feed-forward layers of a transformer reduce to.
- Tensor cores
The GPU units that do small dense matrix multiplies in one instruction, at far higher rate than the general-purpose cores.
- Triton
A Python-embedded language and compiler for writing GPU kernels at the level of blocks of data rather than individual threads.
- FlashAttention
An attention kernel that never writes the full attention matrix to HBM.
- sources/
- topics/