ThunderKittens 2.0: Even Faster Kernels for Your GPUs

https://hazyresearch.stanford.edu/blog/2026-02-19-tk-2

by matt_d • 5 months ago

3 2 5 months ago

hazyresearch.stanford.edu

maralom• 5 months ago

We had read the PTX documentation countless times, but only at this point did we realize that tcgen05.copy was a typo of tcgen05.cp; the hypothetical tcgen05.copy instruction never appears again anywhere in the document.

Wow

matt_d• 5 months ago

Some highlights (by Stuart Sul):

Tensor core and memory pipelining: it turns out some tensor core instructions are implicitly pipelined, without proper documentation. Identifying these implicit semantics and the resulting pipelining tactics can boost your throughput by up to 10%.
Hinting the PTX assembler properly: even logically identical PTX code can compile into meaningfully different SASS instructions, depending on how you write it. Signaling the assembler with the right instruction patterns is significant for minimizing latency.
Occupancy: with all the modern GPU features, it gets tricky, and it is (again) poorly documented. Distributed shared memory doesn’t behave identically across all SMs, and 5th-generation tensor core instructions silently cap occupancy.

ThunderKittens 2.0: Even Faster Kernels for Your GPUs

Related Stories