Modern GPU Programming for MLSys

https://mlc.ai/modern-gpu-programming-for-mlsys/

by crowwork • 30 days ago

80 25 30 days ago

"Modern [NVIDIA GPU] Programming for ..."

Everything after "Pipelining GEMM with TMA" (inclusive) is specific to NVIDIA. Which is fine but the title (of the guide itself) is clearly misleading.

nh23423fefe2• 26 days ago

Our main target is the Blackwell generation,

misleading?

mathisfun12310• 26 days ago

what is it with hn people where they willfully misinterpret the simplest observations;

the title (of the guide itself) is clearly misleading.

...

title: the distinguishing name of a written, printed, or filmed production

do you understand now? or do i need to also define for you the word misleading?

hazard• 26 days ago

This looks great, but I'd really like to see associated exercises (and solutions) to make it useful for self-study

throwaw122• 26 days ago

So many frameworks are being built.

What are the state of the art frameworks in ML programming area? Similar to what React is for web and tailwind for CSS

Triton, ONNX, JAX, PyTorch, cublass, .....

I know they might be for different purposes, but having some idea what is for what and when to use would be helpful

mathisfun12310• 26 days ago

ONNX, JAX, PyTorch

these are model-level frameworks

Triton

this is a kernel DSL

[cublas]

this is a BLAS library built atop CUDA

I know they might be for different purposes, but having some idea what is for what and when to use would be helpful

when people ask this question i always ask: who are you and what is your job? if you're not an ML/DL/AI person then you knowing the specifics is about as useful as me knowing the specifics of react/express/angular/tailwind/django/whatever as an ML person. this is not meant to be condescending, this is meant to allay your anxiety, ie that if you ever find yourself in the position where you have to know these things for your job, it won't be that hard to figure out (just like it isn't that hard to figure out the difference between react and express and django if you're a webdev).

throwaw122• 26 days ago

I am a product engineer in yet another enterprise SaaS CRUD shop, who wants to learn more about the landscape and find the way to enter it eventually.

reinitctxoffset• 26 days ago

I can't signal boost this enough.

I spent months, months of late nights watching commits to nvfuser and shit, I wrote a SASS decompiler instrumented everything trying to learn Blackwell.

This is the first time I've seen something so clean, just a real work of scholarship on it.

My hat is off to the authors and the contribution it represents.

If I would caution a reader anything it's that the 2CTA (sm_100 sm_110) patterns here are different on 1CTA in important ways and it's not a better / worse thing, they are good for different workloads.

Really outstanding work. I proves q lot of this in lean4 and published but I got lazy short of really doing the pedagogical work.

This is what you should be starting with if you want to max out 2CTA gear, it's immaculate.

zander_jiang• 24 days ago

The book is really well written.

Modern GPU Programming for MLSys

Related Stories