Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration

https://github.com/mlecauchois/micrograd-cuda

by cataPhil • 2 years ago

178 13 2 years ago

cataPhil4• 2 years ago

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.

quibono2• 2 years ago

Is your work publicly available anywhere? I'd love to have a look if possible.

cataPhil4• 2 years ago

Hi :) Yes it's in the post url or here: https://github.com/mlecauchois/micrograd-cuda

lagrange77• 2 years ago

Awesome, thank you for publishing it. I was planning the same, but didn't find the time yet.

coolThingsFirst• 2 years ago

Lovely project, what did you find as the beat place to learn about cuda?

cataPhil4• 2 years ago

Thanks! I highly recommend the book Programming Massively Parallel Processors

lairv• 2 years ago

If you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM

cataPhil4• 2 years ago

Thanks! I will probably do that this weekend

suhacker256• 2 years ago

Very cool!!

zer0zzz• 2 years ago

Cool

kosolam• 2 years ago

Awesome. Thanks

Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration

Related Stories