What loss.backward() actually does

https://oraziorillo.com/blog/what-loss-backward-actually-does/

by oraziorillo • 11 days ago

18 3 11 days ago

Following up on my post on microcrad — many of you liked and supported the project, thank you for that — I published a long-form piece on automatic differentiation, where I explain backpropagation in simple terms so that everyone will (hopefully) be able to understand it. Questions/suggestions are welcome!

imInGoodCompany• 6 days ago

As someone who has only had a very basic understanding of how backpropagation works, I found this to be really easy to follow along with, so thanks for that.

The article is also clearly entirely written, or at least drafted, by AI. The explanation is great so I'm not sure how much it matters. It does feel like the myriad of voices and writing styles on the Internet is slowly being drowned out, which makes me a bit sad.

busfahrer• 6 days ago

For anybody else trying to grok backprop, I found it very helpful to look at karpathy's microgpt, a barebones LLM in 200 lines of python. The autograd itself is just 40 lines, and reading that code and then trying to do a clean room implementation of it is a great exercise:

https://gist.github.com/karpathy/8627fe009c40f57531cb1836010...

https://karpathy.github.io/2026/02/12/microgpt/

What loss.backward() actually does

Related Stories