Deep distilling: Automated discovery of algorithms from data

https://github.com/pauljblazek/deepdistilling

by dr_kiszonka • 2 years ago

3 4 2 years ago

github.com

dr_kiszonka• 2 years ago

The paper is behind a paywall: https://www.nature.com/articles/s43588-024-00593-9

The repo links to the authors' preprint from 11/2021: https://arxiv.org/abs/2111.08275

HanClinto2• 2 years ago

The repo also links to a non-paywalled version of the paper:

https://rdcu.be/dy2Go

jabowery• 2 years ago

Code...Matters...

You can tell everyone "who is anyone" is in hysterics when this gets virtually no attention at Salamandar's old stomping ground here at ycombinator.

HanClinto2• 2 years ago

My amateur take on this:

This paper makes me think of the underlying premise of perceptrons -- the universal approximation theorem -- which states that a neural network of simple perceptrons can be combined to approximate most any mathematical equation with reasonable accuracy. At least, I hope that's a reasonable approximation of what it says. :)

Even though real-life biological neurons are much more complex (and some can perform complex operations like XOR operations in a single node), a regular perception cannot produce XOR by itself.

But combine a bunch of perceptrons together across a few layers, and you can not only perform XOR with a neural network, but pretty much any mathematical equation.

This paper makes me think that they're attempting to go the reverse direction with it -- after training a neural network to model a quadratic formula or something -- can we then go backwards and approximate the neural network's behavior with a discrete math equation?

This is a fascinating idea, and if it works, would send us a LONG way towards building interpretable AI!

Deep distilling: Automated discovery of algorithms from data

Related Stories