Researchers Discover a More Flexible Approach to Machine Learning

https://nautil.us/researchers-discover-a-more-flexible-approach-to-machine-learning-261703/

by dnetesn • 3 years ago

155 32 3 years ago

C. elegans are weird. They are a crazy minimalist system mostly without spiking neurons. From the perspective of building intelligence they aren't a great model. They are like reading code from a demoscene competition; terse, multi-purposed, highly compressed.

In contrast the mammalian neocortex looks more like reading a high level language with all the loops unrolled. Highly repetitive, relatively standard and massively verbose. I much prefer studying that for inspiration of what I want to turn into a modular code base.

While I love the authors focus on real-time perception at varying resolutions of timescale, I just don't see their approach catching on while the current progress in more standard neural nets remains so rapid.

zadler3• 3 years ago

Ok so this is talking about biological code as if it were programming… fun to read, Tell us more? Do the stages of evolution have different “coding styles”? Is everything a monad like in Haskell?

RosanaAnaDana3• 3 years ago

I mean, most people in this space don't even venture out of animalia.

There are many highly networked systems in biology where we know computation occurs, but we rarely consider it because we don't consider it intelligent. The casual example is fungal networks, but those don't hold a candle to the level of networking in a plant body thanks to plasmodesma.

the whole conversation around intelligence has been great, but it's mostly reminded me of how lacking and myopic our considerations of what intelligence actually is. The recent chapter of the saga has been great in terms of goal post movers, but I feel like we're no closer to a coherent definition of intelligence other than 'ill know it when I see it'. My prediction from this recent spat is that we'll have AGI before we know it, we won't know it as or recognize it as AGI, it won't fit convenient definitions, we'll have almost no clue how, and definitely no clue why it works, and we still mostly won't agree if it's intelligence or not. Effectively, it feels as if right now we're arguing about whether or not automobiles are horses, and laughing about how silly it is they can't do some of the things horses can do. Sure, but it all seems rather besides the point.

IIAOPSW• 3 years ago

honestly, the first one you described sounds like the far better model to try and turn into a code base. No boiler plate, no frivolous framework jigging, distilled to its raw functional essence. A simple system of emergent complexity strikes me as a much better candidate for intelligence than a kludge that gains complexity from just having a lot of parts and rules to begin with.

xg15• 3 years ago

Small system aren't always simple or easy to understand. Famous code examples are Duff's Device [1] or the "wtf" fast inverse square root function [2]. Both functions are just a few lines long but usually leave people scratching their heads until they learn about the trick.

Especially with Demoscene code, it's common to exploit all kinds of specific hardware effects or re-parse the assembly code to a different instruction sequence [3]. This kind of stuff may be a lot more complex than a larger system, it just hides its complexity in a compressed representation.

[1] https://en.wikipedia.org/wiki/Duff%27s_device

[2] https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...

[3] https://reverseengineering.stackexchange.com/questions/20587...

joe_the_user• 3 years ago

Neuroscientists simply don't understand the mammal nervous system with any degree of completeness and accuracy. Looking at a mammal isn't like looking at code of any kind but very partial map of code. Maybe you can get inspirations from this but no one currently has anything very convincing as model of how learning happens at this level.

Retric• 3 years ago

Neuroscientists have a solid enough model of learning to realize what people call learning is really several different things operating at different scales. Aka Learning the state capitals is very different than learning dance moves. Thus the focus tends to be on specific things like memory formation.

How well we understand each of these systems is up for debate. It’s far from nothing but not comprehensive either.

super256• 3 years ago

Related discussion with 66 comments: https://news.ycombinator.com/item?id=34707055

flashfaffe22• 3 years ago

TEDx video from Ramin hasani explaining his algorithm.

https://www.youtube.com/watch?v=RI35E5ewBuI

flashfaffe22• 3 years ago

Same author but More technical video: https://youtu.be/IlliqYiRhMU

shepardrtc• 3 years ago

Code for anyone interested: https://github.com/raminmh/CfC

Researchers Discover a More Flexible Approach to Machine Learning

Related Stories