Why Is the Human Brain So Efficient? Massive Parallelism

http://nautil.us/issue/59/connections/why-is-the-human-brain-so-efficient

by nbmh • 8 years ago

129 52 8 years ago

posterboy2• 8 years ago

Like a quantum computer, computing thousand calculations at once and the true answer is what remains? I guess there's a ... parallel there. And I know some people don't like this diductic reduction, analogy, metaphor, callitwhatyouwant. I'd welcome corrections.

dsr_• 8 years ago

The problem with trying to correct this is that it's not wrong, because it doesn't really state a hypothesis.

Are you trying to say that brains are quantum computing devices? Maybe, but we don't have any particular evidence for that, so it's on a par with saying "The Flying Spaghetti Monster has a tiny noodly appendage that reaches through hyperspace into each neuron, and that's how we think."

We can observe brains in a lot of different ways - active electromagnetics, chemical sampling, microscopy, MRI... and we can say that injuring certain areas of the brain will impede certain functions. We know that excesses or droughts of some chemicals are associated with emotions, depression, and some diseases.

But we don't have a good model for how brains think, and among the good models we don't have, quantum computing is one of them.

posterboy2• 8 years ago

I was being sarcastic, but then wasn't sure whether the comparison was actually wrong.

partycoder2• 8 years ago

The side-to-side comparison is not very accurate.

The brain consists of many structures, many of them involved in non-cognitive tasks.

blattimwind2• 8 years ago

The "speed of basic operation" column is given as "10 GHz" for a computer and 1 kHz for a human brain, but later is clarified that the 1 kHz refers to the maximum observed frequency of a neuron's output. Arguably these are unrelated metrics. I don't think a thing like "speed of basic operation" can even be defined for a human brain, because fundamentally it's a programmable computer metric that simply doesn't transfer to a brain.

Of course, one could go ahead and simply compare the conscious compute power of a human to that of a computer (e.g. FLOPS), but then that number would be something like "1/100 to 5 FLOPS".

gerbilly• 8 years ago

maximum observed frequency of a neuron's output.

IIRC the frequency of a neuron's output is used to encode the magnitude of the signal.

Take for example a neuron transmitting a pain signal, low frequency is used to transmit low levels of pain.

High frequency, extreme pain.

partycoder2• 8 years ago

Another problem with the analogy in this article is that there is no division between memory and processing element in a nervous system. They're essentially the same.

Imagine running a program on a computer where everything is somehow memoized, on a system that is always powered on.

nojvek• 8 years ago

The brain definitely has an edge on compute power. The computer has an edge on storage power. I would bet the brain doesn’t store more than a Terabyte of information.

We just need more cores without spiking energy usage.

drchiu• 8 years ago

Interesting observation. I think most of us rely on some sort of world view or paradigm to operate. Probably as a result, the actual info stored is even less than a TB.

reubenswartz• 8 years ago

The human brain has over 100 trillion synapses, which is somewhat analogous to storage or memory (in both senses). Some estimates for the storage capacity of the brain are as low as a terabyte. Others are an order of magnitude or two higher.

Being electrochemical, the brain is “slow” compared to a computer (around 10hz). But it’s massively parallel and an interesting combination of digital and analogue.

laythea• 8 years ago

Erm, and we just need the software to run it all :)

213• 8 years ago

The computer also has an edge on precision and determinism.

oldandtired• 8 years ago

Considering the kinds of experiments that have occurred in the past on electrical stimulation of the brain and memory recall, I would say that the amount of memory being used in the brain exceeds any limit we may well put on it by many, many orders of magnitude.

The major problem is that we are not trained today to recall most of what gets laid down. We are far to dependent of external storage mechanisms, books, videos, etc, that we no longer train ourselves to actually recall and pass on our knowledge.

I am not saying that external memory is bad, just that we should be using all options available to us.

Our understanding of the brain and its mechanisms is very much in the very early stages - we have just scratched the surface.

iandanforth• 8 years ago

"a professional tennis player can follow the trajectory of a tennis ball after it is served at a speed as high as 160 miles per hour"

This is false or at least highly misleading. A reader might imagine that the player's eyes track the ball in flight. This is not what happens. A pro player reads the position of the serve and begins moving before the ball is hit. The predictive power of the brain is much more important than the speed and precision the author was trying to highlight here.

dig1• 8 years ago

Agree. As someone who was playing table tennis for a bit time, where basically you can't see the ball due speed, prediction training is essential and long term process. At some point you get so versed you know where the ball will be just by looking at opponent elbow or arm.

213• 8 years ago

The way other articles describe it, a pro player is not really aware of the incoming ball, lower level non-conscious (reflex) systems track the environment and react before you are aware of it, just like when a regular person dodges a sudden obstacle coming at them and only a bit later they realize what's going on.

Retric2• 8 years ago

That's a meaningless distinction when talking about the brain. The brain is very much doing calculations that we are not aware of, but they still occur.

BLKNSLVR3• 8 years ago

Sounds like one of the concepts that inspired the book Blindsight by Peter Watts. The subconscious 'reptile brain' is able to process certain old-school dangers, like an object moving quickly towards us, faster than our conscious brain, and can cause involuntary, potentially life-saving, reactions to the stimulus.

The conscious mind processing these things can often get in the way but, in the case of sports, alter the instinctive behaviour into something advantageous. An augmentation of the conscious and subconscious into better performance.

A converse example, being conflict between conscious and sub-conscious, is if you drop a mug of boiling water, the instinctive reaction is to try and catch 'thing you dropped', but the conscious reaction is 'don't burn yourself' so don't catch it and jump backwards to not get splashed.

Very interesting stuff.

https://www.goodreads.com/book/show/48484.Blindsight

https://en.wikipedia.org/wiki/Blindsight_(Watts_novel)#Consc...

Beware (or since it's the HN crowd, Recommended!): hard sci-fi.

rproctor• 8 years ago

There is a related anecdote in "The Sports Gene" that suggests that MLB hitters do not track the ball in flight, but instead react to the pitchers' arm motion. As a result they have trouble hitting a motion they haven't visually trained against:

http://www.eugenewei.com/blog/2013/8/23/the-pitchers-who-con...

jjtheblunt• 8 years ago

https://www.imdb.com/title/tt5434972/ is an _excellent_ documentary about just this topic, science and all...and it's funny.

amelius• 8 years ago

How many milliseconds does it take for one layer of neurons to adjust its output? Let's call this number X.

Can we conclude that if a task takes a human Y milliseconds, then a neural network with Y/X layers is sufficient for that task?

blattimwind2• 8 years ago

AIUI/AIBT there aren't really discrete layers in brains and even the same type of neuron can act in very different ways, so I wouldn't be surprised if this question doesn't translate to brains.

213• 8 years ago

A biological neuron is many magnitudes more complex than a typical neural network node.

pranjal1029• 8 years ago

Human brain mechanism is a mystery

coss• 8 years ago

I feel like there must be some way to 'clean' our brains. As we get older it must accumulate 'gunk' that slow it down or weaken connections.

I guess sleep is the closest thing we have but to me that's like saying drink water to fix tooth decay.

Bluestrike2• 8 years ago

We already do (in a multitude of ways) when we sleep. There's research that indicates that sleep is used to clear away neurotoxic waste.[0][1] The role of deposits of tau and beta-amyloid proteins in Alzheimer's[2] and aging are also of particular interest. Other papers supporting the synaptic homeostasis hypothesis suggest that sleep is when synaptic strength is renormalized.[3][4] Basically, memories are reinforced (the signal) in relation to existing knowledge while daily noise is effectively pruned (among other processes).

The analogy is flawed, but the brain already has some pretty intricate garbage control methods. As we learn more about them, it'll open up new paths for clinical treatments for a host of problems.

0. https://www.ncbi.nlm.nih.gov/pubmed/24136970

1. https://www.nih.gov/news-events/nih-research-matters/how-sle...

2. http://www.sciencemag.org/news/2018/01/alzheimer-s-protein-m...

3. http://science.sciencemag.org/content/355/6324/507

4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3921176/

rmwaite• 8 years ago

Drinking water (instead of sugary and/or acidic drinks) is actually a very good way to prevent tooth decay.

Retra• 8 years ago

Even if you drink sugary/acidic drinks, water helps dilute them and rinse them away.

BLKNSLVR3• 8 years ago

Taking your question conceptually, rather than physically, memories / experience would be 'gunk' if they're outdated (don't relate to how the world works anymore, or don't relate to how your body works anymore).

Your memories and experiences (potentially one and the same) colour your experience of the world and make you react in different ways, not always beneficial - but the phenomenon wouldn't exist if it wasn't historically, evolutionarily beneficial.

timvisee• 8 years ago

I wonder how this would compare against quantum computing, as that allows maximum parallelism too if in correct.

rfw• 8 years ago

Quantum computing is not parallel computing: https://physics.stackexchange.com/a/3400 briefly talks about the mechanism of quantum computing and how it is not general parallelism.

fallingfrog3• 8 years ago

The article seems to be about how it is that the brain can process so much information at so slow a cycle speed, but it doesn't really address power efficiency at all. Even assuming 100% parallel operation, our current chip designs use something like a factor of a million more energy to do the same amount of computation. I wish I knew why- it's not like we're ignoring power efficiency. Depolarization of an axon must be incredibly power efficient.

fallingfrog3• 8 years ago

A litte online research reveals that the depolarization potential is on the order of one or two millivolts, whereas silicon has a bandgap of 700 millivolts. That's plausible; since power is proportional to voltage squared, a 500 fold voltage improvement would equate to a 250000 fold power improvement.

dlwdlw2• 8 years ago

I think it's about context. If I say "order pizza" a ton of information is transferred via shared environmental context. I'm not transferring information directly but manipulating contextual levers to map certain types of information.

An example is if I say something to you in say japanese, I also need to teach you japanese in the same phrase unless there is shared context. computers run on a very impotent form of language, logic, that is rigorous and general purpose but has no "escape hatch". (basically the sentiment of Godem, Escher, Bach with regards to rigorous systems. )

fallingfrog3• 8 years ago

Oh ok you're talking about data compression. That's also not what efficiency means, in a strict sense.

nootropicat2• 8 years ago

I recently realized the (obvious in hindsight) fact that general intelligence better that brute force doesn't exist, as intelligence is equivalent to compression.

Given the recent discoveries about neurons using mRNA capsids to communicate [0] it's not that farfetched to posit that we are really dna computers [1]. The processing time (for new problems) seems human-like: "The slow processing speed of a DNA-computer (the response time is measured in minutes, hours or days, rather than milliseconds)"

The evolutionary argument: as DNA computing is already used by microbes [2] how could the nervous system made of (relatively) dumb neurons compete with that? Synapses still make sense - as a way to request a rna packet and/or inform that it's coming and from where.

One neuron with capability of ~10M pattern matches per second (encoded in dna/rna) would mean that the human brain executes ~2^60 pattern-matching operations per second, utilizing zettabytes of imperfectly copied data. Enough to brute force its way through lots of problems.

Memory as dna would explain high-level memory quirks: each read would be destructive, by splitting dna into rna, interacting with other rna under the presence of appropriate enzymes, then copying and disseminating the resulting rna, transforming the memory each time it's retrieved.

It would also explain urban legends about people's personalities changing to resemble their organ donors in some way - as a donor's memory packets that somehow ended up on the donor's organ and, with the help of immunosuppressants, managed to infect the receiver's brain.

[0] https://www.nature.com/articles/d41586-018-00492-w

[1] https://en.wikipedia.org/wiki/DNA_computing

[2] https://en.wikipedia.org/wiki/Microbial_intelligence

return1• 8 years ago

mRNA capsids to communicate

In this case "communicate" is in a narrow sense - this communication is very slow and Arc has more to do with plasticity rather than the electrical response of neurons.

justifier2• 8 years ago

general intelligence better that brute force doesn't exist

What do you mean by this?

It sounds like you are assuming nature requires exactness and are really making a philosophical argument about the nature of computation, namely an np solution in p 'doesnt exist'

Memory as dna would explain high-level memory quirks

If we are conjecturing than so too could approximate results 'explain high-level memory quirks'

we already have many approximate algorithms that are significantly better than brute force, and I would argue that any read procedure would necessarily be algorithmic, which then would require an explanation as to why this natural process failed to evolve over time

If such an explanation is simply, though arguably counterfactual, 'nature requires exactness' and so is unable to utilize the incremental improvement of evolving algorithms for approximate results, I would argue this implies p!=np because otherwise I think, if it were able, nature would tend toward improving on exactness over the 13B+ years it's been expressing mathematical truths

Being as my intended inference in regard this specific unsolved problem is to develop an algorithm to show p=np, I wonder if the process we refer to as conciousness may be such an algorithm

neurons using mRNA capsids to communicate

I wonder if the rna is raw memory data or architectural plans for nuerons which, when constructed, express memory

nootropicat2• 8 years ago

What do you mean by this?

The goal is to reduce n bits of data to x<n bits. Because there are less variables you gain predictive capability of n-x bits. Or stated differently, the goal is to get closer to the kolmogorov complexity of whatever you're trying to model.

Yet it's not possible to compress n bits in the general case. That's because the kolmogorov complexity is a function of your assumed knowledge (assumptions). All you can do is start checking every possible transformation from your assumption starting with the most probable one - the probabilities are based on your knowledge itself.

we already have many approximate algorithms that are significantly better than brute force

Yes - but that means the algorithm itself, along with its execution, is the shortest (in the used metric, which can include execution time) answer for a particular problem. How do you generate the algorithm in the first place?

drdeca• 8 years ago

Not sure how you are defining "general intelligence".

Why wouldn't AIXI count under your definition? (I recognize that AIXI can't be truly implemented, but I don't see where your explanation fits in a "unless you have a halting oracle")

whatever1• 8 years ago

Question: Has there been a comprehensive study on the activation function of a single (biological) neuron?

gimagon• 8 years ago

Yes, see the Hodgkin Huxley model [1].

[1] https://en.wikipedia.org/wiki/Hodgkin%E2%80%93Huxley_model

nitwit0052• 8 years ago

We don't know what operations the brain is actually doing, so declaring that it is performing them efficiently is pure speculation.

Similarly, the brain surely has limits to parallelism, for the same sorts of reasons a computer does. You only have one mouth, so if two parts of the brain tried to speak, fully parallelized, you'd get nonsense at best. They have to agree on what to say, which is effectively serialization.

antris• 8 years ago

The mouth as an example with regarding to speech doesn't work very well. People can say one thing, but say it in a way that means the complete opposite. So already we have tone incorporated into our speech. Who knows what kind of subconscious thoughts we embed into our speech that are not yet understood well.

Then there's the combination of tone and the words, accent, body language and every other context embed into our language that produces a very complex meaning, even for one word.

nitwit0052• 8 years ago

It doesn't matter what was factored into the speech, you still can't speak in parallel. It's not physically possible. At some point you need to get agreement as to what the output should be.

Arthanari• 8 years ago

A brain is not equal to a computer or a processor.

You will never get the answer if you look at it from a narrow minded perspective.

A brain is equal to billions of processors connected in a highly naturally efficient network functioning almost effortlessly.

Something along these lines seems like a plausible comparison to a brain.

What is a brain is not equal to a processor but a neuron is equal to a processor?

tabtab• 8 years ago

My brain isn't efficient. I forget basic things sometimes.

Why Is the Human Brain So Efficient? Massive Parallelism

Related Stories