mcilai

Ironically, it is unbelievably damaging for individuals to believe that one needs a lot of luck, as this belief feels good, destroys resilience, and prevents one from recognizing and going after the (initially) modest opportunities that do show up.

Sure, you won’t be like Elon Musk, but you could be better than 90% of the population, since most people don’t want to try hard. Much more “fun” to believe that life is unfair, and therefore no action is required on one’s part.

Don't forget the very broken incentives.

Overpromising? Underpromising, if anything. I cannot understand the point of view that denies the _overwhelming_ progress that has taken place in AI. And to deny GPT-3’s mega advance — we now have a system that can achieve state of the art or near state of the art requiring a negligible number of training cases (i.e., few shot learning) — seems wrong to me.

Deep learning will continue to surprise

because it most likely is

"Ark estimates that Deep Learning has created $1 trillion in market value so far. " -- we might be headed towards a warm winter

(from https://www.nextbigfuture.com/2020/01/ark-invests-big-five-t...)

It's an unbelievably valuable resource to humanity.

If Coursera succeeds it'll be worth far more than 2.5B.

There's one minor difference between past models of the brain and deep learning: deep learning can actually perform difficult and useful cognitive tasks that cannot be accomplished by any other means.

Deep learning is as close to being a "newtonian theory" of the brain as it gets -- deep learning abstracts away a lot of the complexity of neural systems (e.g., a simple artificial neuron vs a highly complex biological one) while maintaining a number of essential characteristics: massively parallel computation, error tolerance, graceful degradation, distributed representations, information is stored in slowly-changing synapses, and, most importantly: a simple, local, and powerful biologically-plausible-if-you-squint-hard-enough learning rule.

The important question to ask is: is the deep learning abstraction any good?

There's a very strong case to be made that the answer is yes: deep learning systems can perform many (of course, not all, at least not yet) tasks that involve perception (computer vision/speech recognition), motor control (the recent openai robot), language understanding (machine translation/BERT/GPT), planning (alphago/dota/the deepmind protein folding), and even some symbolic reasoning (the recent work from facebook on symbolic integration https://ai.facebook.com/blog/using-neural-networks-to-solve-...). Some of these tasks are performed at such a high level that they become commercially useful, and in some cases, surpass "human level".

So here we have a "model family" -- deep learning -- with a set of principles so simple that it can be studied with intense mathematical rigor (for example, https://arxiv.org/pdf/1904.11955.pdf or https://papers.nips.cc/paper/9030-which-algorithmic-choices-...), and that produces many of the behaviors we want out of brains (and not just behavioral: see, e.g., https://arxiv.org/abs/1805.10734: " Interestingly, recent work has shown that deep convolutional neural networks (CNNs) trained on large-scale image recognition tasks can serve as strikingly good models for predicting the responses of neurons in visual cortex to visual stimuli, suggesting that analogies between artificial and biological neural networks may be more than superficial." -- this is just one of many papers that show that even under the hood, trained deep learning systems exhibit many properties of biological neural networks).

These reasons strongly suggest (imho) that deep learning is in fact the newtonian theory of neuroscience. More strongly, no other theory comes remotely close in its simplicity and explanatory power.

Deep learning?

Number 6 is not a myth!!!

My view is different: the number of people who are willing to truly do evening this post recommends is just small. But those who are willing to pay the cost of success have a real chance of getting it. It's just that most people likely conclude that success is not truly worth it.

No!!!!!!!!!!!!!

Are humans permanently inscrutable? In the long run, AI may be similar. Given how massively complex the world and the thing AIs learn are, it is not self evident that interpretable systems are possible. However, it's really worth trying.

More likely, we'd have to use AI systems to understand how other AI systems work. Or maybe, once AI systems get very good at language, develop a "truth serum" that will cause it to reveal its true opinions.

That's a good point.

Quite incredible that he was interested in NNs back in 1990. He closed this thread very well.

(ml expert here) This doesn't seem true. Neural networks are great at detection, so they should be similarly good at determining whether two objects are identical or not. The fact that their networks didn't do better than chance (as opposed to a "disappointing 10%" error rate") suggests that they trained their networks wrong.