AI pioneer Geoff Hinton: “Deep learning is going to be able to do everything”

https://www.technologyreview.com/2020/11/03/1011616/ai-godfather-geoffrey-hinton-deep-learning-will-do-everything/

by jonathanbgn • 6 years ago

72 47 6 years ago

technologyreview.com

peteradio2• 6 years ago

I wonder what humans will learn about themselves as we create synthetic competitors.

moconnor• 6 years ago

That enabling returns on capital entirely without labour functionally breaks capitalist democracies, probably.

jacobush• 6 years ago

A new twist on the Terminator series. Skynet starts a war because the humans tries something which would influence the stock market.

Come to think of it, not much different to current reality... we already do that to ourselves.

robertlagrant• 6 years ago

Probably not - absolute standards of living will get even higher, but there will always be things only some people have, and more people want.

(This applies with or without synthetic life doing stuff for us.)

Agebor• 6 years ago

That perhaps it was already done before...

JHonaker2• 6 years ago

I’m tired of Hinton over promising. It’s not going to do everything and he’s promoting an irresponsible position.

Has there really been a huge breakthrough since the initial wave of CNNs? I guess transformers/attention, but I don’t consider GPT-3 to solve any problem at all.

brutus12132• 6 years ago

I'm generally skeptical on research and work in the field. However, the progress I've seen on GANs/deep fakes, natural language understanding, multi-modal, video understanding/activity recognition .. it is unbelievable how fast the field is moving. This may not result in new widgets next CES but may be game changing to society in a decade.

In academic research, GPT-3 and in general language models are applicable to a host of downstream tasks across different topics.

Edit .. reworded since it seemed insulting (unintentional).

Filligree3• 6 years ago

Having spent the last two months writing a novel, with GPT-3's help, I take issue with the claim that it's useless.

It's anything but useless. Sure, I do 80-85% of the work, but that remaining 15% is the portion that would otherwise have me stuck for days. In practice, it's a 10-50x improvement in writing speed.

pdimitar3• 6 years ago

This may not result in new widgets next CES but may be game changing to society in a decade.

And I am tired of hearing that ever since the 90s.

Clearly, deep fakes changed society. Made people more paranoid and less believing of what they see. Is that a positive change though?

Meanwhile, you still have to manually load dishes in the dishwasher, and robot vacuum cleaners stumble and block on the easiest of obstacles.

Call me a cynic but AI work needs to get out of its comfy well-funded bro-club corner and start to seriously try and solve real-actual-physical-world problems. It's all well and good that scientist X can model a text paragraph with this or that NN but when will he solve a car driving itself?

And don't get me wrong. I know science can take a long time to make a breakthrough. It's not adhering to the same laws as everyday work -- I am aware and I am clamouring for science. But I don't feel the AI area is even heading in any direction at all. Could be wrong though.

JHonaker2• 6 years ago

I mean, don’t just take my word for it. Here’s Yann LeCun [1].

I will concede that I forgot about DeepFakes. How’d I do that? But still, cool now we can live in a dystopian future.

The point that I was trying to make still stands. Assuming that more data and more compute will solve all of our problems is a ridiculous position. The field might be moving incredibly fast, but most of the work I see is not great. Everyone is hyped. People claiming that we can replace all of statistics and machine learning with pure deep learning models forget about how hard it is to actually get these things working in the first place. Do you have 100 TPUs to run full blast for a week? I mean consider the series of Nature papers a few years ago. Google says, “I can predict earthquakes using NNs!” Somebody comes along and says, “I thought about the problem for a little bit and beat you with logistic regression.”

Self-driving cars have been just around the corner for the last 10 years. Just like with the rest of stats and ML, it’s not the place you have a lot of data that’ll kill you. It’s the edge cases.

[1] https://m.facebook.com/yann.lecun/posts/10157253205637143?no...

mcilai• 6 years ago

Overpromising? Underpromising, if anything. I cannot understand the point of view that denies the _overwhelming_ progress that has taken place in AI. And to deny GPT-3’s mega advance — we now have a system that can achieve state of the art or near state of the art requiring a negligible number of training cases (i.e., few shot learning) — seems wrong to me.

schonbild• 6 years ago

Sure, if "everything" includes making humanly mistakes as well!

BobbyJo• 6 years ago

Hire a human that can only work 8 hours a day, or run and AI that can work 24 (and probably costs less). If their miss rate per hour is the same, definitely go with the AI.

peteradio2• 6 years ago

Will the machine recognize them as mistakes? If so then its bootstrap time.

giardini2• 6 years ago

Hinton wants a Nobel prize awarded while his name is still in the ring!8-)) But that will likely happen a generation or two after Hinton is gone.

gwern2• 6 years ago

A Nobel for what? They don't do psychology, math, AI, statistics etc. And he already won the Turing Award.

giardini2• 6 years ago

“Hope springs eternal in the human breast,

Man never is, but always to be blest."

-Alexander Pope, Essay on Man

cs7023• 6 years ago

Hinton qualified that statement by adding, "but I do think there's going to have to be quite a few conceptual breakthroughs." He explicitly conditioned his belief on future conceptual breakthroughs that have not been made yet. Also, his statement is not a prediction; it starts with "I do believe."

Here's the full quote, copied verbatim from the article:

I do believe deep learning is going to be able to do everything, but I do think there’s going to have to be quite a few conceptual breakthroughs. For example, in 2017 Ashish Vaswani et al. introduced transformers, which derive really good vectors representing word meanings. It was a conceptual breakthrough. It’s now used in almost all the very best natural-language processing. We’re going to need a bunch more breakthroughs like that.

Please don't criticize him or the article without first reading it in full.

bo10242• 6 years ago

Yeah, it's a bit tricky ... like saying "Turing Machines are going to be able to do everything, but we'll need quite a few conceptual breakthroughs." Hard to say what percent of progress we've made since 1936 toward artificial general intelligence (whatever that means).

cs7023• 6 years ago

In principle, yes, of course, because we use Turing machines for deep learning :-) but I think Hinton's point is much less broad than you imply: He believes "AGI" ultimately will be achieved with deep learning approaches, as opposed to other approaches, e.g., symbolic AI.

6gvONxR4sf7o• 6 years ago

What do you believe to be your most contrarian view on the future of AI?
Well, my problem is I have these contrarian views and then five years later, they’re mainstream.

Has every single one of his contrarian views panned out? What an arrogant quote. This just diminished him in my eyes.

etiam• 6 years ago

Relax. It's an expected wisecrack at a question that is frankly not of very high quality.

phenkdo2• 6 years ago

I am very skeptical of the current approaches (supervised learning) making the quantum leaps being promised. This needs a paradigm shift of weakly-supervised learning and fine-tuning for specific tasks (a la human learning).

ebalit• 6 years ago

It is indeed where a lot of the current research is heading to if you count self-supervision as part of weak-supervision. Self-supervised learning brought massive improvement in NLP and is bringing state of the art result in vision after some time where purely supervised learning showed better result.

bonoboTP4• 6 years ago

Weakly-supervised and self-supervised methods are extremely mainstream today. There is a flood of papers on these topics. Conferences are full of them. This comment was perhaps insightful 5-6 years ago, today it's absolute orthodoxy. The thing is though, supervised just works way better. It's not that people don't want to do weakly-supervised, but that it's very hard!

phenkdo2• 6 years ago

My point exactly. Weakly/self-supervised learning is "very hard" because it's still an active area of research and breakthroughs are needed. I feel those a pre-requisite before we can credibly approach the promise of "doing everything"

mellosouls• 6 years ago

Now it’s hard to find anyone who disagrees, he says.

Either that's an accurate claim or representative of a cliqueyness and echo chamber approach to AGI in some quarters.

Either way it's depressing.

mcguire• 6 years ago

Does this mean we are at the top of another AI hype cycle? Anyone have a pool on when the next AI winter starts?

WanderPanda• 6 years ago

Winter will be gone forever, AI climate change it is :p

gwern2• 6 years ago

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... According to HN, AI winter has already started in 2017, 2018, 2019, 2020...

mhh__• 6 years ago

I basically buy that DL will be effective, but I think the real innovation will have to be power efficiency e.g. how many GPUs and GBs of ram does GPT-x need?

lootsauce• 6 years ago

If a cyborg were to "do everything" on deep learning would it have a meaningful model of reality or would it simply be behaving as if it did?

ilaksh• 6 years ago

In particular, some recent work at Google has shown that you can do fine motor control and combine that with language, so that you can open a drawer and take out a block, and the system can tell you in natural language what it’s doing.

Does anyone know which paper that was?

AI pioneer Geoff Hinton: “Deep learning is going to be able to do everything”

Related Stories