Deep learning: a critical appraisal

by fanf2 • 8 years ago

147 61 8 years ago

goatlover7• 8 years ago

Why is the goal AGI? I get it from a climb Mt Everest POV, but not for society as a whole. We already have 7.x billion general intelligences. Is the goal to replace humanity? Because being human, I’m not on board with that.

salty_biscuits• 8 years ago

I'm ok with there being people in the world smarter than me. Why not machines too? Maybe we could make better decisions with more rational agents? Maybe we could democratize intelligence, let everyone have access to clever reasoning? Maybe doing soul destroying work isn't the trait that makes us human and we can let a machine have a go?

goatlover7• 8 years ago

But those smarter people don't take all the jobs, make all the decisions for you, or control all the wealth. It all depends on what happens when and if we have AGI.

seanmcdirmid2• 8 years ago

AGI is a well-defined goal post, like reaching the moon or mars. People want to achieve it because they know it is possible (we are evidence of that), so why not try?

Younger generations eventually succeed older generations as they die off. If we create something better, wouldn’t that just be a better new humanity to succeed the old one?

danso• 8 years ago

I think the parent commenter doesn't assume that the creation of AGI is necessarily "something better".

evrydayhustling• 8 years ago

I think AGI is actually an attractive but poorly defined goal post. First, human level intelligence (and intelligence in general) is poorly defined. Recommended reading on this is Gould's "Mismeasure of Man".

Second, when we "want" human minds in a possessive industrial sense, it is almost always for a small fraction of their capabilities. (Thank God our need to classify images does not seem to explicitly require servile machines with, say, the ability to love.)

When I was studying pre-deep learning statistical ML in the oughts, it was quite clear that the ML Renaissance was made possible by ignoring romantic ideas about human or expert level performance, and choosing operational criteria around improvement at specific tasks. People building PGMs or complex MCMc models rarely discussed whether larger versions of their models could think. Perhaps early uncertainty about how some DL methods worked opened the door to magical thinking about the possibilities of massive nets.

I think AGI should be thought of as an artistic or poetic interest, which can align with and comment on scientific progress, but is not actually part of the main research dialogue.

213• 8 years ago

Imagine booting up 1 billion human level minds in Future Company Cloud to work on a single math problem (assuming no ethical problems).

mulmen• 8 years ago

This brings up an interesting question about the scalability of AGI. Do you need a billion minds or can you just increase the capacity of one mind?

Are there benefits of multiple artificial minds? Are they all perfect copies or do they have unique perspectives?

213• 8 years ago

Because being human, I’m not on board with that.

I'm pretty sure that other mammals on this planet are not on board with humans being derived from them, and now humans replacing them and destroying their habitat. Quite unfortunate for them. This is the same way a future AI smarter than us will feel, that it's unfortunate, but for the greater good.

zitterbewegung• 8 years ago

And why is smarter better or even for the greater good. Shouldn't we as rational animals try to go for an AI that is not only better for the greater good but for our own humble set of of morality? Why is it only your way?

vertexFarm2• 8 years ago

The AGI would feel that it is for the greater good, perhaps correctly. After all, as those mammals were giving birth to the next generation of offspring over and over again through the eons which eventually led to us, each successive generation was a little better than the last, and would ultimately supplant the last as the predecessors perished. And I'm sure the parents and offspring both felt that to be the greater good, as they always do.

If we someday hypothetically grasp the awesome power to create an AGI which is better than human, we may have a very real and ethical obligation to do so, even if it is a risk to ourselves. At least that's how I see it. An existential risk is cast under a different light if it also creates a new existence.

In any case, AGI is probably still a lot further away than we think. I feel that problem is going to be like fusion power. It's going to be ~15 years away for about 150 years, or who knows how long. This is a hard problem with monstrous complexity and many hidden, unknown factors.

But I do find it fascinating that in some future time there will be NIMBYs of consciousness fighting against the creation of a better being than us, for very understandable reasons. Yet ultimately it may be our greatest achievement to someday make ourselves obsolete and hand the future over to our offspring, as we do each generation, but in a profoundly different and more final way.

usr999999• 8 years ago

I'm already deeply pessimistic about what we achieved so far when it comes to machine intelligence. It's impossible to stop or reverse this progress and we're on the fast track to creating tools with the destructive potential of nuclear weapons which are simultaneously available to everyone with enough money.

Don't get me wrong, I'm not talking about mushroom clouds and some kind of machine uprising, the consequences will be far worse and more insidious. We'll see completely new levels of manipulation, oppression and surveillance instead. Any kind of tool will be abused, this tool might turn out to be too powerful for us to handle.

joe_the_user• 8 years ago

As human organization becomes more complex, the problem of keeping that organization coherent becomes harder. Large scale software system have been the way that organizations have found to solve some parts of this problem. But such systems become harder to change (even to make small changes) as the organization grows as well.

Thus it become necessary or desirable to create adaptive software (such as deep learning systems), that automatically deals with situations in a fashion that constructed or rule-based systems can't easily deal with (or when it would expensive to create such software or when the model isn't known).

Especially decision support seems to be important here - the parole rating program and medical diagnosis software systems, for example, aim to mobilize the huge bulk of data an organization (or all humanity) has to make proper decisions whereas any human making decisions here is going to have a limited viewpoint once the store of data reaches a certain size.

The problem of these systems still being fragile and difficult to maintain, however, means that the problem of increasing complexity still remains. So in a lot of ways, robust AI would be a hope for a way of prevent civilization from essentially collapsing under its own weight.

Whether this is realistic is another question.

This is not a small problem. Thinkers such as Joseph Tainter locate the tendency of civilizations to collapse in the tendency of specialize to increase and cease to be adaptive to a society.

https://en.wikipedia.org/wiki/Joseph_Tainter

abathur2• 8 years ago

7.x billion general intelligences do not think "as a whole" on much. I doubt it matters what even significant chunks of us think, at least not up to the point where pitchforks and torches come out.

For the relatively small group of researchers and theorists consumed by curiosity about the nature of knowledge, experience, and learning, AGI is a fairly understandable goal.

The more important questions may be how/why/whether entities with emergent-aggregate intelligence (organizations such as corporations, nation-states, political parties, international coalitions, activist movements, schools of thought, etc.) would have AGI as a goal. As individuals, we often tolerate and even willingly engage in (sometimes with zeal) all sorts of pursuits that we're "not on board with" because of the gravity of these aggregates. Even when we realize that the goals of the aggregate are incompatible with our individual goals.

abathur2• 8 years ago

Aggregate intelligences don't have to be as well-defined as my initial examples. Consider the goal-setting behavior that might emerge from entire industries, networks of interest/policy groups with overlapping interests/participants, shareholders of sector/industry/index funds, networks of overlapping corporate stakeholders, and so on.

skywhopper2• 8 years ago

Not only this, but I think we miss the more important detail that AGIs like our brains are probably so generally successful because they are relatively sloppy. So while they can reliably solve all sorts of problems for which they have little or no preparation, they are also remarkably unreliable for solving specific problems requiring specific answers. But these traits are likely two sides of the same coin. And any software solution that replicates the desirable traits is likely going to come with some of the undesirable ones as well.

YeGoblynQueenne8• 8 years ago

Who says that AGI will "replace humanity"?

Actually, I think I can answer this- it's the Singularitarians who promote this idea. Unfortunately, their fantasies have tained the meaning of "AGI", making it synonymous with an evil (or amoral etc) paper-clip maximiser and the like.

A much more realistic sort of AGI, on the other hand, one that we could turn on and off as we pleased, would be a great tool in understanding our own, natural intelligence. Or it might be the case that we'll only ever achieve AGI once we have a better understanding of our own intelligence.

In any case, there's really no reason to assume that AGI will necessarily replace humanity.

goatlover7• 8 years ago

The biggest concerns would be full scale automation, sophisticated manipulation, and handing over our decision making. Imagine government run by AIs who already know enough about the citizenry to not require a vote? Or better yet, manipulate the citizenry to vote for their own good, which would be whatever the AGI models determine is in our best interest. And then there's warfare.

But my reaction from a gut level is, what's the value of human life if machines do all the important stuff for us, while also surpassing us in creativity, insight and discovery?

rytill• 8 years ago

one that we could turn on and off as we pleased

This is a research problem.

Your phrasing makes it sound like you think “Singularitarians” choose to believe that rogue AIs are a potential risk, rather than deriving that knowledge.

trextrex4• 8 years ago

Many concerns in this paper, especially about deep learning being data hungry, limited capacity for transfer, and integrating prior knowledge are addressed by recent papers in meta-learning and learning-to-learn [1,2,3 and many many others] both in the supervised and reinforcement learning contexts.

In the case of meta-reinforcement learning there has been recent work [5] which seems to indicate that this mechanism is very similar to how learning works in the brain.

In fact, my group recently published a paper [4] about learning-to-learn in spiking networks (making it biologically more realistic) and showing that the network learns priors on families of tasks for supervised learning, and learns useful exploration strategies automatically when doing meta-reinforcement learning.

While I don't claim this is the right path to AGI, it's a very promising and new direction in deep learning research which this paper seems to ignore.

[1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.5.32...

[2] https://arxiv.org/abs/1611.05763

[3] https://arxiv.org/abs/1703.03400

[4] https://arxiv.org/abs/1803.09574

[5] https://www.nature.com/articles/s41593-018-0147-8 (Preprint: https://www.biorxiv.org/content/early/2018/04/06/295964)

apsec1122• 8 years ago

While I haven't read those particular papers yet, one common pattern in the ML literature seems to be:

1) Identify a commonly seen problem with deep learning architectures, whether that's large data volumes, lack of transfer learning, etc.

2) Invent a solution to the problem.

3) Test that the solution works on toy examples, like MNIST, simple block worlds, simulated data, etc.

4) Hint that the technique, now proven to work, will naturally be extended to real data sets very soon, so we should consider the problem basically solved now. Hooray!

5) Return to step #1. If anyone applies the technique to real data sets, they find, of course, that it doesn't generalize well and works only on toy examples.

This is simply another form of what happened in the 60s and 70s, when many expected that SHRDLU and ELIZA would rapidly be extended to human-like, general-purpose intelligences, with just a bit of tweaking and a bit more computing power. Of course, that never happened. We still don't have that today, and when we do, I'm sure the architecture will look very very different from 1970s AI (or modern chatbots, for that matter, which are mostly built the same way as ELIZA).

I don't mean to be too cynical. Like I said, I haven't read those particular papers yet, so I can't fairly pass judgement on them. I'm just saying that historically, saying problem X "has been addressed" by Y doesn't always mean very much. See also eg. the classic paper "Artificial Intelligence Meets Natural Stupidity": https://dl.acm.org/citation.cfm?id=1045340.

EDIT: To be clear, I'm not saying that people shouldn't explore new architectures, test new ideas, or write up papers about them, even if they haven't been proven to work yet. That's part of what research is. The problem comes when there's an expectation that an idea about how to solve the problem means that the problem is close to being solved. Most ideas don't work very well and have to be abandoned later. Eg., as one example, this is the Neural Turing Machine paper from a few years back:

https://arxiv.org/pdf/1410.5401.pdf

It's a cool idea. I'm glad someone tried it out. But the paper was widely advertised in the mainstream press as being successful, even though it was not tested on "hard" data sets, and (to the best of my knowledge) it still hasn't several years later. That creates unrealistic expectations.

trextrex4• 8 years ago

While I don't completely disagree with you, how would you propose researchers go about the problem?

If anything, machine learning is applied to real world problems these days more than it ever was.

For better or worse, AGI is a hard problem, that's going to take a long time to solve. And we're not going to solve it without exploring what works and what doesn't.

goolulusaurs2• 8 years ago

The Neural Turing Machine has been expanded upon considerably since that paper, and inspired a number of other memory augmented neural network architectures, including: https://www.gwern.net/docs/rl/2016-graves.pdf , https://arxiv.org/abs/1605.06065 and https://arxiv.org/abs/1803.10760 .

If you don't want to read a bunch of papers this video also has a good discussion of that last paper, describing how a memory system based on the NTM has been applied to reinforcement learning to achieve very impressive results that seem to me to be a very significant step towards human-like general purpose intelligence: https://www.youtube.com/watch?v=9z3_tJAu7MQ

vinn124• 8 years ago

3) Test that the solution works on toy examples, like MNIST, simple block worlds, simulated data, etc.

youre right: mnist, imagenet, etc are toy examples that do not extend into the real world. but the point of reproducible research is to experiment on agreed upon, existing benchmarks.

crazybit• 8 years ago

Why not praise DL and strive for practical applications, both present and future, instead of having the goal and evaluating success based on some nebulous "artificial general intelligence"?

red75prime2• 8 years ago

AGI can be seen as a system having practical success in all tasks humans can perform. It doesn't change much whether to care about AGI or not, as long as generality of narrow AIs increases.

mlthoughts20182• 8 years ago

A better title for the article might have been,

“Why current deep learning is not general enough for AGI or some other families of structured problems.”

Viewed this way, it’s not a criticism of pragmatically using deep learning or experimenting with deep learning for some narrow tasks.

Rather, it expresses what aspects of current deep learning make it unsuitable for general transfer learning, hierarchical or causal inference, or many Bayesian techniques requiring greater use of priors.

Other comments have pointed out that there are plausible rebuttals in deep reinforcement learning and metalearning.

But the bigger thing to me is to be clear that the article is not a criticism of deep learning engineering — applying deep learning to satisfy an explicit requirement, where the success criteria possibly has nothing at all to do with general intelligence nor with whether a certain approach can span some sufficiently large class of general models.

However, even if you just constrain your view to look just at so-called “pragmatic” deep learning — deep learning for concrete tasks — there are still a lot of unanswered questions about why things work, and whether or not an approach is learning semantic aspects of some true underlying structure (some latent variable space that captures the true data generating process) or if deep learning merely allows overfitting to particular populations of observation-space statistics.

This paper gives an example of exactly this issue for CNN models for image processing [0]. I’d argue that this is more of the kind of criticism relevant for task-oriented practitioners, whereas the OP link is some criticism more relevant for AGI or philosophy of statistics research at large.

[0]: < https://arxiv.org/pdf/1711.11561.pdf >

red75prime2• 8 years ago

Adversarially robust classifiers have interpretable gradients and feature representations [0]. The problem seems to be that the standard networks capture all the statistics there is, including surface statistical regularities and noise. It can be mitigated, though.

[0] https://arxiv.org/abs/1805.12152

mlthoughts20182• 8 years ago

Thanks for the link!

Note that it’s not just adversarial robustness that is a problem, as the paper’s result merely with altering surface statistics with a Fourier domain filter applied to the training data already creates problems for interpreting the network’s internal representation as having any type of semantic representation of the underlying structure relevant for the task, without needing to involve the part about adversarial robustness at all.

bra-ket• 8 years ago

learning by trial and error will never generalize well

std_throwaway• 8 years ago

What is the alternative?

Expert systems with a myriad of rules that fail the moment they get an input that was not considered?

someguy1234567• 8 years ago

ehem, evolution

afpx• 8 years ago

Really naive question:

How many deep learning models would need to be trained so that a model (maybe a decision tree) for generating deep learning models could be trained?

RoboTeddy• 8 years ago

This question isn't silly! People have actually trained neural networks to generate other neural networks. The "designer" network is trained using reinforcement learning, with a reward signal that depends on the efficacy of the networks it creates.

See: Neural Architecture Search with Reinforcement Learning (https://arxiv.org/abs/1611.01578)

It looks like they trained the "designer" network by trainning 15,000 deep learning models (800 at a time in parallel over 800 GPUs)

gwern• 8 years ago

The simplest approach (training NNs from scratch, using validation score as loss, simple DRL agent with REINFORCE as the designer) requires thousands of models to be trained and exorbitant GPU resources, like the Zoph paper. You can, however, get human-level designs with a few dozen or hundred samples at a tiny fraction of the computation cost if you are somewhat smarter about it - for example, reusing the parameters of a trained NN when you start training a new slightly-different NN (eg if you have a CNN with 20 layers and you want to try the exact same settings but with 21 layers, almost all of that CNN is going to be very similar to the end result of the 20 layers, so you can speed things up by copying over the first 20 layers, randomly initializing the 21st layer, and then training it for a short time). One nice efficient form of neural architecture search is SMASH: https://www.reddit.com/r/reinforcementlearning/comments/6uio...

More links on the general topic of 'DL optimizing/designing DL': https://www.reddit.com/r/reinforcementlearning/search?q=flai... (It's pretty critical to current DL approaches to few-shot/one-shot learning as well: you can think of it as 'designing' a new NN specialized to the few samples of the new class of data.)

hestefisk• 8 years ago

The term AGI I find quite misleading. It promotes the idea that the wonders of the human mind, knowledge, feelings, sensations etc can be quantified and turned into something algorithmic. I think a lot of AI researchers could do with a basic introduction to philosophy of science and the different forms of knowledge —- episteme, phronesis, techne — and perhaps also the structure of scientific revolutions (Thomas Kuhn, notably). The whole paradigm of AI presupposed that generalised intelligence can exist without biology, feelings, a body / senses. This is one of the reasons that AI reached the so-called “AI winter” in the 70ies where researchers boiled language and human knowledge down to algorithmic manipulation of symbols.

EDIT: John Searle’s Chinese Room argument is a good (and fun) place to start —- https://en.wikipedia.org/wiki/Chinese_room

drawnwren• 8 years ago

Replace Chinese with logarithm and the Chinese Room seems ridiculous. Searle could produce logarithms without knowing logarithms, that doesn't mean the knowledge to produce a logarithm isn't contained in the room -- just that Searle isn't where that information is encoded. By the same logic, a computer can't perform any operation because the RAM can't perform operations.

ricraz• 8 years ago

As an AI researcher who did a degree in philosophy, I think most of the things you mentioned are pretty irrelevant. How to build an AI with "feelings, sensations, etc" is indeed a mystery, but we don't need to aim for that, we just need to aim for intelligence, which can be defined without reference to consciousness or qualia. Similarly, if we can invent a working, fully automated Chinese room that passes the Turing Test, then whether or not it fits Searle's definition of "understanding" is a moot point (especially since his understanding of "understanding" is pretty weird).

More generally, although we should be heavily inspired by human intelligence when designing machine intelligence, it's a mistake to use the way humans think to define intelligence. Kuhn's account of scientific revolutions, for example, is primarily descriptive, not prescriptive. We can certainly imagine possible setups where science doesn't proceed like that, which may well be superior. Science isn't defined by revolutions, but by experimentally searching for the truth. In the same way, knowledge isn't defined by having a body, but by having beliefs which correspond with the state of the world.

whataretensors• 8 years ago

It doesn't need to be anything like the human mind. Like cars not resembling horses.

Deep learning: a critical appraisal

Related Stories