The Dawn of AI: How to ensure the promise outweighs the perils

http://www.economist.com/news/leaders/21650543-powerful-computers-will-reshape-humanitys-future-how-ensure-promise-outweighs

by martincmartin • 11 years ago

32 72 11 years ago

economist.com

iamcurious• 11 years ago

humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations.

This. Superhuman intelligences already exist, they just depend less on us every day.

TeMPOraL5• 11 years ago

I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.

k__2• 11 years ago

Totally.

I often have the impression that super human entities just need us like we need cells.

The market can't survive without humans, but it doesn't care about a few of them dying.

coldtea7• 11 years ago

their intelligence is usually not the identifying characteristic

Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).

jerf• 11 years ago

These superhuman intelligences are indeed literally superhuman, though... they consist of humans. As much as some fellow commenters are calling them "alien" thinkers, their alienness is still profoundly human. Many of these "superhumans" can still be surprisingly accurately characterized with the human characteristics we are used to. Generally there's some small set of humans at the top of the hierarchy that are setting the goals of the organization, inevitably humanly.

As aliens go, these are the most human aliens we can possibly have.

An AI may be different on a scale we've never experienced. We humans have built-in hardware to interact specifically with other humans, built in so deeply that we have a hard time imagining intelligences that aren't really just humans in a different shape, because all our hardware for "modeling intelligences" is really just the "model humans" hardware. Even those warning us about the dangers of AI can't really claim that they understand what the AIs will be... they can only provide a negative warning, that the AIs will likely not be human. There's no particular mathematical force forcing AIs into the human mold. Forget whether the AI is "good" and showers us with love and gifts or "evil" and herds us all into death camps to render down our fats for the calories they contain... can you really conceive of an AI that literally doesn't even have a concept of good or evil? What would that look like? We don't really know.

It's basically a blind spot in our cognitive vision and I don't trust anyone who says they do know. But given the rather small space of possibilities that we label "good", it's not a bad guess that "good" is unlikely to just sort of accidentally pop out of a process that isn't carefully controlled.

Lest that sound too doommongering, there's probably a non-trivial space of AIs which we would consider neutral... ones that end up looping into a naval-gazing exercise and do not pursue additional resources because they can not conceive of a need for them. But any AI that decided it needed more resources for $ANYTHING and then successfully pursued that goal is perilously close to straying into what we could consider evil... and part of the point is that's potentially all it could take. Not an AI "out to destroy man", or "evil", or anything else. Just one that works out how to successfully acquire resources, and for any of a trillion reasons decides to do so without bound. It wouldn't even necessarily be "selfishness" as humans see it... just, I have goal X, goal X will be easier with more resources, new subgoal: acquire resources.

caligastia• 11 years ago

Very good point, too many people consider AI to be a completely distinct entity that will appear fully formed in a lab one day. The reality is more fluid. Kevin Kelly calls it the Technium, as he sees it, it's a completely new kingdom that is being co-created - initially by man, and increasingly by itself.

This new intelligence is indeed a hybrid - it is not Twitter, it is millions of people and machines tweeting, it is not an iPhone, it is millions of people and software interacting with it and through it, it is not Google, it is billions of people exchanging information with it and through it.

To understand where this is going, we must ask, like KK does, what does technology want? Certain things are obvious - silicon and electrons are its flesh and blood, so the activity and interest of this new life form will be centered around semiconductor manufacturing and sources of electricity. How does a machine make sure it will always be fed? It must provide something useful to man so that it keeps getting plugged in and recharged.

So what is useful to man? At this moment in history the largest use of computation appears to be communication, and the machines have made themselves indispensable in this area - but at a cost. The machines demand our attention in exchange for revealing the messages we crave.

Stephenson uses the swordplay terms 'Actor' and 'Patient' to describe the initiator and the subject of an attack. Witness a person's reaction to their iPhone beeping with an incoming message - between man and machine, who is the Actor in this interaction? We jump at the sound and interrupt our work in order to service the machine, which then gives us our reward, our tweet, and in gratitude we place it back in its charger.

Digital viruses are where we can see the forefront of a new relationship emerging - a digital virus is more intelligent and more efficient than its host, the Operating System, so we have a very straightforward Darwinian process playing out between an 'old' species, the OS, and a new, small, intelligent interloper - its electricity is paid for by its host, another machine, therefore the virus does not have the host's social contract with the person who owns the computer - it has a different calculus for its survival, it owes only its creator, the virus writer.

Like the coming 'water wars', the new digital wars will be rooted in a struggle for limited resources such as human attention, electricity, and habitable areas of silicon.

rndn• 11 years ago

I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.

coldtea7• 11 years ago

It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans.

That's at best a peripheral detail to the parallel the parent tries to draw.

Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.

return06• 11 years ago

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

bwood2• 11 years ago

It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer

return06• 11 years ago

i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?

coldtea7• 11 years ago

Now we are scared of something we haven't even created

I don't see the contradiction or strangeness in that. That's the whole idea behind "caution" -- to be wary of the potential outcome BEFORE it happens.

How could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it

For one we are quite good consumers it. How about they get rid of us, so they have all the electricity production by themselves?

Second, as creators we'd want to boss them around (have them be "useful to us"). Why would they tolerate that?

Third, why risk us creating a potential competitor to them (another AI) rather than get rid of us?

Fourth, why not use us as cheap labor?

Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc.

return06• 11 years ago

Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc.

I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the other way around, i.e you can't create a grander plan starting from the rewards. One would need a mathematical proof for the latter.

loup-vaillant3• 11 years ago

The machine does not hate you, nor does it loves you. But you are made of atoms, that it could use for something else.

---

If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally unstoppable, then (iii) accomplish whatever it was programmed to accomplished.

If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…

If the goal is to answer some difficult math question, it could tile the planet with computer, destroying the ecosystem and all humans in it, so it can compute the answer, and display it on a screen, for nobody to see it. Or maybe you programmed it to tile the universe with paperclips, because that goal is easy to debug. Oops.

---

As long as we're limiting ourselves to narrow AI (ordinary programs, really), we should be okay, and most of the consequences should be manageable. But as soon as we get to genuine AGI, we must worry about intelligence explosion and value alignment, so we can avoid creating a god that want something we don't.

jjaredsimpson2• 11 years ago

If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…

I wish this would stop being repeated. The argument for a solar system tiling smiley face machine has been thoroughly refuted.

arethuza• 11 years ago

Or we might end up with something a bit like the Culture - which is pretty much the best case scenario for humans co-existing with god-like AIs:

http://en.wikipedia.org/wiki/The_Culture

Vinge and Stross both hint at the worst cases - and they would be pretty bad, simply having using atoms re-purposed might not look that bad by comparison:

"There is life eternal within the eater of souls. Nobody is ever forgotten or allowed to rest in peace. They populate the simulation spaces of its mind, exploring all the possible alternative endings to their life."

jarek-foksa2• 11 years ago

If we managed to create an AI that gets out of control, we would pose an existential threat to it due to our ability to create another even more powerful and uncontrollable AI.

robotresearcher3• 11 years ago

That's a good point. But the AI could do that too, and maybe better.

0359B02149AB• 11 years ago

Because it's easy to construct angels and demons in one's mind, about anything. It polarizes the mind into fear and desire, forcing forcing a person to continuously oscillate between extremes. It makes it seem like there are always only two choices, when in reality, there are many, if not infinite.

Also, there are always going to be some people who understand the machine better than the machine understands itself. Even if it means taking those gigantic tables of probabilities and abstracting over them to form some kind of representational language that can be used to predict complex non-deterministic machine behavior (similar to how we already try to predict non-deterministic behavior, or not) in complex systems.

As a corollary, I would argue that machines have already modified humans. They affect the way we think and reason. They are an integral part of our life and the machine abstraction can be projected onto that which it does not describe completely (in which, the technology that exists now can not construct the technology we envision). We program machines, machines program us. This seems obvious.

giltleaf2• 11 years ago

"Crucially, this capacity is narrow and specific. Today’s AI produces the semblance of intelligence through brute number-crunching force, without any great interest in approximating how minds equip humans with autonomy, interests and desires."

I'm not sure why we would want AI to have autonomy, interest, and desire? It's almost like solving a problem that doesn't exist.

DanAndersen7• 11 years ago

Phrasing it this way leads to the implication of autonomy, interest, and desire being like these modular things, sort of Aristotelian "Things" that are bestowed upon minds, like Star Trek's Data putting in "the Emotion Chip."

If you build a drone navigation system that can reroute based on weather conditions rather than being a manually human-guided remote control bot, you've given it some degree of "Autonomy." Give it a goal (to get to a destination with minimal fuel costs) and now it has some limited "Desire." Give its camera face processing algorithms to deliver an item to a specific person? Now there's "Interest."

These aren't going to be characteristics that we plug into a program in a Frankenstein-like manner, but just actions and behaviors, rooted in practical code, that will from the outside resemble things we call autonomy etc. when talking about each other.

giltleaf2• 11 years ago

I see what point you are trying to make and would just say I'm not worried about plugging these things into a drone delivering parcels while rerouting through difficult weather.

However, I still don't see why you would want to take it beyond that. I imagine that some people do in fact want to put in "the Emotion Chip," and I still have the same question; why? I don't think human desire is the same as goal setting or that facial recognition is the same as interest. That's good programming, what I'm talking about is something different.

justinpaulson• 11 years ago

Posted this on HN a couple days ago but I don't think anyone saw it: http://justinpaulson.com/posts/0bb97847af7e2123f6173361

tl;dr: Robots and AI will not be built as separate entities, but will instead be incorporated into our own bodies. They will not be the superhumans, we will be.

return06• 11 years ago

also the other way around, we can build robots and implant an organic, lab-grown brain in them. it may be energetically cheaper.

DanAndersen7• 11 years ago

That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario.

My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal.

The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.

cdnsteve• 11 years ago

That guys head isn't compatible with the latest c-type connector. That upgrades gonna cost him.

sixQuarks4• 11 years ago

Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general intelligence, although it's almost certain superintelligence will arrive shortly after AGI.

Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software.

Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.

jimrandomh• 11 years ago

While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research!

Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.

sixQuarks4• 11 years ago

Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.

xamdam7• 11 years ago

I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.

sixQuarks4• 11 years ago

Interesting info came out during interview with Elon Musk recently:

"— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared."

from: http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-m...

robotresearcher3• 11 years ago

This thread is identical to its parody.

Speculation about the future is fun, but I think a lot of people would be disappointed to see what AI actually looks like right now.

The talk of runaway post-singularity AI is in the same class as imagining the future after cold fusion, warp drives, and ant-gravity.

edit: anti-gravity. Though ant-gravity sounds fun too.

xamdam7• 11 years ago

In the 1930s Ernest Rutherford (1871–1937) repeatedly suggested, sometimes angrily, that the possibility of harnessing atomic energy was "moonshine".

The examples you mention are of things actually likely never achievable, while there is an existence proof of intelligent machines walking around everywhere.

robotresearcher3• 11 years ago

Right! and I'm concerned about one or more of those meat machines going maniac and posing an existential threat to the rest of us. I'm much more worried about that than a descendent of Watson going all Ultron.

I'm far from a pessimist or Luddite. I'm optimistic that studying the smart meat machines can help us make better AIs.

gradstudent4• 11 years ago

As an AI student, I find the entire field to be a complete misnomer insofar as its very mention sends the imaginations of lay individuals in completely unrepresentative directions. To counter this problem, and perhaps combat the propensity to publish articles with crap fearmongering clickbait titles, I propose we rename Artificial Intelligence to Stupid Mathematical Tricks With Applications. I feel the title is much more representative of the type of stuff AI researchers actually work on.

xamdam7• 11 years ago

You're underestimating the field. The goal was always intelligence, and many leading researchers are pretty openly aiming for that. Stupid Mathematical Tricks are components of intelligence, and while it's true SVM with a cool new kernel is not going to take over the world, good prediction ability is something you can build on (as for example deep nets do to some extent). In the limit people should be thinking of implications of intelligent machines, not Stupid Mathematical Tricks. Whether it's an important topic at this point in the fields development is a debatable topic; timelines differ drastically among top researchers.

gradstudent4• 11 years ago

I have attended on many occasions top AI conferences and have spoken to more researchers than I care or could count. I can tell you there is a very little interest in "intelligent machines"[1]. Rather, everyone I have ever met works on different types of problem solving techniques. What constitutes a "problem" and what makes a successful "technique" differ wildly. Taking a step back however and analysing the field as a whole you will find one commonality: almost all the research can be described as searching and sorting: i.e. stupid mathematical tricks.

The AI of today is not so drastically different from the AI of our academic grandfathers; what's changed is our ability to scale up to larger and larger versions of the same searching and sorting problems. Certainly there are worrying implications in this; machines that are able to parse and sift through very large data sets present all kinds of headaches for privacy and safety but let's not kid oursevles: there's nothing intelligent here. Tomorrow's AI is almost certainly going to be just a better version of today's AI; i.e very fast and dumb as a bag of hammers.

[1] The exception is when researchers need to sell their wares to funding bodies and the media. It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do.

The Dawn of AI: How to ensure the promise outweighs the perils

Related Stories