Artificial intelligence risk research

https://80000hours.org/career-guide/top-careers/profiles/artificial-intelligence-risk-research/

by BenjaminTodd • 11 years ago

43 74 11 years ago

andreyf16• 11 years ago

The possibility of human-level artificial intelligence poses significant risks to society [...]

No. It simply does not outside of the realm of science fiction. Understanding HGI will help us understand enough about ourselves and our motivations and ethics that it won't be an issue.

Humans augmented by machine learning systems, on the other hand, are here now and a wholly different question. Institutions empowered by AI research have a power / person ratio unlike any of those of the past.

Moshe_Silnorin3• 11 years ago

It simply does not outside of the realm of science fiction.

Why?

andreyf16• 11 years ago

In short, because human intelligence evolved in a different environment than AI. We are the results of the need to survive and reproduce, and the majority of our "intelligence" is based on this foundation. Algorithms have no such darwinian pressures, and there is no point in developing AI replacements for them, since humans abound.

platz5• 11 years ago

I'm not sure that because you envision a different, more plausible scenario, that completely invalidates the original claim, which was only simply denied before sketching out the alternative.

* I don't see how "Understanding HGI" prevents a "human-level artificial intelligence" from doing anything it pleases.

andreyf16• 11 years ago

My second claim is not there to invalidate the first, but rather to point out that there are two "AI problems", and that one is a lot closer to science fiction than the other.

I've expounded on the denial: we will need to first figure out what motivates humans before we can replicate their levels of intelligence. So in order to solve HGI, we will need to understand HGI, which means we will be able to control HGI. We will understand personalities and values in a systematic way, and we will be able to choose the personalities and values of any one instance of HGI.

conceit• 11 years ago

the problem probably is human-level intelligence is still stupid, so it's a risk either way

jerf2• 11 years ago

"It simply does not outside of the realm of science fiction."

If you could prove that, you'd put a lot of minds at ease. By "prove" I mean not necessarily a full mathematical proof, but certainly more than a few paragraphs of English text.

I'm skeptical myself, seems like the more we learn about biology the more it is doing that we did not realize, but skepticism is at best the first step to proof.

andreyf16• 11 years ago

I think it will be obvious once HGI is sufficiently understood, in a way that will not need more than a paragraph, if that.

The fear and idea of somehow creating an HGI without understanding how to control it came from The Terminator. It's pure fantasy.

BenjaminTodd4• 11 years ago

The purpose of the profile isn't to argue a risk exists. We largely defer to the people we take to be experts on the issue, especially Nick Bostrom. We think he presents compelling arguments in Superintelligence, and although it's hard to say anything decisive in this area, if you think there's even modest uncertainty about whether AGI will be good or bad, it's worth doing more research into the risks.

If you haven't read Bostrom's book yet, I'd really recommend it. http://www.amazon.com/Superintelligence-Dangers-Strategies-N...

manish_gill5• 11 years ago

I read this book. Got about 4 chapters in before I had to give up at the sheer ridiculousness of the whole thing. The problem with this entire line of reasoning is that at this point it is nothing more an a thought experiment. Many of the key underlying assumptions that are required for Artificial General Intelligence simply have not been realised, and while Weak AI is progressing at a strong rate, we are hardly anywhere close to a place where we should start worrying about all this stuff.

It's about as likely as a meteor hitting the planet and wiping out the entire human life. Possible? Sure. Should I be panicking about this right now? Nah.

The book is nonsense. I'll start paying attention when someone who has real experience in the field of AI research (and I'm not talking charlatans like Yudkowsky here), but someone like say Norvig come out and say it's a reasonable concern today.

ctogden• 11 years ago

What is HGI? Google/Bing/Wikipedia disambiguation are no help.

andreyf16• 11 years ago

Human-level General Intelligence.

danieltillett5• 11 years ago

Human level AI would have a profound impact on the value of human labor. If you don't consider everyone being unemployable an issue to society then I am not sure what would pose a significant risk.

Most people that worry about this topic don't worry about human level AI, it is the vastly greater than human level AI that is the concern.

punee5• 11 years ago

How does human level AI make everyone unemployable?

andreyf16• 11 years ago

I think ML will first make (some) people significantly more productive, but yes, the economic shift in productivity we're starting to see now is very much worth thinking and talking about.

throwaway808• 11 years ago

Two examples outside of science fiction:

- The horse population declined dramatically after their work abilities were exceeded on every dimension by machines

- Homo Neanderthalensis disappeared after the appearance of Homo Sapiens.

I'm not saying these are perfect analogies. But you are saying they're completely irrelevant. Why is that?

nairboon3• 11 years ago

Good old Homo Sapiens is already disappearing thanks to Homo Oeconomicus. I don't see a problem if we move on to Homo faber next.

threatofrain• 11 years ago

How are you so confident that the risks of machine learning are in the realm of science fiction? Machine learning, at least, has the plausible potential to permanently reduce demand on labor by capital owners. Undemanded laborers lose their ability to negotiate or strike as a power counterbalance to capital owners.

At the very least, that future is too complicated to form such a confident opinion that the risks of machine learning are in the realm of science fiction.

andreyf16• 11 years ago

There are certainly changes happening (and more on the way!) as a result of a combination of Moore's Law and ML research. I don't think they are inherently significantly more risky any more than other technical advances as long as we have a chance to think about and understand them. We definitely need to think through the social implications of computers.

I think it's important to separate those real developments from the movie-plot-threat of a super-AI escaping human control.

rhaps0dy• 11 years ago

Be sure to de-lurk if that's the case!

I guess. A lot of people already did.

http://lukemuehlhauser.com/if-youre-an-ai-safety-lurker-now-...

Artificial intelligence risk research

Related Stories