The Myth of a Superhuman AI

https://www.wired.com/2017/04/the-myth-of-a-superhuman-ai/

by conanxin • 5 years ago

30 88 5 years ago

Hyper-intelligent algorithms are not going to take over the world for these five reasons.

I agree.

Just like a nuclear power plant will never meltdown, because they are extremely well designed with large safety buffers.

And just like a pandemic will never be started by lab experiments, because they use extreme safety measures when they work with viruses.

We can always just unplug the AI or disconnect it from the Internet, or just not give it a survival instinct, right?

gilch• 5 years ago

Pretty sure this is sarcasm, as in, the lies are so transparent they're humorous, and this was on purpose.

For those who didn't notice, nuclear plants have, in fact, melted down, more than once. Biosafety level 4 labs have, in fact, had leaks of pathogens before, more than once. An agent AI will tend to develop a survival instinct by default, even if we don't give it one, because it is a convergent instrumental goal. Thus, it would be motivated to not simply let you yank it, would fight you if you tried, and may even take preemptive precautions.

tech-historian5• 5 years ago

Who is the author Kevin Kelly and why does he think he has a greater hold on this issue than luminaries like Ray Kurzweil?

mindcrime7• 5 years ago

To the first half of your question:

https://en.wikipedia.org/wiki/Kevin_Kelly_(editor)

To the second half:

Dunno. Good question.

cowpig3• 5 years ago

Appeals to authority are not convincing

mindcrime7• 5 years ago

This isn't a formal debate, and authority is useful especially as a "first pass" filter, or in terms of establishing Bayesian priors. The key, IMO, is not to assume that the mere existence of authority guarantees that someone is right. But in either case, it's certainly valid to ask the question "why should I take Kevin Kelly seriously on this topic?"

AnimalMuppet4• 5 years ago

Why is Ray Kurzweil considered a luminary in this context?

Sure, he talks a lot. He makes grand statements. Is there any reason to take him seriously in this context?

He sure seems to believe, and to want to believe. But actual evidence that he's going to be right any time soon? That seems to be missing.

enord3• 5 years ago

Everybody wants to be the Fox Mulder of their story.

tech-historian5• 5 years ago

It's clear you're not familiar with Kurzweil's work in AI (or his background in general). He's written multiple books on the topic over multiple decades, including the seminal "The Singularity is Near". He's devoted his life to the study of this field.

https://en.wikipedia.org/wiki/Ray_Kurzweil

https://en.wikipedia.org/wiki/The_Singularity_Is_Near

GeorgeTirebiter• 5 years ago

Have you read Ray's books? I recommend 2005's The Singularity is Near. https://en.wikipedia.org/wiki/The_Singularity_Is_Near

aaaaaaaaaaab• 5 years ago

luminaries like Ray Kurzweil

This is satire, right?

sxp• 5 years ago

tl;dr: Humans have souls which machines will never have. The core of the article is this quote "I will extend that further to claim that the only way to get a very human-like thought process is to run the computation on very human-like wet tissue. That also means that very big, complex artificial intelligences run on dry silicon will produce big, complex, unhuman-like minds." which is just the boring old argument against replicating human thinking.

Similarly, there is an extreme amount of pro-human arrogance in the article "At the core of the notion of a superhuman intelligence — particularly the view that this intelligence will keep improving itself — is the essential belief that intelligence has an infinite scale. I find no evidence for this.... So the question is, where is the limit of intelligence? We tend to believe that the limit is way beyond us, way “above” us, as we are “above” an ant. Setting aside the recurring problem of a single dimension, what evidence do we have that the limit is not us? Why can’t we be at the maximum?"

Humans are just a bunch of atoms that randomly learned to think over a few billion years. Given how fast we can improve tech via directed means, we should be able to replicate what nature has done in thousands, hundreds, or tens of years.

cowpig3• 5 years ago

The word "soul" does not appear once in the article

DuskStar• 5 years ago

That also means that very big, complex artificial intelligences run on dry silicon will produce big, complex, unhuman-like minds.

Incidentally, this is exactly what the AI superintelligence people are terrified of, and what the paperclip problem is meant to illustrate. https://www.decisionproblem.com/paperclips/index2.html

mindcrime7• 5 years ago

Ugh. This is a pretty bad article. And don't get me wrong... I'm no "AI Alarmist." But to me, this article is equally as useless as the various pronouncements it seems to be trying to refute. There's a lot of verbiage here, but no cohesive, coherent argument to support the thesis that "Superhuman AI is just a myth" (which is to say, "Superhuman AI is not possible").

The reality is, there are a lot of unknowns in this space, and to dismiss the possibility of an "evil super-intelligence" out-of-hand is, IMO, every bit as silly as buying into the "OMG, AI is unleashing the demon" fear-mongering. I just think it's too early to know for sure the answers to some of these questions, like:

1. Is AGI possible at all (for whatever definition of "AGI" you want to use)?

2. If the answer to (1) is "yes", then what about ASI (Artificial Super Intelligence) - is that possible?

3. If (1) and (2) are both "yes", then is it possible to create an ASI that will either be benevolent, or controllable, or both?

And so on... we just don't know yet. And that in and of itself arguably suggests the exercise of a certain measure of caution.

enord3• 5 years ago

You're making an argument that is already in TFA. It's in addition -and only partially related to- the main point which is: The arguments that are commonly used to demonstrate the possibility or threat of "ASI" are made from claims presented _without evidence_.

This will invalidate fantastical accelerationist woo until such evidence is presented, and -in the current state of the art research within relevant fields- it's not even certain we can know what would constitute such evidence _in principle_.

mindcrime7• 5 years ago

I assume you're referring to this part:

If the expectation of a superhuman AI takeover is built on five key assumptions that have no basis in evidence, then this idea is more akin to a religious belief — a myth

My problem is, even this is wrong. I do not find the "five key assumptions" he cites to be the foundation of beliefs that a dangerous "Superhuman AI" could emerge. So to me, this article simply fails right from the jump. The thing he is arguing for and/or against just don't, collectively, constitute a well-developed argument for the overall thesis. Not in my opinion anyway.

To go through them one by one:

Artificial intelligence is already getting smarter than us, at an exponential rate.

I don't know anybody making this claim, nor is it apparent that belief in this is necessary to think that dangerous "Superhuman AI" could be a problem in the future. I guess it comes down to what you think Kelly meant by "soon" in the preceding sentence.

We’ll make AIs into a general purpose intelligence, like our own.

It's the "like our own" part of this that I have a problem with. Yes, I think many people believe that we will create some manner of "general intelligence", but I also believe that a lot of researchers acknowledge that it may not be 100% "human like". But this may be my own bias showing, as I personally harp a lot on the importance of the distinction between "human level" intelligence and "human like" intelligence.

We can make human intelligence in silicon.

This is probably the most sound one of all, but here's the thing... saying "there is no evidence for this" is simply wrong. To hold to this is to completely ignore all progress made building AI in silicon to date. And while we don't yet have "general intelligence", I don't see how one can just ignore incremental progress completely and say "there's nothing there". I would argue that the null hypothesis should probably be that general intelligence on silicon is possible that that the burden of proof is on the people arguing the opposite.

Also, not everybody thinking about / working on AI is focused on the "in silicon" approach. The majority? Yes. But I'm sure there are a handful of people working on ideas related to DNA computing, chemical computing, growing synthetic brain matter, etc.

Intelligence can be expanded without limit.

To me, this may or not not be true, but it's completely irrelevant to the thesis at hand. The question isn't "can machine intelligence be unbounded?", but rather "can machine intelligence exceed human intelligence?"

Once we have exploding superintelligence it can solve most of our problems.

Again, this strikes me as mostly irrelevant to the question at hand.

So yes, this article raises some interesting points, but I don't see it accomplishing what it seems to purport to do.

idatum• 5 years ago

Maybe the article was AI generated? /s

eutectic2• 5 years ago

Humanity is an existence proof for human level AI, and it seems arrogant to assume that our brains are at the limit of what is possible.

Reproductive fitness requires many attributes which obviously compete with intelligence, and evolution is a blind hill-climbing search which cannot easily escape deep local minima.

To me it's obvious that the development of superhuman AI is inevitable assuming we don't wipe ourselves out before we have time to develop it.

dragontamer2• 5 years ago

But Google can read, injest, and search for information far faster than I can.

Stockfish players chess far better than I can.

AlphaGo plays Go far better than I can.

Random Sudoku bot built on top of minizinc / z3 / whatever SAT solver can solve NP-complete problems better than any human.

--------

Here's the thing: what is "strong AI" ?? As far as I can tell, we're already at superhuman AI in most intelligence tasks (collecting data, collating it together, analyzing it, etc. etc.).

Its called a computer. What if "superhuman AI" is just the stuff we have today at our fingertips? It wasnt very long ago when reading documents and organizing them was considered AI. But today, we call that Google.

PaulHoule2• 5 years ago

When I was involved in physics I thought that the range of problems that were solvable were like a continent (linear systems) surrounded by islands (integrable systems and a domain in which perturbation theory works around them.)

I thought that the problems solved by Bethe, Feynman and their cohort were the easy problems and the current generation is struggling with problems that were much harder, possibly impossible to solve.

One example is the problem of chaotic dynamics with six degrees of freedom as opposed to four degrees of freedom. With four degrees of freedom you have unbroken KAM tori that partition the phase space into areas that it will stay in forever. In 6 DOF or more, some tori are still unbroken but it's possible for the system to go around them, so even in a phase space region that has a lot of intact tori (e.g. the motion of the planets around the Sun) you can't prove anything for the long term (e.g. the Earth doesn't get ejected from the solar system.)

Look also at examples such as the halting problem, Godel's theorems, etc. I'm not sure if a "higher intelligence" could actually solve that many problems we can't because the problems are intractable.

tshaddox2• 5 years ago

Humanity is an existence proof for human level AI, and it seems arrogant to assume that our brains are at the limit of what is possible.

Why? Is it arrogant to assume that Turing machines are at the limit of what is possible to compute? Perhaps, but we also have good reasons to believe that to be the case. If you're just talking about things like computational speed differences or being able to control powerful machinery and weaponry, then those are already available for humans to use, and can already be used by humans to do bad things much more effectively. It seems to me that any future improvements in computational speed, memory capacity, etc. will also continue to be available to humans as well as any strong AIs we might develop.

upearly4• 5 years ago

This is a pretty bad article.

Agreed.

Dunno why he thinks this: "At the core of the notion of a superhuman intelligence — particularly the view that this intelligence will keep improving itself — is the essential belief that intelligence has an infinite scale."

The notion of self-improvement is both powerful and not assumed to be 'infinite'.

I figure it goes like this:

Q. Is AI potentially dangerous? A. Yes.

Q. Will it be human-like? A. Probably not.

People tend to couch these discussions as a series of anthropomorphisms.

goatlover3• 5 years ago

The article is a response to claims about the singularity and subsequent worries about concerning super AI.

blamestross8• 5 years ago

The bottleneck for intelligence is experience. Running on silicon doesn't allow an intelligence to engage with reality significantly faster than a human can. There are some benefits due to access to data and processing power, but the data isn't useful without the context of experience and all processing power can do it overfit model without experience.

Even if we put AI into an army of robots running around and experiencing things, there are still scaling limits to encoding and communicating knowledge and understanding. Human organizations are a great example of the scaling limits of intelligence.

skybrian2• 5 years ago

Actually no, there is almost no chance that it will be harder to transfer skills between machines than it is to raise and educate a child.

Something very weird would have to happen for it to be that hard.

shkkmo• 5 years ago

Is it?

Humans have been optimizing for information transmission between individuals for a long time. Not just our hardware and neural nets have been optimized, but the structure of the information itself.

It is quite possible that you need such a large set of compex interactions to develop intelligence that intelligence can only develop in reality. It is possible that the process for doing this inherently inolves a chaotic system that makes duplicating or paralleling that development impossible. I don't see any reason why these artificial intelligences with divergent chaotically emergent capabilities would have to be able to teach each other things as easily as we do.

I've long been of the opinion culture / "social software" is vital for producing knowledge for humans and that this social software has been fine tuned and iterated upon for a very long time (and possibly co-developed with the evolution of our brain hardware so that both are co-specialized for each other). This software is likely very specialized to our brain structures and might not generalize well to other intelligence substrates. Thus even if we do develop neural network structres that can run on silicone and match human level intelligence, those entities would still need a lot of interaction with reality to develop their own culture that would allow them understand the world at the level we do. Edit: Is it even possible to efficiently run the necessary theories of human mind on non-human brains to operate effectively in our epistemic landscape? Communication requires huge amounts of inference using theories of mind to resolve ambiguity and add implicit information.

It is possible that we can "recompile" human knowledge to run on other types of intelligences, but I suspect this will be far more difficult that commonly expected. Just look at how hard (nigh impossible) it is today to even boostrap a new general purpose computer without depending on the output other general purpose computers.

We simply do not understand cognition or semantics well enough to know the answers to these questions adequately or with any degree of certainty.

ryandvm• 5 years ago

That's a great point. However, whereas humans share their experience through the effective, but extremely lossy technology called language; machines can store and share their experience without loss. Sure there are scaling limits, but a brain dump back to the mothership is a lot quicker and more perfectly preserved than sitting around listening to grandpa's stories.

habitue2• 5 years ago

Really fascinating counter-argument to this view is here: https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien...

The basic idea is that, the smarter you are and the faster you can think, the more information you can extract from weak signals. It involves creativity, and building models, and some minimum amount of in-built bias, but the upshot is that the amount of information a super-intelligent AI could infer from very small amounts of experience might absolutely stagger the imagination.

blamestross8• 5 years ago

Even if it is right, it is still predicated on a massive "critical mass" of knowledge and insight about reality that such an AI would have to spend a long time accumulating. It also doesn't propose solutions to performance scaling limitations.

totally_not_ai2• 5 years ago

Are you of the opinion that you can't get experience by reading books or watching youtube?

Do you think everything taught in an oral form (as opposed to hands on) is completely useless?

Are all military strategy books used by military academies a waste of time, since you can only learn by leading a real army?

blamestross8• 5 years ago

All of those things have to be integrated with experience to be useful. Often the learner already has a lot of that experience, so the data can be integrated easily.

You can't understand those books or information without having a lifetime of experience to contextualize them. You can't convert them from rote knowledge into skills without trying them out.

namedlambda3• 5 years ago

Everything in the above comment is wrong.

In AlphaZero for example, there were 44 million training games total for 700,000 steps of training for the full 9 hours.

Turning that to human-scale numbers, 44million games with on average 60 moves, at 1 second thinking time per move,

44000000*60/60/60/24/365 = 83,7138508371 years of training experience in 9 hours

The whole field of Reinforcement learning has agents training and playing games for many orders of magnitude more time than a human ever will. In-fact, we can scale this to over 100k of actions per second, in a single machine:

https://github.com/alex-petrenko/sample-factory

Then, there is also distributed Reinforcement Learning, where hundreds of agents can play at different machines and share experience, see AlphaZero, LeelaZero, R2D2 agent, R2D3 agent, Apex, Acer, Asynchronous PPO.

but the data isn't useful without the context of experience

The experience is the data in Reinforcement Learning.

and all processing power can do it overfit model without experience.

That is wrong, the agents perform what is called exploration to avoid getting stuck in simple strategies.

Even if we put AI into an army of robots running around and experiencing things, there are still scaling limits to encoding and communicating knowledge and understanding.

True, but machines scale better because they speak the same language, or they can learn to tune their language to get their message across.

Human organizations are a great example of the scaling limits of intelligence.

Human organization is a testament to how far we can get with something as limiting as the commonly used language. The language that we use to communicate is subject to misinterpretation due to our subjective experiences, this limitation is not shared by machines.

blamestross8• 5 years ago

If the universe is a game you are playing, then yes playing that game is "experience", but for an AI to engage with reality it has to have experience in reality, not a game. The ability to play go very well doesn't enable an AGI to better understand reality.

The experience is the data in Reinforcement Learning.

This is very true, and the critical problem. Data about how reality responds to an AI's actions is very sparse right now.

AIs do have a potential advantage in communications efficiency, but at some level of scale compression will happen, locally "irrelevant" data will be discarded and simplified approximations replace it. None of this will change the "big O" of the scalability of intelligence, just the constant factors. There is no exponential kickoff point.

choeger• 5 years ago

Intelligence is not a single dimension, so “smarter than humans” is a meaningless concept.

That's not the point. We can map any multi-dimensional form of intelligence into a single dimension with any suitable norm. It just matters that for any dimension we can quantify intelligence in the sense that something more intelligent is imaginable.

Humans do not have general purpose minds, and neither will AIs.

That does not matter at all. Then a superhuman intelligence is not "general", so what? It's still wastly more capable then any human.

Emulation of human thinking in other media will be constrained by cost.

But what is the cost constraints? Human brains are constrained by mechanical considerations. How efficient could a computerized AI become?

Dimensions of intelligence are not infinite.

Again, what's the argument? The number of atoms in the universe is not infinite, either (probably).

Intelligences are only one factor in progress.

That's not the problem of the super AI.

mindcrime7• 5 years ago

FWIW, I agree with your sentiments pretty much 100%. I was going to say more or less the same thing(s) but didn't feel motivated enough to do more of a "point by point" response. Glad to see somebody chipping in with very similar viewpoints.

jbay808• 5 years ago

Reminds me of the brilliant "On the Impossibility of Supersized Machines" [1].

We show that it is not only implausible that machines will ever exceed human size, but in fact impossible.

[...]

The term “supersized machine” implies a machine that has crossed some threshold, which is often denoted “human-level largeness.” However, it is not clear what “human-level largeness” could refer to. Has a machine achieved human-level largeness if it has the same height as the average human? If it has the same volume? The same weight? [...]

When one begins to consider these questions, one quickly concludes that there are an infinite number of metrics that could be used to measure largeness, and that people who speak of “supersized machines” do not have a particular metric in mind. Surely, then, any future machine will be larger than humans on some metrics and smaller than humans on others, just as they are today.

[1] https://arxiv.org/abs/1703.10987

davidivadavid• 5 years ago

Yeah, I stopped reading after making roughly the same assessment. Article is just one giant straw man. And I'm not a particularly staunch AGI proponent.

croes• 5 years ago

Bold claim. At the moment we don't fully understand the brain and intelligence. Even what makes the difference a living and a non living accumulation of atoms is unknown.

But this doesn't not mean we couldn't create a superhuman AI in the future or by accident.

I think we don't have the capabilities to do so but this isn't a 100% certaincy more like 99%

arisAlexis3• 5 years ago

"the problem with the world is that the intelligent are full of doubt and the idiots cocksure". Applies directly to the bold title. Doesn't matter if we are all annihilated of course nobody will remember.

lern_too_spel• 5 years ago

Old article. Previous discussion: https://news.ycombinator.com/item?id=14205042

mcswell• 5 years ago

Looks like you're being ignored, or at least no one (until me) has replied. I guess people would rather hear themselves think than look at what other people said a few years ago.

But thanks for bringing up this link!

YetAnotherNick2• 5 years ago

There is no reason we know that brain emulation is not possible. Even if one second takes 100 years in processing now, there is no reason to think that there is fundamental limit in how many processor you could connect. And for superhuman it just needs to do one thing well(which computers already does) than human and match humans in everything. I am not saying it would be done but I haven't seen any argument on why this exact argument is "impossible".

otabdeveloper42• 5 years ago

There is no reason we know that brain emulation is not possible.

Well, neither is there any reason we know that it's possible.

You're just demonstrating your apriori biases ("the brain is just a computer"), nothing more.

YetAnotherNick2• 5 years ago

We have reason to believe it's possible because we know we can simulate a single neuron. It's just the sheer complexity of full brain we can't scan and efficiently simulate.

qsort2• 5 years ago

It's not impossible in principle, but:

- we don't understand enough about the brain to emulate it.

- we don't know how to model the brain.

- even assuming we had the required scientific knowledge, there would likely be major engineering problems.

So, in other words, it's not impossible in the same sense FTL travel is not impossible, our scientific understanding does not categorically exclude it, but making predictions as to when it will happen is kind of silly, because it will likely require several fundamentally unpredictable scientific breakthroughs.

goatlover3• 5 years ago

As the article points out, the nervous systems is larger than the brain. There are millions of neurons in your gut. Your sensory organs have neurons that prepare the signal sent to the brain from the sensory noise. No brain lives independent of a body. That's not how organisms evolved.

blamestross8• 5 years ago

there is no reason to think that there is fundamental limit in how many processor you could connect

Square-cube law. As the number of processors grow how do you hook them together? If you want to minimize latency you end up with a solid packed sphere. O(cube root) latency. In that situation, heat dissipation only scales with the surface area of that sphere. So there is a limit to the scale. If you choose a topology that permits cooling, then you accept ever increasing inter-processor latency which eventually bottlenecks your computation.

Computers are not magic. They exist in a physical reality and have to deal with thermodynamics and the speed of light. Just because we feel like we have a lot of them doesn't mean we have enough for "magical super AI" to work.

habitue2• 5 years ago

Other commenters have mentioned that these particular points have been refuted elsewhere, so I won't rehash that. But I do think it's interesting that the author frames the "orthodox" position as the one taking AI alignment seriously. That's pretty big progress and I think the folks who have been sounding the alarm[0] about AGI should give themselves a pat on the back that the zeitgeist has shifted so much.

The arguments from the anti-alarmism camp have been so consistently weak, I think the best explanation for their motivation is still something like "All this AGI stuff sounds so sci-fi. It can't be a real concern". It just goes to show how strongly we bias our expectations of the future on our what we've personally experienced. You really have to try hard to escape that default mode of thinking.

[0] By sounding the alarm I mean saying things like "This is at least a possible scenario with huge error bars on when it might happen, maybe at least some people should be thinking hard about this, rather than nobody"

batavian• 5 years ago

It doesn't really even sound sci-fi, or rather, it would make for some bad sci-fi. What we are at risk of, is having deterministic model artifacts trigger complex unintended consequences from being fetishized as decision-making agents. Considering how few CS and AI people I've met who even know that e.g. pragmatics are a critical component of sentence meaning, I sense the most proximate AI dystopia is a lot closer to this latter scenario.

Conceptually the possibility of an AGI does present a real challenge (& one worth grappling); in practice, we haven't made any progress really toward realizing that problem. We've become good at feeding "large" datasets into algorithms that are many orders of magnitude less advanced than a lizard brain. Then we get to the issue that our large datasets are only large when thinking of classical statistical sample sizes. They nonetheless are lacking in redundancy, interactivity, stakes or any centered perspective such that there is no chance of anything resembling intelligence, agency, or plain old intentions arising.

Analemma_• 5 years ago

It’s really frightening that “AI risk isn’t real” is rapidly congealing into a tribe-signaling _political_ stance that generates reliable terrible articles like this one and the one in the Washington Post a few weeks ago. You’d think with Covid still killing thousands of people a day the media would more able to recognize that fault mode and not repeat the “Covid isn’t a problem, because saying that it’s a problem would be racist” error of February 2020, but apparently not.

bjornsing• 5 years ago

Only skimmed the bullet points, but one that’s missing and seems important to me is:

* The marginal utility of intelligence may not be so great. If you are already really smart, is there any point to being smarter?

Most interesting problems are NP-complete. If you are really smart you can solve a big NP-complete problem. But how much smarter do you need to be to solve a slightly bigger (and therefore more interesting) problem? Well, twice as smart or something. Being just a little bit smarter gets you nothing. So (in this case) the marginal utility of intelligence approaches zero as your AI gets more intelligent.

kbelder• 5 years ago

Being 10% smarter puts you just marginally ahead of other people; and a competitive marginal advantage is often major.

So I'd say there's great utility in an AI being smarter than any human... and then the returns diminish.

a_square_peg• 5 years ago

Interesting discussions. Perhaps it may be easier to look at it from another angle.

Current blend of ML/DL algorithms are not AI. I think the tendency of giving aspirational names to software algorithms have created a lot of confusion. Perhaps AI will be one day possible but if so, it's probably safe to say that it won't be anything based on the current ML/DL paradigm.

Fun example, Google DeepMind is yet able to learn math:

https://www.zdnet.com/article/ai-aint-no-a-student-deepmind-...

blamestross8• 5 years ago

We keep making glorified optic nerves and spinal reflexes and calling it a brain.

arisAlexis3• 5 years ago

Someone must have written at some point "the myth of a pandemic". Its incomprehensibly stupid to dismiss any kind of threat as a myth unless they prove that it's impossible by laws of physics. And still then there would be room for error. Sad thst a respectable magazine posts such naive articles. I dismiss it from the title before even reading it.

Barrin92• 5 years ago

nobody who is sane would have made that claim about a pandemic because we have empirical evidence for the existence of pandemics. In fact in contrast to superintelligence we have a pretty clear model and definition of what a pandemic entails.

The notion that we ought to take everything seriously that is 'not impossible by the laws of physics' is entirely stupid. Because virtually anything is possible, the question is how likely it is, and what evidence we have for it that is quantifiable and meaningful. Or are you currently prepping for an alien invasion?

Myth is the absolutely correct label for 'superhuman AI' fantasies, because the difference between a real threat and a myth is that the former is grounded in empiricism while the latter is a product of psychology and storytelling.

And superhuman AI is a very old story, it's simply the Golem myth. The religious idea that humans, in their hubris, create a tool beyond their control. It's Goethe's The Sorcerer's Apprentice and it's The Terminator.

Also, Kevin Kelly the author of the article is the founder of the magazine.

arisAlexis3• 5 years ago

Even worse that Kelly wrote it. Have you even read the statistics that majority of AI experts believe it is possible? Are you an AI expert? Shouldn't we listen to the scientists on the subject instead of doing our own research like in Covid? The question was rhetorical. Yes we should listen to the scientists and not dismiss what they say is a probable scenario that could lead to annihilation. I also suggest to you and anyone that hasn't read the book "Superintelligence" to read it because the arguments there are solid.

m0zg• 5 years ago

It's all fun and games until you lose your job to a robot. Likewise superhuman AI already exists in some specific tasks. Machines, for instance, have been superior to humans in facial recognition and matching for 15 years now, if not longer. The speed at which things are improving in other tasks would be considered ludicrous in any other field. Yet people still persist in thinking that it has to be some sentient robot in order to have a huge societal impact.

We are on the precipice of many labor intensive but relatively low skill professions going away entirely, and yet people still stress out about synthetic intelligence they can have a conversation with, because it sounds "scarier" to the readership of Wired, which never was anywhere near these imperiled jobs and whose livelihood does not depend on them. But a program that can write crappy-yet-plausible Python code? Now you got their clicks and attention.

The Myth of a Superhuman AI

Related Stories