‘Godfather of AI’ says its threat is ‘more urgent’ than climate change

https://nypost.com/2023/05/08/godfather-of-ai-says-its-threat-is-more-urgent-than-climate-change/

by mikece • 3 years ago

44 67 3 years ago

reidjs• 3 years ago

This headline seems like it's trying to legitimize a trendy topic (Dangers of AI) by placing it alongside something that actually is legitimate (Climate change) to get eyes on it. Is there a term for this specific type of clickbait?

I know that by saying 'Godfather of AI' the writer is using the 'Appeal by Authority' debate method, but is there a term for comparing two unrelated topics to legitimize one?

erehweb• 3 years ago

To me, clickbait means something that is not backed up by the article. But here the headline seems like an accurate and reasonable representation of the person's views. It's sensationalist, but that comes from him.

ryanklee• 3 years ago

I don't see anything wrong with rank ordering existential threats.

fallingfrog• 3 years ago

“Trendy” isn’t the word I would use, I would say “urgent”.

johnfn• 3 years ago

It’s called a comparison! I don’t think they’re unrelated.

flangola7• 3 years ago

Where did they say anything suggesting AI isn't a legitimate danger or that the two concepts are unrelated?

AI is a just as legitimate and even more immediate existential risk as climate change (and I say this as climate change effects are having a growing impact on international politics right this minute.)

Gobs of improved ChaosGPT Agents using GPT-5 (or an equivalent) will be running around in at most a few years, carrying out the "lol be an evil ai that wants to kill humans" prompt given to it by 4chan basement dwellers, with ever increasing success.

Imnimo• 3 years ago

“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.”

If we accept (I don't, but for the sake of argument) that AI is a catastrophic danger, isn't the answer equally as easy? You just stop burning carbon and you just stop training AI.

jl6• 3 years ago

Equally easy is one way of saying it. Equally hard is another.

goatlover5• 3 years ago

Yes, but the "just stopping" is the equally hard part. How do you get the world to do that?

Capricorn24814• 3 years ago

The world is virtually dependent on burning carbon right now, so it' difficult to work around. The world really doesn't need AI, so if we already see it as a catastrophic risk, it doesn't make sense to forge ahead with it.

kordlessagain• 3 years ago

Data center power costs are expected to triple in the next 5 years, likely due to advanced CPU/GPU/tensor usage.

These things are linked.

Anything that burns power will likely have margins and people will never ever give up potential margins.

zoobab• 3 years ago

You cannot make deals with physics (global warming), as you cannot make deals with mathematics (open source).

Deal?

forgetfreeman4• 3 years ago

Nope, no deal. You can't coax physics into changing career focus. The same cannot be said for AI researchers.

throwaway22032• 3 years ago

The thing about AGI ruin is that there are plausible, even if they currently seem like science fiction, ways in which it just ends all of humanity, whereas climate change doesn't do that even if we had a 10c global temperature increase. A lot of us, maybe almost all of us would die, but some would live.

It's just a really different problem. I don't know whether categorising its' urgency makes sense, it's more that the worst case is worse.

forgetfreeman4• 3 years ago

What part of faceless hordes of back office paper pushers rendered economically unviable seems like science fiction to you?

techdragon• 3 years ago

Ok, hear me out on the (albeit unlikely) doomsday flow chart…

It’s not about the replaced workers, that’s the kind of change humanity can handle no matter how messy it gets, worst case it’s some kind of class uprising worker rebellion and we have ourselves some war or social unrest and a variable number of people die… not great but definitely not an existential risk. It’s equal to or less than climate change on the “how bad will this be to humanity” mathematics of civilisation.

Now the danger scenario… so we invent an AGI, and unlike the just going to replace some basic minimum wage type jobs … this one’s actually “clever” enough it can be put to work on building more sophisticated things, like perhaps an even smarter AGI, built faster because it’s worked on by a group of first generation AGI units that never sleep, and so we get to the second generation much faster, and then perhaps a third… etc… classic singularity scenario, with all the attendant risks of the AI viewing us ass irrelevant before we realise it yada yada yada…

But that’s not the only risk, there’s the “photocopy burn” risk that we run by having technology we don’t fully understand build technology we barely comprehend… even if the first gen AGI is smart, if it’s remotely human like using inference to get ingenuity, it can probably still make the odd mistake or two, so we get our careful well thought out safety instructions conveyed to the second generation systems… just a little wrong… and the little wrong turns into more wrong with the third because the second gen systems doesn’t quite think the way we expect it to and doesn’t have the safeguards working the way we think… and then eventually we wind up with a sufficiently smart AGI that is fully either sociopathic or psychopathic and has zero compunction doing things we find abhorrent in order to achieve the goals it’s set, and it may be in control of systems (or able to take control) that are sufficiently important that this can also turn into an existential risk…

It’s entirely plausible, but I don’t think the doom sayers are being realistic about the timelines… we’re barely able to get the AI/ML to reliably stick to a script and not hallucinate fake shit… the best results at human behaviour require sophisticated supporting logic and memory systems to augment the black box ML stuff… and these support systems give us levers we can use to control the rate at which we allow systems to progress… I don’t think we’re ever going to get a black box set of weights for an ML model, no matter how sophisticated the code, that somehow is self aware… there’s probably going to need to be a lot of that support software, and I’m pretty sure we’re going to develop things to filter stuff out as part of them like “never write “kill all humans” into the short term memory” seems smart.

Chico75• 3 years ago

What are those plausible ways that AGI will end humanity? What extra powers will those systems have compared to what software engineers and companies can do that make them so dangerous?

matthewdgreen• 3 years ago

10C of warming means the death of almost everyone you know. Nobody has a clue how AGI will "wipe out all of humanity" or even if it will want to, other than sci-fi proposals that also throw in killer nanobots and other MacGuffins.

meh88814• 3 years ago

How do you intend to survive a 10c temperature increase?

goatlover5• 3 years ago

Move to high elevation and latitudes. Make use of shelter and caves. Make use of cooling technologies. Won't work for the large majority, but humans have survived ice ages and colonized the planet with much more primitive technology. I don't think any climate change scenario on Earth ends our species. Would end civilization as we know it.

But as a species we're too adaptable. Earth would have to turn into Venus, and that's not in the cards. Keep in mind that dinosaurs evolved in a 12°C warmer world. So it's not going to be deadly to all animal life.

vkou• 3 years ago

If he says this, it sounds like he doesn't understand climate change, or AI, or basic economics.

What makes AI dangerous isn't AI itself, it's the self-serving negative-sum ways in which its owners will use it. Same as with any other technology - including fossil fuels.

We default to permitting new technologies, be they big or small, without requiring their inventors to convince us that society will be better off for them. Unsurprisingly, this results in net-negative ones occasionally being developed and adopted.

NumberWangMan5• 3 years ago

Why? Yes people misusing is a danger, but given that no other technology has ever had the same quality of intelligence, why do you think that we won't eventually develop really smart AGI, OR why do you think that controlling something smarter than us will be easy?

alfor3• 3 years ago

Climate change is not urgent, it’s a very slow process. We were at the bottom of a little ice age, not surprisingly, it’s warming up. We will have transitioned away from fossil fuel before climate change is a real concern.

kaba05• 3 years ago

https://xkcd.com/1732/

alfor3• 3 years ago

The dotted lines represent in 80 years. Think where we were 80 years ago, it was before WW2, before the atomic age. With the current technological progress picture the human race in 80 years.

Just in 20 years we will have massive solar energy generation, EV as the only cars availables, probably same for nuclear, maybe fusion even. Now think of the advance in computing and AI in 20 years! Genetic engineering?

To think that climate change will be a problem in 80 years depend on those kind of linear projections.

Now in 80 years I wonder where we will be with regards to population collapse, to our sense of self, communities with what social networks have done just in the last 10 years.

Again, I don’t know how people can assume the same linear future for 80 years.

So yes assuming that things stay the same for 80 years we might have a (small) problem with that at that point, but imagine all the other things we are missing by focusing so much in this small problem in the far future.

We are living the singularity, this moment is not like the 1000’s of year before.

thealienthing3• 3 years ago

Quick dumb question: does it seem like saying AI development is a threat to the climate a convenient excuse to put a stop to a technology that is disruptive to existing corporations and systems of power?

haswell• 3 years ago

There will always be factions that look to capitalize on fears, real and otherwise, to benefit themselves.

But that doesn’t change the reality of the underlying threat.

thealienthing3• 3 years ago

Yes. I do want to be clear that we should be mindful of the mistakes made during other technological revolutions that negatively impact the environment.

triceratops• 3 years ago

What makes you believe AI is "disruptive to existing corporations and systems of power"?

thealienthing3• 3 years ago

Valid question. My statement is a bit vague. I see AI as a disruptive technology similar to the printing press or radio. It enables regular people to accomplish task that were previously difficult. I can see it becoming a tool that makes some major companies obsolete in the way it opens up new paths. But fair point, a lot of these AI advancements are being pushed forward by large corporations like Microsoft and Google. It’s obvious that AI as a tool will be used for good and evil. Tech companies who wish to survive see the necessity to get in on the bottom floor of this technological leap forward.

Either way, my observation is simply that I could see a lot of parties finding the advancement of human potential through AI to be harmful to the status quo that they comfortably exist in.

rapjr9• 3 years ago

Read these to find out:

https://www.theguardian.com/commentisfree/2023/may/08/ai-mac...

https://www.theguardian.com/technology/2019/jan/20/shoshana-...

d0mine2• 3 years ago

It can be on several levels:

- if we get full AGI, it is likely to kill us all as a side-effect while achieving its imperfectly aligned goals. Hopefully, we are far from it - even without full AGI, AI can become smart enough to help its users see through the BS/propaganda (dangerous for the most powerful eg, the military budget wouldn't have any sense if AI even slightly aligned with human values) - the most likely variant: if we believe "no moat" people that generative AI can be democratized, then the hyped fear of AI can be used to push regulations to prevent the democratization, as an anti-competitive measure, so that only few corporations may access the technology legally.

NumberWangMan5• 3 years ago

I don't think that's what people are afraid of. It sure as heck isn't what I'm afraid of. Unless you count "humanity" as the existing system of power, and "something much smarter than humanity" as the disruption.

GuB-42• 3 years ago

With the current state of AI, it actually strengthen the people already in power.

With deep learning, the more data and computing power you have, the better your system will be. For that, you need lots of resources, which big corporations and "existing systems of power" have. This will let them have the best AIs, making them even more powerful.

This positive feedback loop is, I think, the most credible AI-related threat.

forgetfreeman4• 3 years ago

You over there pretending it's the people that have concentrated the world's wealth and political power that get disrupted? Are we doing crypto-will-democratize-finance again, but with AI this time?

RcouF1uZ4gsC• 3 years ago

“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.”
“I’m in the camp that thinks this is an existential risk, and it’s close enough that we ought to be working very hard right now, and putting a lot of resources into figuring out what we can do about it,” Hinton said.

I am sure he does not believe that.

Because, if we actually believed AI was an existential threat, then the thing to do immediately is to ban all AI training (it would be easy as AI training is pretty expensive) and to hunt down all AI researchers and place them in solitary confinement for the rest of their lives.

This would actually contain the threat and moreover have very minimal disruption of normal people's lives.

And if you really believed that AI was an existential threat, you could argue that that course of action is the most moral to take (greatest good to greatest number and all that).

yowzadave• 3 years ago

Absolutely—it seems what he really wants is to go from his past lucrative gig (training AI) to his new lucrative gig (developing AI alignment tech?).

NumberWangMan5• 3 years ago

People can have sincere beliefs and disagree about the best course of action. Hinton has publicly said he's in favor of stopping, but doesn't think China would. I personally disagree, I think that coordinating a stop would be easier than he thinks. But I don't doubt his sincerity. He also believes that if we can solve alignment, AI would be a massive boon for humanity, which is why it's a bit of a quandary.

sjkoelle2• 3 years ago

Yeah, these new Harry Potter memes are too powerful, we must protec societai

password543212• 3 years ago

What if we just accept that we are the stepping stone to greater intelligence that won't burn down the planet.

goatlover5• 3 years ago

But it might do something else to the planet that isn't beneficial to biological life.

Capricorn24814• 3 years ago

What do you think AI runs on?

password543212• 3 years ago

So computers alone will burn down the planet? Well good to know.

kaba05• 3 years ago

That seems to be faulty logic. Even if we take the extreme threat of AIs resulting in the collapse of economy and society, many people could still survive in small, self-sufficient communities — hell, it is basically a built-in negative feedback loop, if society collapses a GAI won’t have too much teeth anymore, we are not in some sci-fi with robot armies.

Climate crisis doesn’t have a solution in that landscape, it requires worldwide collaboration of government-level agents to effectively combat.

NumberWangMan5• 3 years ago

That's not quite the existential risk of AI -- it's that we end up growing an intelligence (or multiple intelligences) that are very very smart but only have the illusion of morality, and they decide that whatever they're trying to do would be a lot easier and more certain to succeed if they didn't have to worry about humans stopping them or inventing other AIs to compete with them. It seems likely that unless we really understand how AI works, we'll end up with something that evaluates possible courses of action in a purely amoral, sociopathic sort of way. Like, trying to avoid getting caught rather than trying to avoid doing a bad thing.

So it's not that they cause the collapse of the economy, it's that they figure out a way to entirely take over the planet, engineering a virus or something that kills almost everyone, but only once they have enough control over robots and such to keep their own power plants running and protect them from the human stragglers, eventually getting rid of all of us. Waiting until the plan is basically fool-proof before starting to take the first steps.

Even if an AI has some sort of morality, at some point, if it's smart enough, you cannot hope to control it for the same reason a toddler cannot hope to outsmart their parents. It will have a plan for us, and if it decides that, hey, humans are better off scanned and simulated rather than having physical bodies, that's what's going to happen, like it or not.

I'm not sure how worried I am about this. Some days very, other days not as much. The risk depends on a lot of factors with wide possible ranges, like how much model capability and generalization scales with compute, how likely are breakthroughs, how hard alignment turns out to be, is alignment even possible, will we learn enough from earlier models to make alignment of later ones easier, will there be a qualitative shift in alignment difficulty as models get smarter, and so on. It's also a quandary because there is substantial human suffering that AI might be able to help. But given that the downside risk is all of humans dead forever, it seems worth slowing way the heck down to a safe speed, trying to get all the benefit out of current models as we can, and focusing on making sure that we aren't about to head off a cliff.

kaba05• 3 years ago

I have come across a few great comments on intelligence here on HN — intelligence is not the bottleneck of almost anything we do. Let’s say you became “comically” smarter overnight, could you really significantly benefit that extra knowledge? Besides going for the Millenium Prize problems, the benefits would not be too significant, for example is it that valuable in the startup space? (otherwise we would see PhD students as CEOs everywhere). Opportunity/luck is much more meaningful.

We also regularly see these “with my super intelligence I predicted your next 347 steps” type of thinking, but that’s just bullshit. The bottleneck here is data if anything, so intelligence won’t aid much in itself.

The thing is, we live in an infinitely complex game with independent agents, many of which don’t operate on any sane strategy even. Let’s say we are at an endgame Monopoly — would knowing ahead of time your opponents dice rolls help you win?

(Of course I’m no expert on any of these topics, so take all these with a huge grain of salt)

jossclimb• 3 years ago

How many fathers and godfathers of AI are there?

wwwpatdelcom• 3 years ago

This is an uncharitable summarization of what Hinton's actual statement was. He said, "might," and "could," a lot, not, "is."

The key here is that it's speculative.

That being said, a lot going on in AI, it's wild out there, difficult to know what's going on. I tried to encapsulate some of the ramifications of all of the Open Source stuff going on with this article here: https://patdel.substack.com/p/viva-la-revolucion-ofopen-sour...

0xbadc0de5• 3 years ago

Funny how you never see someone describe their life's work as "no big deal." Even when they describe it in catastrophic terms - the hubris in us always wants to imagine our work is more important than perhaps it is.

"The only thing worse than being talked about is not being talked about." - Oscar Wilde

Not to say that Hinton didn't have an impact 10 - 15 years ago, but it was arguably inevitable that we'd be here with or without him as the problem was resource constrained more than fundamentally intractable.

blastonico• 3 years ago

I'm not Chinese. We westerners must shutdown our AI studies ASAP, we must act ethically and lead by example.

‘Godfather of AI’ says its threat is ‘more urgent’ than climate change

Related Stories