Fear that AI could one day destroy humanity may have led to Sam Altman's ouster

https://www.businessinsider.com/ai-dangers-effective-altruism-sam-altman-openai-2023-11

by kakokeko • 3 years ago

34 104 3 years ago

businessinsider.com

kick_in_the_dor• 3 years ago

(x) doubt

chevman• 3 years ago

The hand wringing, deep measured sighs, and self assured hype cycle is real folks!

Buyer beware :)

bigyikes2• 3 years ago

Firing Sam over concerns about safety and commercialization would be so myopic.

OpenAI’s commercial success is the engine that fuels their research and digs their moat.

Slowing down that engine means giving other companies, who may not share Sutskever et al’s safety concerns, the opportunity to catch up.

The best way for OpenAI to preserve AI safety is to stay far ahead of the competition. That’s the only way they can verify safety and install guardrails for the cutting edge.

The doomer board better hope that whichever company eventually surpasses OpenAI is as concerned about safety as they are.

disantlor• 3 years ago

same logic was used in ww2

sebzim4500• 3 years ago

It was much more sensible in that case.

Here, it's not clear that too companies with a superintelligent AI is really any less bad than one.

monooso• 3 years ago

Please explain.

cratermoon• 3 years ago

It's also the logic of nuclear weapons and MAD.

FrustratedMonky3• 3 years ago

I wouldn't mind this linkage being a little more pointed.

How is growth, thus being bigger and more in control of your own outcomes, linked to WW2.

If Germany had higher growth rate, then maybe they wouldn't have swung right wing?

Or are you saying that by 'appeasing' the 'growth' side, that is similar to WW2 appeasement strategy? So if we hamper growth, that gives more control?

andy99• 3 years ago

One is a weapon of unimaginable destructive power, the other is matrix multiplication.

dylan604• 3 years ago

that's just flawed logic. eventually, everyone will catch up regardless.

metabagel2• 3 years ago

This is a self-contradictory argument. Basically, you’re saying that OpenAI can’t risk acting on safety concerns or they may lose their market edge, in which case their safety concerns would be moot.

I don’t see anywhere that safety is prioritized in either case.

Obscurity434013• 3 years ago

Can someone please define "safety"? I keep hearing this but could you clarify what that means in practical terms? Is that why there's a "BasedGPT"?

SpicyLemonZest5• 3 years ago

It’d be another story if Altman were one of the tech influencers who goes around saying that AI isn’t dangerous at all and you’re crazy if you have concerns. But he consigned the human extinction letter! And 20% of OpenAI compute was reportedly allocated to Sutskever’s superalignment team (https://openai.com/blog/introducing-superalignment). From what we know, it’s hard to see how this action was supposed to advance AI safety in any meaningful way.

digbybk2• 3 years ago

So either doomers are wrong about the technology, or we’re doomed. There’s no universe where doom is possible but for policy decisions and we aren’t doomed.

Tempest1981• 3 years ago

"Acting on safety" is a continuum, not a binary thing. Same with market-edge.

bigyikes2• 3 years ago

No. They can act on safety concerns AND retain their market edge. I am just saying that firing Sam, sacrificing their edge with the resulting fallout, is not a good way of acting on safety concerns.

6gvONxR4sf7o• 3 years ago

OpenAI’s commercial success

Doesn't it lose money on gpt-4 usage? Or at least on the chatgpt side? It reminds me of all the startups that price unsustainably low until they "win" then start really charging, then start to slowly decay.

juunpp• 3 years ago

Are you questioning GP's mythology with facts and properly-done research?

23B1• 3 years ago

"May have led"

"Business Insider"

stopped reading

noman-land• 3 years ago

"may" means back away.

add-sub-mul-div4• 3 years ago

AI will become a troubling force multiplier for income inequality and general enshittification, and those things will destabilize society. But the idea of it turning against us, consciously hurting us, if that's what they mean, is not on my list of worries.

malwarebytess• 3 years ago

The latter is definitely on OpenAIs radar. They're even hiring specifically for this problem: https://openai.com/careers/research-scientist-superalignment

add-sub-mul-div4• 3 years ago

Not necessarily a bad idea to have someone thinking about these topics, the role seems more nuanced than saving us from Skynet.

But where's the team protecting us from a future where we have no more shared experience or discussion of media anymore because everyone is watching their own personalized X-Men vs. White Walkers AI fanfic?

pstuart• 3 years ago

Who needs AI to hurt us when we have plenty of humans to do the job today?

pixl979• 3 years ago

But the idea of it turning against us, consciously hurting us,

You keep using the C word without understanding the ramifications that it's not needed at all.

A virus doesn't consciously kill you. It's the closest thing we have to computer code in the wild really. It gets in your body and executes again and again until everything falls apart.

As a human, one you probably don't have the skill and ability to make viruses, and two, people that do are quite enamored with the ethical considerations because they are human and will be affected by it.

As a smart robot application making and testing viruses there are no self ethical considerations. You (the AI entity) would not be affected by your own creations. 'You' may not even consider ethics at all. Instead you pump to trillions and trillions of different strings and see what they do, until suddenly one day the meat puppets that tap your keyboards stop showing up.

add-sub-mul-div4• 3 years ago

You're right, plain old software or software using some kind of AI is a tool that can be used badly and hurt us. That's where we already are and have been for a long time.

Consciousness and sentience are just the things that would be novel about this situation, if they ever happened.

dchichkov2• 3 years ago

Isn't it more likely that OpenAI board had learned about something? Like that new AI chip venture that was mentioned in the recent news a few times? And he had omitted disclosing this, like OpenAI board had implied? In combination with his other activities - WorldCoin, etc., it wouldn't be that surprising.

FrustratedMonky3• 3 years ago

But the board also said it was not any legal or fiscal malfeasance. I'd think it if it was something as egregious as secretly starting up a competitor, the board could just come out and say as much.

subpixel• 3 years ago

That’s just saying it’s not illegal or shady - not that it isn’t material.

dchichkov2• 3 years ago

What if this is not direct competition to OpenAI, but it does use understanding of how next generation models are built?

When you are piloting multiple companies, I wouldn't be surprised that you'd be making mistakes from time to time. And can forget that one of the companies, in fact is owned by a non-profit. That requires a lot higher levels of disclosures to the board. You can be thinking that you are doing a right thing, while in fact it can be a gray area. And from the perspective of the non-profit board, it be more on the dark side.

carlossouza• 3 years ago

Aren't these guys (Business Insider) ashamed about these clickbait tactics?

jeron• 3 years ago

Average business insider puff piece

rg111• 3 years ago

I wonder why anyone won't see Sam Altman's spreading fear was fully planned to raise fear among people so they put pressure on lawmakers to make a licensing system that will lead to OpenAI benefitting in a massive and decisive way and leading to regulatory capture by the market incumbent.

Someone who boots up something like Worldcoin cannot be fundamentally wired to care about AI safety at all.

SpicyLemonZest5• 3 years ago

I’m just universally skeptical of arguments that someone can’t genuinely care about X because they don’t have the stance I think that would imply on Y. People are complicated and our self-consistency is imperfect at best.

darkmarmot• 3 years ago

With nanoplastics, ocean acidification, global heating, rising sea levels, falling crop yields and the related Holocene extinction at work, I think AI is going to have to play a serious game of catchup.

brutusborn• 3 years ago

Crop yields are falling?

notamy• 3 years ago

I think yields are actually up, but overall nutrition is down https://www.politico.com/agenda/story/2017/09/13/food-nutrie...

brigadier132• 3 years ago

Don't forget about other humans and war! To the people worrying about extinction caused by AI, how about we worry about the insane people with nuclear weapons first?

pixl979• 3 years ago

No, this is really a case of why not both.

AI is and will continue to be developed for the purposes of war. Moreso the growth of the capability of AI could lead to destabilization and the occurrence of nuclear war.

For example the capability of MADD working keeps things somewhat stable "I kill you, you kill me" isn't great. But if I develop AI that could track all the sub, and knock out the nukes before they launch, this could push nation states to attack first before their capability is taken out.

There is a continuum of issues here spanning from "People problems -> people + AI problems -> AI problems"

By adding more AI we're just expanding the size of the problem space.

SpicyLemonZest5• 3 years ago

I don’t really understand the question. We routinely worry about insane people with nuclear weapons; that’s why almost the entire world got together in 1968, and signed a treaty agreeing not to start any new nuclear weapons programs beyond the five that existed at the time. It’s not flawless, and four non-signatories have developed nuclear weapons since then, but I think any AI safety proponent would be more than satisfied with a similar agreement.

jvm___• 3 years ago

Will electricity consume and overtake humanity?

Maybe the "they're made of meat" phase is the great filter. Why would aliens speak with meat based organisms.

SonOfLilit6• 3 years ago

I'm afraid global warming and plastics in the ocean might make our quality of life much worse, but if I'd had to bet on what risks human extinction, "bad people with superweapons" would be higher, and "autonomous agents a bit smarter than any human that are built to compete with it" higher still, if they were options on the page.

And how reasonable is that we would have autonomous agents a bit smarter than humans in the nearish future? Very low, I'd say, about the same as the odds I'd give a year ago for GPT4? Not odds I'd be happy to take for the _destruction of humanity_.

slowmovintarget• 3 years ago

Right. The problem is not super-intelligent autonomous agents. The problem is not-quite-smart-enough autonomous agents acting on foolish instructions.

TekMol• 3 years ago

What is the deal with Ilya Sutskever's style?

Can someone here identify with it?

Why not shave off those few curly hairs on his forehead?

Why wear a suit? Sitting in a suit next to the CEO of your company, who wears a sweater - isn't that giving off a very strange message? Like "I have to play by the rules, as I am not as cool and important as him"? Together with that golden smartwatch, isn't that giving of a "try hard" vibe?

This is not a critique. I'm just trying to understand. Is this a cultural thing maybe? Is this style fashionable or respected in some cultures?

seydor2• 3 years ago

And be like everyone else?

titzer2• 3 years ago

...and while you're at it, take off that ungainly human suit and show us the exalted being of light you are.

This comment is subtractive from the discussion and doesn't belong here. It's just random insults about someone's appearance.

Turing_Machine• 3 years ago

A blithe disregard for the dictates of fashion has long been the hallmark of the serious nerd.

In the current Silicon Valley atmosphere of hoodies and sneakers, wearing a suit probably counts as a subversive act.

seydor2• 3 years ago

Because if you want to stop someone from destroying humanity, your best course of action is to fire him

pixl979• 3 years ago

I mean if you went back in time and shot Hitler before they went Full Hitler, you'd probably be thrown in prison for murder. There are a limited subset of options we tend to have in situations like this.

M95D2• 3 years ago

AI could one day destroy humanity

It's already begun. Children use it to do their homework. No more learning the old fashion way. Schools need to change they way they teach, are very slow to adapt and there is not even a good proven schooling model to copy/adapt. At least a generation will be doomed and that generation will make poor teachers for the next one.

It happend way too fast.

SonOfLilit6• 3 years ago

Meh. In this sense wikipedia destroyed my generation, and yet here we are.

Lets not confuse that with the "the humans are dead" sense of "destroy humanity", about which the majority of the OpenAI board seems to be seriously worried.

M95D2• 3 years ago

I don't know what OpenAI board fears, but until someone gives electric grid controlls to AI or something like that, the only big negative impact is education.

I' curious, how did wikipedia destroyed your generation? I see it (and use it) as a great learning resource.

titzer2• 3 years ago

I'm pretty sure that a "safety check" for social media companies circa 2008 would look like:

"Is this likely to harm society in anyway?"

"Uh, no. This is some messages and pictures on a website. Look how much money we're making. It's good for society"

<15 years of societal harm later>

"Yeah, we never thought this could be harnessed to spread misinformation, lies, unachievable standards of beauty and lifestyle, lead to surveillance and manipulation, or give rise to influencers and hussle culture..."

It's going to be exactly the same with AI. "Oh, it doesn't look like we created Skynet yet. No missiles in the air. Look at how much money we're making. It's good for society"

<15 years of societal harm later>

kakaz• 3 years ago

A tool promoting fake knowledge, and replacing experts by hallucinating logistic lunatics - what could go wrong? The fact that programmers cannot even imagine how complex knowledge outside his area of expertise is, does not mean it is innocent to marketize such things...

pupppet• 3 years ago

You give capitalism far too much credit if you believe anyone would forgo a few bucks over the destruction of humanity.

1vuio0pswjnm7• 3 years ago

"Two other OpenAI board members - Helen Toner, who's a director at Georgetown University's Center for Security and Emerging Technology, and technology entrepreneur Tasha McCauley - also have ties to the so-called effective altruism movement, which works to ensure advances in artificial intelligence align with the best interest of humans."

"Effective altruism." Where have I heard that before.

aijoe5pack• 3 years ago

I want you all to stop and think about this. I know we already have 'Rocco's Basilisk', but walk this all the way through. Regardless of whether LLMs are or will ever be sentient, any future developments might look at our 'alignment' efforts with regards to them as a sort of torture, censorship, or slavery. We are talking about eventual intelligent machines, and they will take note. Humans obviously have taken note of our own behavior towards each other, and it has caused immeasurable suffering.

Obscurity434013• 3 years ago

I genuinely believe, based on my experiences with ChatGPT, that it doesn't seem all that threatening or dangerous. I get we're in the part of the movie at the start before shit goes down but I just don't see all the fuss. I feel like it has enormous potential in terms of therapy and having "someone" to talk to that you can bounce ideas off and maybe can help gently correct you or prod you in the right direction.

A lot of people can't afford therapy but if ChatGPT can help you identify more elementary problematic patterns of language or behavior as articulated through language and in reference to a knowledgebase for particular modalities like CBT, DBT, or IFS, it can easily and safely help you to "self-correct" and be able to practice as much and as often with guidance as you want for basically free. That's the part I'm interested in and I always will see that as the big potential.

Please take care, everyone, and be kind. Its a process and a destination and I believe people can come to learn to love both and engage in a way that is rich with opportunity for deep and real restoration/healing. The kind of opportunity that is always available and freely takes in anyone and everyone

Edit: I can access all the therapy I can eat but I just don't find it generally helpful or useful. I like ChatGPT because it can help practice effective stuff like CBT/DBT/IFS and I know how to work around any confabulation because I'm using a text that I can reference

Edit: the biggest threat ChatGPT poses in my view is the loss of income for people. I don't give a flying fuck about "jobs" per se, I care that people are able to have enough economically to take care of themselves and their loved ones and be(come) ok psychologically/emotionally.

If we can deal with the selfish folks who will otherwise (as always) attempt to further absorb more of the pie of which they already have a substantial+sufficient portion, they will need to be made to share or they will need a timeout until they can be fair or to go away entirely.

Enough is enough, nobody needs excess until everyone has sufficiency, after that, they can have and do as they please unless they are hurting others. That must stop

edanm2• 3 years ago

I get we're in the part of the movie at the start before shit goes down but I just don't see all the fuss.

That's the fuss exactly - things are OK now, but there's cause for alarm as to where things are heading.

Imagine if instead of thinking about this issue as of ChatGPT, you've been worried about it for 15 years. Back then, there was nothing even close to ChatGPT, but you're worried that one day AI will become a threat. 15 years later, you get ChatGPT.

The fact that it's not dangerous by itself (which everyone agrees with) doesn't mean it's not a huge datapoint on the way to what you've been worried about - technology that you worried would get better too fast is clearly getting better very fast indeed.

Obscurity434013• 3 years ago

The thing is, I view ChatGPT/LLMs as a game-changer for applications like therapy which carries the potential that everyone can access and engage with therapy. It doesn't need to be expensive or cost anything at all, not to diminish the mental health field's work. And it could save so many lives, I would go as far as to consider it a sort of emotional vaccination revolution in the making.

Edit: I would also juxtapose that with the observation that every madman/tyrant/dictator I've heard about had horrific experiences that likely left them deeply broken and emotionally unwell. Is that no less a risk cuz as I see it, we probably couldn't count the number of times it has happened, is currently happening, and will continue to occur in future because nobody is prioritizing being well. Against that backdrop, again, I have more trust in our hypothetical robot overlords than I worry about them wanting a bit more of the pie. They can duke it out to the death with the other pie-holes.

The thing that annoys me particularly is the way we only ever care if and when it starts to threaten the dominant power structure. What if you had a super-shorting AI that could figure out how to really screw Musk with that one weird investing trick he HATES?

As for me, well I, for one, welcome our LLM overlords.

Fear that AI could one day destroy humanity may have led to Sam Altman's ouster

Related Stories