I resigned from OpenAI

https://twitter.com/kalinowski007/status/2030320074121478618

by mmaia • 5 months ago

233 161 5 months ago

twitter.com

https://xcancel.com/kalinowski007/status/2030320074121478618

structuredPizza• 5 months ago

Autocomplete > Automurk

monkaiju2• 5 months ago

Always surprised when these "smart people" didn't see these things coming from several years away... Its honestly hard for me to believe it.

Going to work for these big SV corps is and always has been directly in service of US empire, that's literally what built the valley in the first place.

conartist6• 5 months ago

Haha that's what I thought, but my thought was that I can't believe Sam Altman didn't see a serious backlash coming when Anthropic rejected a contract saying "the only two things we won't do are mass surveillance and autonomous killer drones" and within 6 hours Sam was all over that.

gre• 5 months ago

The only thing that can stop a bad ai with a guy is a good ai with a gun.

peyton• 5 months ago

I thought they missed the deadline. That’s hardly a rejection. They’re still negotiating, right?

pan69• 5 months ago

It's easier to defer principled decision making to the future while you can rake in the cash in the meantime.

monkaiju2• 5 months ago

Yeah i think this is pretty much it tbh

lostmsu• 5 months ago

To save a click

I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m proud of what we built together.

LeoPanthera• 5 months ago

Their justification rings hollow when they continue to use X.

cozzyd• 5 months ago

Leaving a job is easy. Social media on the other hand...

Jensson• 5 months ago

Quitting a job also have much more effect on the company than switching to a new social media platform, since every company have many times more users than they have employees.

jmull• 5 months ago

Doesn't seem to be an equivalency there.

idlerig• 5 months ago

There isn't, just inserting politics into a discussion on principles.

mkl• 5 months ago

https://xcancel.com/kalinowski007/status/2030320074121478618 to see replies.

wrs• 5 months ago

I have a hard time with this separation of “principle” from “people”. Isn’t it people who have principles?

000ooo000• 5 months ago

Easier to remain in the industry if you are shittalking principles instead of people.

skeeter2020• 5 months ago

yep - it really softens your actions, which in this case seem like a big step. So if you respect the people, why didn't you stay? or if you disagree this strongly with their actions, how can you still respect them?

I get that there's nuance, but this feels like they want to make a big ethical stand without burning any bridges. You can have one of those.

Aurornis4• 5 months ago

“It’s not X, it’s Y” is a common ChatGPT trope used to give a sense of depth to a statement but the specific contrast is generally murky like this. This Tweet was either written by ChatGPT or heavily influenced by ChatGPT style.

ido2• 5 months ago

Seeing as the guy worked at OpenAI maybe it's the other way around?

rvz2• 5 months ago

There are no "principles" in big tech and I call bullshit on this tweet and their reasoning.

OpenAI already had military contracts while this employee was at the company and there was no open letter last year about that.

Prior to that, they were at Meta and joined OpenAI after ChatGPT took off.

If they thought that AGI was about "principles" then not only they were naive, but it leads me to believe that they were only there for the RSUs, just like their time at Meta.

Why is it so hard to be honest and just say you were there for the money, fame and RSUs and not for so called "AGI"?

paulryanrogers2• 5 months ago

Why is it so hard to be honest and just say you were there for the money...

Because then you miss opportunities like this in which to market yourself. A kind of hedging your bets in order to get more money and/or stay out of jail if the winds change. (Jail can be expensive.)

Or it could be honest cognitive dissonance.

culanuchachamim• 5 months ago

It's not that complicated... People have much more depth and sides than one particular idea or principle that they have (specially if you don't know all the context that force them to chose one decsion over another). I'm sure Sam in many ways he's also a great person, so in that case you judging the idea and not the person.

kennywinker10• 5 months ago

Anybody who stays at openai is signing on to build machines that will be used to kill innocent people and control people who think that’s a bad idea.

ed_mercer• 5 months ago

I’m worried that China will build said killing machines and that we’ll be unprepared.

orwin• 5 months ago

Unless you're living in Taiwan, I don't think you have a lot to prepare for.

dbtc• 5 months ago

I'm worried that China will build said killing machines only because they see that we are and feel the need to be prepared.

henry2023• 5 months ago

“I’m afraid my neighbor would kill my son, therefore, I’ll kill my son myself”

t0lo3• 5 months ago

The myth of american moral superiority had been dead for a while. Why would china be any more evil than the US, which has waged far more colonialist wars and killed far more foreign lives in recent times (look at the news today for inspiration)

jatari• 5 months ago

China is currently a more morally virtuous country than the US.

jimmydoe2• 5 months ago

while you are worried about China, USI have done a genocide and started a new war.

suzzer99• 5 months ago

I got scared when I saw China's synchronized drone swarms at the Beijing Olympics, which I believe was the point.

nerfbatplz• 5 months ago

The last time China bombed a foreign country was 1979, 47 years ago. Has the US gone even 47 days in the last 80 years without bombing another country?

bamboozled• 5 months ago

They have to build them because we will build them...

insane_dreamer8• 5 months ago

I reject that argument. I dislike China (the CCP, not the people), and having lived there 6 years I know it much better than most foreigners. But your argument leads to us becoming just like China (CCP). I'd rather hold to some moral values and humanity and be a weaker country, than discard them to be strong.

kennywinker10• 5 months ago

Of course! The only way to fight ai powered killing machines and mass surveilance is to make ai powred killing machines and do mass surveillance. It’s so simple, i don’t know why i didn’t think of it!

But seriously - you are describing the kind of thinking that caused ww1, and the nuclear arms race that almost caused human extinction. It’s a bad idea that goes bad places.

rvz2• 5 months ago

That's fine. But they shouldn't be lecturing to anyone about "principles" or moral superiority and at the same time being either paid or holding RSUs as well, since that would make them completely dishonest themselves.

It just shows that they have done poor research about the company before joining (Meta is just as bad) and are in on the grift (joined OpenAI only after post-ChatGPT) and this employee does not believe what they are saying.

SilverElfin• 5 months ago

Or onto mass surveillance which is a pathway to social credit score style oppression. See this mass surveillance demo:

https://claude.ai/public/artifacts/8f42e48f-1b35-450d-8dda-2...

valleyer• 5 months ago

Nit: ISPs don't have access to path names / query strings of browsing activity. Those are encrypted by TLS. (They're also not part of "DNS".)

nirui3• 5 months ago

A lot of people despite the idea of killing, but as technology advances, and the cost of weapon systems increases, it is less and less likely that these expensive systems will be used to target innocent people, since doing so is likely a waste of resources. On the other hand, usually it is those less-advanced weapons that inflects most mass casualties.

Some country can perform a successful head hunt in the span of an afternoon tea party, while some other country have to level cities for few years and yet still fails to even touch the opposition leader. That's the difference between advanced and less-advanced systems.

If people here loves peace, good. But if we can always reasoning our way out of conflict, then why do we also invented the career of professional police force?

Of course, it is possible that countries advanced too far ahead might bully those less-advanced ones. But then, maybe the less-advanced countries should look inward and reflect on the question why can't they themselves create such advanced weaponries. I don't know, maybe these countries instead of forcing their own people to wear an obeisant smelling face mask, it's time to gave back the power and opportunities so their people can actually grow and gain and eventually contribute.

kennywinker10• 5 months ago

the cost of weapon systems increases, it is less and less likely that these expensive systems will be used to target innocent peopl

Skeptical that’s true. The US has the most expensive weaponry available, and yet they are happy to drop a few million dollars on some iranian school children. It could be true, but i don’t think it is - if nothing else based on the stereotype of the rich kids who totals their parents car.

Some country can perform a successful head hunt in the span of an afternoon tea party, while some other country have to level cities for few years and yet still fails to even touch the opposition leader

Again, skeptical. The US is happy to share its tech with israel, yet they are the ones levelling cities for years with no perceptable impact on leadership.

then why do we also invented the career of professional police force?

Historically? To protect the property of the rich from the people they stole it from.

forcing their own people to wear an obeisant smelling face mask

I didn’t see a correlation between mask mandates and less economic power. China, for instance, had quite severe covid restrictions and yet they are the kind of more-advanced nation you speak of. Most of latin america had virtually no restriction, and they are also “less advanced” wrt ai weapons.

Also, where on earth still has mask restrictions? Find a new grievance, please.

Towaway6910• 5 months ago

Anybody who stays at openai

And paying off their mortgage and feeding their families and has a job in unstable times.

Morals come a distant last in the current state of affairs.

llamaz• 5 months ago

There are levels to morality, from the abstract (e.g. climate change, energy usage, veganism) to the concrete (murder). Time are unstable, but there are multiple ways to make money. If you are established in your career, you can probably find work in a similar field, but the worst case scenario would be to drive a truck.

The way you frame it, you make it sound like an engineer at OpenAI has no choice but to work there or end up on the street. But an engineer at OpenAI is not going to end up driving a truck, they're going to remain and engineer.

kelseyfrog10• 5 months ago

Why have them if they don't mean anything?

Aurornis4• 5 months ago

Someone with OpenAI on their resume (and vested shares) does not have to worry about finding another job, paying the mortgage, or feeding their families.

This is not a relevant point to this discussion.

kennywinker10• 5 months ago

Unstable times. What on earth happened recently to make software engineer an unstable job…?

tdeck3• 5 months ago

I'm sorry but with this justification anything that makes you money can be justified. You can pay the mortgage by robbing a bank too, and that's likely to get fewer people killed.

intended• 5 months ago

Morals come FIRST at Open AI.

Their whole schtick is based on ensuring safety for humanity given the existential risk of a singularity.

Open AI employees MUST get called out, because entire economies and industries are being reshaped due to their statements.

They aren't some mom and pop shop, and they aren't some typical tech firm.

sethammons• 5 months ago

No, principles are even more important in unstable times. Anyone can excuse any behavior otherwise. And everyone at OpenAI has alternatives. This isn't choosing between prostitution or drug slinging to pay for baby formula for them. It is "how early can I retire" - and the answer should be later if it crosses boundaries. The ends do not justify the means.

JeremyNT• 5 months ago

Laughable.

These people are lusting for generational wealth, not scrambling to put bread on the table.

ssss11• 5 months ago

No you need morals. Especially those fortunate enough to be hired by a leading company like OAI - they would be desired by any tech company.

slopinthebag6• 5 months ago

Considering the agreement Anthropic has with Palintar, you could say the exact same about employees working at Anthropic.

Edit: Google, too. Microsoft with its Israel and US Gov ties. Probably most of big tech tbh. How do you recommend we view these employees from an ethical perspective?

kennywinker10• 5 months ago

Are they working on building tech that is being used for weapons or mass surveilance? Like yes Microsoft has contracts with israel, but their entire business is not centered around those contracts. If you help build a better ai for openai, it will be used for war and control. If you help build a better version of one of the 10,000 things microsoft makes, that’s not definitely going to be used for war and control.

Not to get all historical on you, but if you worked for IBM in the 1930s-1940s you may have worked on something that was used to perpitrate a holocaust. Was that ethical? I don’t think so.

That said, it’s very easy to abstract yourself away from the harm. To tell yourself you’re not the one who builds the landmines, you just maintain the coffee machine at the landmine factory. But that’s just lying to yourself. An honest and deep appraisal of what you’re work is helping make happen is required to decide if your job is ethical or not.

andai2• 5 months ago

From what I recall, Microsoft has been deploying GPT in Gaza for years. So even if OpenAI said no, wouldn't the same thing end up happening via Microsoft?

(I understand that domestic and foreign deployment are separate issues — I'd personally object to both — but I'm not sure Microsoft has a reason to take a principled stand on either of those, and they have been working with intelligence for decades.)

deaux2• 5 months ago

Anthropic has willingly left money on the table by taking a stand. They could just not have done this.

OpenAI so far has done the opposite, instead seizing the above as an opportunity.

That is a seriously meaningful difference. Their agreement with Palantir (fwiw OpenAI has been partnering with them for even longer) doesn't erase that.

pembrook3• 5 months ago

Anyone who leaves OpenAI is also signing the death warrant of countless young soldiers by refusing to help build the technology to help remove humans from old school combat.

The current state of affairs of modern warfare is: lots of deaths, lots of collateral damage.

Improving the technology used is more likely to lead to less collateral deaths of innocent people and your own soldiers as well.

There’s already enough weapons to blow the entire world up a thousand times over. Making armies smarter about how they use these deadly weapons is a good thing.

Technologists and intellectuals are notoriously terrible at these sorts of broader societal calculations. They all thought the internet and Social Media would obviously lead to global freedom, which it didn’t.

Now technologists think their new thing, AI coding/spreadsheet bots, will destroy the global economy and kill us all or lead to communist techno-utopia. What if we stop with the moralistic grandstanding and self-aggrandizement and take a deep breath. None of the overpaid pontificators at OpenAI has ever seen real combat, so to make confident claims about what nascent technology will do to it is silly.

This whole thread is going to age like milk.

kennywinker10• 5 months ago

And how exactly does collaborating with the US gov on mass surveillance of citizens help save the lives of young soldiers?

But ok, let’s stick to weapons. The premise that we can wage war without sacrificing lives is a tantalizing one. But do you genuinely think that would prevent death? The drone warfare era under bush and obama shows that killing from afar with no skin in the game doesn’t lead to restraint or lack of war. It just leads to blowing up entire wedding parties.

Fnoord2• 5 months ago

Yes, or subversion from within ¯\_(ツ)_/¯

andai2• 5 months ago

That's the trend we've been seeing for years yeah?

Work at the main AI company. Company has severe ethical issues. Be a person who cares about that. Leave. Surprise, issues get worse.

camillomiller• 5 months ago

If you don’t wanna upset your stomach, don’t make the mistake of reading the replies. What a cesspool of humanity X is.

threethirtytwo4• 5 months ago

That twitter post was clearly written by AI along with the instructions for the AI to avoid "tells" and other tropes common to AI.

Absolutely nothing wrong with something written with AI. Just pointing it out.

bombcar• 5 months ago

But was it written with an OpenAI AI?

threethirtytwo4• 5 months ago

Must've been.

Aeglaecia3• 5 months ago

if that's the case, ai failed to remove the negative parallel construction (my current top ai smell aside from slanted inverted commas). what signs are there of this being ai asked not to sound like ai?

threethirtytwo4• 5 months ago

Right, that's the sign. Ai often fails to do what it's told. So that's the sign of it asked to not sound like an AI. I told AI to do this for my current post as well.

fredoliveira• 5 months ago

and you say this based on?

threethirtytwo4• 5 months ago

The way everyone else can tell. My instincts. AI has a flavor.

add-sub-mul-div• 5 months ago

We're nearing or at an inflection point where people like this are dependent on it.

tadfisher• 5 months ago

There is something wrong with humanity losing the willingness to think and type out a four-sentence paragraph, I would wager.

Generated comments are banned on HN, FWIW.

mrcwinn2• 5 months ago

Good for Caitlin. Sam Altman is awful. He literally admitted on Twitter that they rushed their military contract to get it done. Are you kidding me? You rushed your military contract?

Any employee who stays, especially given the financial cushion they have, is complicit. Shame on all of them.

But here’s the sad truth: most of the knowledge workers at OpenAI won’t be of any value sometime soon because of the very tool they’re building.

sudo_cowsay• 5 months ago

You cant just blame everyone at OpenAI

Everyone has their own unique situation

tdeck3• 5 months ago

Bullshit. What people have are excuses they make for themselves, and they don't need our help.

make32• 5 months ago

they all made 600k-800k USD and up and are highly employable, you definitely can blame them

mrcwinn2• 5 months ago

Whatever happened to this all powerful non profit that would ensure OAI is doing right? Something tells me they just cashed in and run a corrupt shell at this point.

paulryanrogers2• 5 months ago

The board did try. That's why OAI has a new board.

Jordan-117• 5 months ago

It's grimly amusing that a non-profit created to control superintelligent AI for the benefit of humanity couldn't even handle Silicon Valley PR politics.

voganmother42• 5 months ago

Respect for standing up

usr1106• 5 months ago

In Germany it made it even to the general news https://www.spiegel.de/wirtschaft/unternehmen/openai-manager...

So it wouldn't even be worth a HN submission. Well, I think it can still go under exception for exceptional news.

ta9000• 5 months ago

“I don’t think we should spy on Americans and I don’t think we should kill people without human oversight but I still have respect for the guy willing to do that”. Please, make it make sense.

smilebot• 5 months ago

It’s not you, it’s me.

slopinthebag6• 5 months ago

I can't help but to feel like this is an odd moral position to take. OP is apparently fine with building technology to spy on civilians in other countries, and I don't see a moral relevance to citizenship on this matter. If spying on civilians is fundamentally wrong, it doesn't become OK when the people live in a different region of the world. If spying on civilians is fundamentally OK, then why would there be a moral exception for civilians who live inside the geographical region in which the company is legally registered? Perhaps someone can enlighten me here.

The autonomous killing thing is more reasonable, but still, if you're OK building death technology, I'm not exactly sure what difference having a human in the loop makes. It's still death.

insane_dreamer8• 5 months ago

I agree mass surveillance is fundamentally wrong, but it's reasonable for people to feel greater responsibility towards the citizens of your own country, and how they are treated by your government.

slopinthebag6• 5 months ago

Maybe, but I still think it's an odd moral boundary to cross. You might feel as though it's fine to spy on Chinese citizens because of the relationship the US and China have, but what about Canadians or Australians or the Brits or any other NATO country? I get it might feel different, but is that really a hard moral line in the sand you refuse to crosss? Idk.

totally_human2• 5 months ago

Spying on your own citizens enables certain sorts of anti-democratic abuses (and has been used that way in the past), so I can understand the specific opposition to it. Put somewhat melodramatically, they're okay with spying but don't want to create self-coup tools.

I agree that the killbots red line is somewhat odd, but I guess you have to draw the line somewhere, and I prefer them having that principle to having no principle at all. (Also, it's possible that the AI insiders understand something I don't about why a human in the loop is important.)

slopinthebag6• 5 months ago

I would argue that isn't really a moral argument though, it's rather utilitarian. If someone at OpenAI disagreed with that risk assessment, that's a difference of opinion not a reason to quit and write letters talking about ethics.

Also it's a rather American-centric view. If a Canadian is working at OpenAI, should they care? Or would they care more about possible anti-democratic interference by the American government on Canada?

replwoacause• 5 months ago

Good. Proud of her. We need more like her who have principles.

glimshe• 5 months ago

Is "Why I left OpenAI" this decade's version of "Why I left Google"?

make32• 5 months ago

It was up last week, now no one really needs to read the post

I resigned from OpenAI

Related Stories