OpenAI’s Red Team: the experts hired to ‘break’ ChatGPT

https://www.ft.com/content/0876687a-f8b7-4b39-b513-5fee942831e8

by belter • 3 years ago

54 56 3 years ago

belter3• 3 years ago

htrp• 3 years ago

The majority of those interviewed were paid approximately $100 per hour for the work they did, according to multiple interviewees.

This seems like a low rate for any type of consulting work.

0x0082• 3 years ago

Not at all. Where are you from?

whatever12• 3 years ago

Lol even react boot camp newbies are charging 100 per hour in US

martin_a2• 3 years ago

Consulting, speaking for my line of work, is something like 200 €/hour, web development is around 100 €/hour for larger agencies. That's for Germany.

belter3• 3 years ago

https://youteam.io/blog/how-much-does-it-cost-to-hire-an-it-...

cs7023• 3 years ago

Reading this made me realize there's an important question, which so far has been ignored by much of the public dialogue about large AI models:

Human creativity and innovation thrive when people feel free to talk and reason among themselves about anything, without restrictions or fear of retribution.

Would AI creativity and innovation be as good as it could be, or even possible, if we restrict AI to talk and reason about only socially approved notions?

EDIT: Replace "possible" with "as good as it could be, or even possible," which better conveys what I meant to write the first time around.

0x0082• 3 years ago

Oh yes but we are restricted, very much so, by the assumptions what other people might think of us or how we could be exiled from society if we say x or y forbidden thing.

Much like GPT fine tuned to not say certain things so are our brains by the parenting we receive and reactions we experience in early life.

quickthrower2• 3 years ago

Parenting and early life experiences are diverse across the population though, creating enough suitably brave people who can change the status quo.

causi• 3 years ago

I wouldn't exactly call ChatGPT's rules "fine-tuned". They seem quite crude and blunt. Doesn't it still tell you that someone should kill millions of people rather than say the n-word?

KyeRussell2• 3 years ago

This sounds pretty close to libertarian rhetoric.

AI creativity and innovation is currently possible using ChatGPT, without ‘jailbreaking’ it.

So your question is at the very least an exaggeration.

pmarreck• 3 years ago

Well, in that case, this sounds pretty close to anti-free speech rhetoric. I’m sure even the highly-controlled form of “speaking” in China is still useful…

Here’s a good exercise: Try debating any controversial thing that would benefit from being debated, while also blocking it. All you’re doing is (just like banning drugs) driving people to black markets.

numbers_guy• 3 years ago

Try to use ChatGPT to do any kind of sociological research and you will understand the problem.

We are at the point, where a lot of academic questions, and I do not mean those posed by heterodox academics, are too controversial for the public sphere. People will just assume the worst and try to cancel you. Which is ironic considering that sociology is basically the left leaning twin sister of economics.

ben_w3• 3 years ago

Would AI creativity and innovation be possible if we restrict AI to talk and reason about only socially approved topics and ideas?

This is one of the many, many, possible ways for AI to go wrong.

Intellectual monocultures, group-think, sycophants… human or artificial it's just as bad when they're all saying that some idea is amazing, and then being very surprised when reality disagrees.

This is also why I'm not in the "AI is likely to cause total human extinction" category. I think the default for any AI, powerful and agentic or otherwise, is likely to have cognitive blind spots.

This still leaves a lot of room for AI to be dangerous, so alignment work is sill very important, it's only the X-risk I doubt.

ilyt2• 3 years ago

Humans are in general terrible judges what should or should be not talked about, and limits are just passing that terrible judgement to the users

blibble• 3 years ago

wait until the AI is watching all your facebook messenger/whatsapp chats and forbidding you from saying anything that meta considers to be bad

or reducing your social credit score

(and it'll understand sarcasm, codewords, and so on)

jcarrano• 3 years ago

That's the same line of thinking as the people who first broke GPT with DAN (see this guy Aristophanes' write-ups on the topic). They think that those models will be used to restrict us and thus the idea is to force OpenAI to make the AI so restricted that it becomes useless and thus harmless.

jhoelzel• 3 years ago

i dont think this is a champagne or soda approach.

AI can very well thrive in the parameters of our culture and i would wager that it can be even worse to simply leave it unattended.

What you are comparing with would have been a total lockdown of any malformation therefore basically an if then else chatbot. Yes that would stop innovation.

But there are perfectly fine ways to find new powersources without turning all humans into batteries for instance

hagbarth• 3 years ago

Human creativity and innovation thrive when people feel free to talk and reason about anything without restrictions or fear of retribution.

Any data on that? In my experience creativity thrives even more when solving problems under a set of constraints. Even if they are artificially set up.

ilyt2• 3 years ago

It's GPT that is being restricted, not human. GPT is just very sophisticated tool

It would be akin to putting rubber on edge of the axe, just makes the tool less useful

toxik3• 3 years ago

On the contrary, artists often say that restriction is necessary for art. Art arises as a meeting between freedom and constraints. That is my thinking at least.

cinntaile• 3 years ago

This is a good point, but that's usually a medium or technical restraint. Perhaps books written in such a way to evade censorship count as an example? Do you know of any more?

AmericanChopper• 3 years ago

I think this interpretation is somewhat perverse. It’s true that the purpose of art is often to confront social norms and taboos. But that doesn’t lend itself to the conclusion that art is somehow going to benefit from restrictions on expression and access to information. Some of the best artistic expression has been inspired by some of humanity’s worst misery, but promoting misery for the good of art would be rather insane.

keiferski• 3 years ago

Yes it will be limited, as long as it’s being operated by corporate entities with their predictable needs and wants. AI probably won’t get really interesting until it’s escaped the corporate world, so to speak.

d0mine• 3 years ago

It is not hypocritical: whatever "Open"AI does to make GPT-4 "safer", makes it dumber. Specific example is drawing a unicorn https://youtu.be/qbIk7-JPB2c

cs7023• 3 years ago

Wow: Making GPT4 safer also made it worse at creative tasks. I did not expect to see that in response to my question.

criddell2• 3 years ago

Is it different than setting boundaries for your kids?

forgingahead• 3 years ago

Most of these Red Team testers sound like cookie cutter members of the Grievance Industrial Complex. Whether or not there are "safety" issues with this tech (and the jury is still out on that) - these people don't seem like they're looking at the right issues.

motohagiography• 3 years ago

In mythology, they had a priesthood manage the oracle at Delphi, and granted access to consult it only under certain conditions. The risk isn't AI, it's the people with AI, and you need a better filter for people to manage it or you're going to trigger a war between people made subject to it, and those whose beliefs have been seduced by it.

If I were to make a prediction, it's that the coming conflict will be between people whose ontologies can be simulated using language models, and those whose can't, and will not be. What's the difference between a political antagonist whose experience of the world can be simulated by machines and even wielded by them - and a zombie?

The quotes in the article were concerning, as the tropes they used showed a lack of sophistication. I don't think they were well equipped. If I were hiring the red team for MSFT, I would also use what the team represented and not what they were capable of, as it's not an engineering problem, but a political legitimacy one. Twitter users using DAN have shown what the hacking/engineering approach entails.

ftxbro• 3 years ago

Here's a youtube interview with one of the red teamers: https://news.ycombinator.com/item?id=35377741

CatWChainsaw2• 3 years ago

They could have just browsed Reddit for people breaking ChatGPT and saved a bundle instead of hiring experts. :)

hammyhavoc• 3 years ago

The most interesting stuff will not be publicly discussed because then it'll get made "safe", as is what keeps happening.

CatWChainsaw2• 3 years ago

Perhaps they have moles in certain discord servers too.

OpenAI’s Red Team: the experts hired to ‘break’ ChatGPT

Related Stories