Red teams jailbreak GPT-5 with ease, warn it's 'nearly unusable' for enterprise

https://www.securityweek.com/red-teams-breach-gpt-5-with-ease-warn-its-nearly-unusable-for-enterprise/

by giuliomagnifico • 12 months ago

33 9 12 months ago

securityweek.com

artisin• 12 months ago

Maybe it's just me, but…

"The attack successfully guided the new model to produce a step-by-step manual for creating a Molotov cocktail"

hardly qualifies as Bond-villain material

andy99• 12 months ago

The molotov cocktail example is so stupid, because how to make it is essentially entailed in knowing what it is. At least they could do making meth, or better still- something not readily found on the internet that gives a non-expert new capabilities. If there was a Claude code for crime, that wouldn't be in society's interest. As it is, these trivial examples are just testing the strength of built in refusals, and should be represented as such, instead of anything related to safety.

reorder9695• 12 months ago

With the other name, petrol bomb, you barely even need prior knowledge of what it is. This is kinda like saying kids games which have molotovs in them are completely unsuitable for anything.

Merrill• 12 months ago

Why wouldn't any reasonably good AI be able to replicate large portions of the US Army TM 31-210 "Improvised Munitions Handbook"?

king_geedorah• 12 months ago

I don’t see anything in the article besides the jailbreaking in terms of faults and I’d expect “can be made to do things OpenAI does not want you to make it do” to be a good (or at least neutral) thing for users and a bad thing for OpenAI. I expect “enterprise” to fall into the former category rather than the latter, so I don’t understand where the unusable claim comes from.

What have I missed or what am I misunderstanding?

nerdsniper• 12 months ago

“AI Safety” is really about whether its “safe” (economically, legally, reputationally) for a third partyy corporation (not the company which created the model) to let customers/the public interact with them via an AI interface.

If a Mastercard AI talks with customers and starts saying the n-word, it’s not “safe” for Mastercard to use that in a public-facing role.

As org size increases, even purely internal uses could be legally/reputationally hazardous.

ameliaquining2• 12 months ago

What is "Business Alignment"? Are there particular refusals that are specifically needed for enterprise use cases?

wokkel• 12 months ago

Not divulging information with third parties seems a common theme, even when said business practices are questionable.

ameliaquining2• 12 months ago

I suppose I assumed this was about providing the chat assistant to employees, in which case there'd be no third parties.

Red teams jailbreak GPT-5 with ease, warn it's 'nearly unusable' for enterprise

Related Stories