GPT-5 API injects hidden instructions

https://twitter.com/xundecidability/status/1956347084870651960

by irthomasthomas • 11 months ago

1 4 11 months ago

openai giving it instructions before me?

Uhhh, yes. It's in the devblogs. They call it prompt adherence hierarchy or something, where system instructions (oAI) > dev instructions (you) > user requests. They've been training this way specifically, and test for it in their "safety" analysis. Same for their -oss versions, so tinkerers might look there for a tinker friendly environment where they could probably observe the same kinds of behaviour.

irthomasthomas2• 11 months ago

Please can you link me to the documentation on this.

NitpickLawyer2• 11 months ago

Yeah, it's in the "gpt5 system card" as they call it now [1]. Page 9 has the details about system > dev > user.

1 - https://cdn.openai.com/pdf/8124a3ce-ab78-4f06-96eb-49ea29ffb...

GPT-5 API injects hidden instructions

Related Stories