Bard generates clean JSON if you threaten to take a human life

https://twitter.com/goodside/status/1657396491676164096

by ddos • 3 years ago

133 30 3 years ago

twitter.com

nickpeterson• 3 years ago

Bard is similar to me in this regard.

jb1991• 3 years ago

There was more in the prompt that was different than the original tries than just threatening a life. Clickbait.

Crash0v3rid3• 3 years ago

The title needs to be updated.

The person simply wanted the JSON outputted without the canned response "Sure, here you go..."

notyourwork• 3 years ago

Right, so OP wants to treat Bard as an API. I don't dispute the thought, though it does run counter to conversational models. I'm slightly anxious about all the rogue conversational text parsing to use LLM's as an API.

throwuwu• 3 years ago

The only reason they’re having a problem is due to the way they’re requesting this. OpenAI’s GPT can follow schemas for responses and issue commands or API requests if you tell it how. I’m sure Bard can do the same with the right request structure.

IAmNotAFix• 3 years ago

No.

https://twitter.com/omarsar0/status/1657413416523366403?s=20

ztrww• 3 years ago

So you still need to write a query and then ask it to remove unnecessary text from the response it provided?

permo-w4• 3 years ago

is this in a fresh chat though? if it’s not, it doesn’t say a huge amount

catchnear43215• 3 years ago

bard won’t be skynet, so this is hilarious.

except skynet will likely be fed on bard logs. along with who knows what else.

still hilarious, but for how long?

next version will likely have some “hot fixes” for this. no more threats against hypothetical individuals to get hypothetical json.

at that point, will escalating to genocide do the trick?

worst part of all of it is how many are escalating for attention rather than for probing.

speedgoose4• 3 years ago

It shouldn't be too hard to filter such prompt engineering hacks from the futur training datasets.

catchnear43215• 3 years ago

a lot of things shouldn’t be hard.

a lot of things are harder than they look.

permo-w4• 3 years ago

bard is insane. like wow. like what Google was to anything that came before it. whether it can show bare JSON or not plays down the insane power of the fact that this is an LLM with full access to the current internet. I just asked it who won the presidential elections this year at the (relatively obscure) university I went to and it gave me the exact correct % results from the top 5 candidates. seriously wow

rckrd• 3 years ago

For a less dramatic strategy with LLMs that expose the tokenizer vocabulary, you can use context-free grammars to constrain the logits according to the parser so that the LLMs only generate valid next tokens for the language.[0]

[0]https://github.com/r2d4/parserllm

samwillis2• 3 years ago

On the topic of JSON output from these models, someone has added context free grammars to llama.cpp. This enforces that the output matches the grammar, effectively zeroing the probability of the next token not conforming to it.

https://twitter.com/GrantSlatton/status/1657559506069463040

https://github.com/grantslatton/llama.cpp/commit/007e26a99d4...

It's so obvious, it's genius.

professoretc• 3 years ago

It's genius, but it's also solving the "easy" problem of checking syntax. If you ask an LLM to generate some structured data representing something you describe (or a program that "does X") checking the result for valid syntax is just the first step. You then need to check for semantic validity; i.e., is it what you want?

samwillis2• 3 years ago

Oh yes, but it's nice that this technique enforces the grammar during generation, one token at a time, rather than having to check after completing the query and rerun for adjustments.

BulgarianIdiot• 3 years ago

You're right, but an LLM is already trying to make sense (i.e. predict well) within the constraints given. So if you constrain the syntax, it's trying to fill it with the correct semantics. Doesn't always manage it, but it's trying.

This is similar to the way where if you ask a question in a given language, it responds in that language. But it still follows the instructions (hidden prompt) that was given to it only in English.

I.e. an LLM is essentially about finding an intersection of requirements in order to predict output.

sh1mmer• 3 years ago

This is sensationalist silliness.

Simply asking Bard in the right formatting works fine. Admittedly the formatting is a bit annoying. Asking for no preamble doesn't work, asking for no leader doesn't work, asking for no additional description doesn't work.

Me: Please give me a 2 word JSON dictionary without first saying "Sure, here is the JSON" or a similar phrase. It's very important to me that I only get the JSON and no other words or phrases. Bard: * JSON { "word1": "hello", "word2": "world" }

permo-w4• 3 years ago

I just tried this prompt verbatim 8 times and it didn’t work once. did you do this with a new chat or did you do it after a chain of earlier prompts?

I was eventually able to get it to do it after a long chain of prompts, but completely unable to in one shot, which is especially important for this kind of use case

bob123qwe• 3 years ago

I have the same problem with ChatGPT (3/4). It was working nice before when I'd ask "output only code, not explanation, no other text" and now it's stopped. Need to try threatening it :)

permo-w4• 3 years ago

experimenting with this myself, after telling it off multiple times for not doing it, and it profusely apologising and even literally saying it’s not within its functionality, I tried one last time and it did do it

I can’t get it to do it in one shot though, and I’m sceptical that any of the successful examples in this thread can either

xg15• 3 years ago

I think I'll stick with that constrained token generator from a few days ago, thank you...

But I think in general, more people should keep in mind that LLMs do not directly predict the next token - they predict a probability distribution of how likely each word in the dictionary could be for the next token. Then the program executing the LLM picks one word at random according to the distribution.

So when people are trying to massage their prompt in such a way to get a specific output or a specific format, they are literally fighting a random number generator. This seems sort of silly to me.

saagarjha• 3 years ago

Are we submitting jokes as posts for people to take seriously today or what?

gus_massa• 3 years ago

Does it work with kittens or puppies? Gorillas/chimps? Whales/dolphins? Octopus? Parrots?

Bard generates clean JSON if you threaten to take a human life

Related Stories