Show me the prompt

by hamelsmu • 2 years ago

142 28 2 years ago

Furthermore, the prompt has a spelling error (Let'w) and also overly focuses on the negative about identifying errors - which makes me skeptical that this prompt has been optimized or tested.

Fixed in https://github.com/langchain-ai/langchain/commit/7c6009b76f0...

Log_out_• 2 years ago

Takeaway: get the llm to populate it's mental space with ideas, then have it evaluate mashups for creativity.. and it's good a framework is doing it..

K0balt• 2 years ago

The whole “prompt engineering “ space has a smell of secret sauce that is ultimately not IP or even particularly clever. Reading the SOTA systems prompts and other prompt optimisations leads me to believe that any business model based in prompt manipulation is ultimately offering a zero marginal value product, not a Zero marginal cost one.

appplication• 2 years ago

Yeah I get the same sense. Prompt engineering is short term need while models haven’t really been productized very well. I would certainly expect to be less of a need in the coming couple of years when LLMs are refined and targeted to specific use cases.

Basically the only reason we need prompt engineering is that current models still aren’t that good (relatively). If they were, they would be able to help themselves with the silly task of prompt engineering without all the superstition.

jandrese• 2 years ago

I am always amazed at these highly engineered prompts where the negative space is full of terms like “grotesque”, “deformed”, “extra toes”, etc…. It feels so much like waving a dead chicken over it. A very superstitious practice.

RugnirViking• 2 years ago

yeah. How many images are in the training dataset labelled with "extra toes" guys? How about "malformed limbs"? Lets at least have some understanding of how these things are trained and work if we're gonna be selling services based on them.

Grotesque might conceivably have some effect. but its pretty clearly cargo culting

MilStdJunkie• 2 years ago

We call those "Snake Oil Mines" in the military industrial complex. They're great! Legendary examples include the Norden Bombsight, MBSE, AS9100 Consultancy[1], and Six Sigma. "Unfalsifiable results" means "you need more".

The beauty is you can make your very own snake oil based on them. All kinds of clever optimizations when no one's sure what the end result is supposed to look like.

EDIT I'm sorry if it seems like I'm picking on "AI" stuff. I'm really not. I'm picking on people buying products without enunciating what it is they're trying to do.

[1] When it's not just straight extortion, i.e., the auditor "selling" consultancy services to make sure you "pass"

JKCalhoun• 2 years ago

I want some kind of genetic algorithm where I can choose between perhaps three images/renders and then have the AI iterate on the one I chose to make three more variations. Perhaps the software varies my prompt for me in clever ways?

Otherwise I have no idea how to take my prompt and a disappointing outcome and massage it.

preommr• 2 years ago

That's how midjourney works (or at least used to). You're presented with 4 images, and you can pick the variation you like best and upscale.

shubber• 2 years ago

I'm extremely amused that one of the trials here was "how do I measure 6 liters of water with a 12 liter jug and a 6 liter jug?" and the article completely glosses over the fact that the framework doesn't help GPT find the simple answer "fill the six liter jug"

jph002• 2 years ago

HN guidelines:

If the title contains a gratuitous number or number + adjective, we'd appreciate it if you'd crop it. E.g. translate "10 Ways To Do X" to "How To Do X," and "14 Amazing Ys" to "Ys." Exception: when the number is meaningful, e.g. "The 5 Platonic Solids." Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize.

The post title is "Fuck You, Show Me The Prompt", but the HN title is "Show Me The Prompt". The removal of "Fuck You, " has significantly changed the title IMO, and is a form of editorialization. Did the HN mods do this, or was it submitted in this form?

mloncode• 2 years ago

Hi Jeremy, Hamel here. I submitted this blog post, and definitely did not submit it in this form. One of the defining features of this blog post was the F word.

I'm also curious about what happened

robgibbons• 2 years ago

As the submitter, you should be able to edit the automated change.

kingforaday• 2 years ago

Since the article discusses interception and rewriting prompts, I find this entertaining. If your statement is true, hopefully others will enjoy the irony as I did.

winwang• 2 years ago

Triple agree. I am also curious about how HN handles (or wants to handle) "profanity"/"taboos".

jph002• 2 years ago

To clarify -- I don't mean this as a criticism. If HN wants to censor swearing, that's fine; but it probably should be mentioned in the guidelines in that case.

dang2• 2 years ago

I changed it because "Fuck you, show me the prompt" is baity. We don't have a problem with profanity (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...) - I'm sure we've restored more "fuck"s in titles over the years than we've removed. We have a problem with linkbait, though, as the guideline you quoted indicates.

Btw, there's another issue. We put the OP in the second-chance pool (explained at https://news.ycombinator.com/item?id=26998308), meaning it would get a guaranteed placement on HN's front page. That's already such a steroid shot that the post can do without a jacked title.

unlog• 2 years ago

imho, just to let you know, I think the original title, more than being baity, it clearly describes a growing sentiment when using the never ending list of AI tools. The prompt is important and not being able to see it, we cant tell if could be improved, and can't test the same prompt on different models. My perception is that the original title is more acurrate. Anyway, thanks!

hersko• 2 years ago

I've become very annoyed with the DALLE api constantly changing my prompts. It drives me nuts.

aaronvg2• 2 years ago

We felt this 100% so we built a DSL (called BAML) to solve the "prompt transparency" problem (amongst other issues). We have a VSCode playground that always shows you the full prompt -- kinda like a markdown preview works.

We are still in beta (and open source) but feel free to check it out! https://docs.boundaryml.com/ .

Some of these frameworks like instructor use like 80% more tokens or only work with OpenAI so we aim to tackle all these problems from the ground up.

Roark66• 2 years ago

Some of these frameworks like instructor use like 80% more tokens or only work with OpenAI so we aim to tackle all these problems from the ground up.

No doubt this is because open source OpenAI API implementations don't support function calling, but now that we have context free grammars in transformers (and TGI) implementation of function calling is not very far off.

Also, about the article, I chuckled, because he listed "guidance". That is (other than an api) a feature of some AI frameworks to implement negative prompting. For example in Transformers. You can send a prompt "what is a tomato", alongside a guidance prompt "you're very verbose" and a (negative) 2 guidance scale. This way the answer will be the opposite of "you're very verbose". This feature is not in the official release yet, but it exists (in git) and it is very neat.

aaronvg2• 2 years ago

Not sure it's related to function calling. GPT4 can do function calling without using the specific function-calling API just by injecting the schema you want into the prompt with directions and asking it to return JSON. It works like >99% of the time. Same with 3.5-turbo.

The problem is these libraries convert pydantic models into json schemas and inject them into the prompt, which uses up like 80% more tokens than just describing the schema using typescript type syntax for example. See https://microsoft.github.io/TypeChat/, where they prompt using typescript type descriptions to get json data from LLMs. It's similar to what we built but with more boilerplate.

ilaksh• 2 years ago

The best way to always know the prompt is to write it yourself. 90% of what those libraries do is just a complex front end to a templating system. So you are often better off just using f-strings or template literals.

epivosism• 2 years ago

I wrote some code to annotate the image with the text of the prompt. I couldn't find a convenient way to do it in EXIF.

For Midjourney: https://github.com/ernop/social-ai/tree/main This one just downloads and annotates all the images in a discord channel/server you have admin in, and can backfill.

For Dalle3 Api: https://github.com/ernop/cmdline-dalle3-csharp This one also submits prompts and does permutations, powersets, block checking etc. Warning: very addictive.

obayesshelton• 2 years ago

You could relate this to using an ORM vs writing your own SQL

Show me the prompt

Related Stories