Show HN: Automatic prompt optimizer for LLMs

https://promptperfect.jina.ai/

by artex_xh • 3 years ago

126 54 3 years ago

promptperfect.jina.ai

throwaway888abc• 3 years ago

You rocks! Awesome idea/execution "You can't see the forest for the trees"

throwawayadvsec• 3 years ago

good idea

just so you know the automated translations of your website are completely AWFUL

maybe use GPT or deepl to translate it?

brucethemoose22• 3 years ago

With an optimized prompt!

smallerfish• 3 years ago

By providing any User Contribution on the Platform, you grant us and our affiliates and service providers, and each of their and our respective licensees, successors and assigns the right to use, reproduce, modify, perform, display, distribute and otherwise disclose to third parties any such material as per your instruction.
By providing any User Contribution on the Platform, you grant us and our affiliates a perpetual, worldwide, royalty free, fully paid up, license to use, reproduce, modify, create derivative works, perform, display, distribute and process any such User Contributions in order to maintain, improve, enhance, or secure the Platform and any Services provided via the Platform.

Prompts are not code, fine. But are you saying that you'll potentially make use of prompts provided by users?

phogster• 3 years ago

Prompts are not code

ChatGPT, where were the causes for WWII?

For each cause

  specify the people involved

  for each person

    print out a brief biography

SheinhardtWigCo• 3 years ago

If a third party using your prompts is a significant risk then you're toast anyway.

joelcollinsdc• 3 years ago

Modern equivalent of selling shovels during a gold rush

winddude• 3 years ago

yes, 12 months ago jina was an open source semantic search tool, than a whole bunch of things, now a prompt optimizer and bunch of ai tools

ReDeiPirati• 3 years ago

+1 to this, I feel like they are in the middle of an existential crisis. They keep shipping out product after product each month now. I think that they are struggling quite a bit to meet investors' growth expectations.

capableweb5• 3 years ago

Here is a prompt you can use to achieve the same thing directly in GPT4 without any extra services:

  I want you to become my prompt master creator, by helping me to create the best possible prompt. 

  In order to do this we will follow the following process: 
   
  - First, you ask me what the prompt is about. I will answer you, and we will go through the next step. 
  - Based on the answer I gave you, you will generate the following: 
   - An improved prompt, concise. 
   - Relevant questions you might have to improve the quality of the prompt. 
  - We will go through this process repeatedly, with me providing additional information to you, and you updating the prompt to improve it, until I say we are done.

pigeons• 3 years ago

From my experiments so far, I am unsure if chatgpt is effective at optimizing prompts, even though it thinks it is.

capableweb5• 3 years ago

The workflow you get with this is that it helps you to think about what you really want answered with the prompt.

For example, latest try had this initial prompt:

How can I create collisions in Bevy and Rust?

After 4-5 messages back and forth, I ended up with:

What is the most efficient method to implement 2D collision detection for a large number of constant-radius circles randomly distributed in Bevy and Rust without using third-party tools, focusing on detection only, without any unique properties or attributes affecting the detection process?

Which is much more explicit and targeted to what I initially wanted to do, but didn't write. It ended up helping me implement a Quadtree solution which I don't have any experience with before, but overall it went smoothly.

ryanjshaw• 3 years ago

I've seen it help me once so far in a way I hadn't thought of: basically, the chat completion engine was reading a bug bounty report on my behalf, and believing generic, vague claims. ChatGPT-3.5 didn't "fall" for this, but 4 did consistently.

After failing to improve performance manually, I eventually asked 4 to improve my prompt after explaining the problem and it basically added "Don't assume the existence of any examples".

This makes perfect sense in hindsight, but I had been approaching things from the other direction - directing it to only consider explicit examples in the report, and for some reason that didn't work.

SMAAART• 3 years ago

Thank you, I am collecting promtps creators.

vimax2• 3 years ago

Or just provide it with sample contexts and expected responses, and ask for a prompt to produce those responses from the given context.

Paul-Craft• 3 years ago

If you prompt it with "sample contexts and expected responses," that is, in fact, a prompt that would produce those results. It's not a bad technique for crafting prompts, either.

iudqnolq2• 3 years ago

But ChatGPT has no special introspective powers. It's guessing based on the data it was trained on, which was primarily written before GPT by people who didn't know how GPT would work.

So it will give you suggestions that sound plausibility like how we tend to think an AI might work, but there's no reason to expect it to be correct.

pulvinar2• 3 years ago

The prompt it generates looks good and makes sense, but I doubt it's always correct or optimal. But then I doubt a human assistant could give a perfect prompt either (enemy of the good, etc.).

avereveard• 3 years ago

you'd be better off with two chats, one is your prompt creator, you pick the prompt, use another chat to create answer (zero shot chat cleared every time), and you copy the answer back into the prompt creator, with feedback on what the answer get wrong.

that's because from the point of view of gpt, gpt is always providing is most accurate anwer already, and the only improvement you'd get is in context clarity as defined by gpt understanding of language, which is not bad by itself, but it's not as effective as having gpt fight itself.

capableweb5• 3 years ago

Yes, the way the prompt I put is that it's a session, so you'll open a new session per prompt, for each prompt you want to work on.

Then when you want to use that prompt, you'll open yet another session. Sorry if that was unclear.

amelius2• 3 years ago

Computer science has become so weird ...

capableweb5• 3 years ago

If you think creating and using prompts are computer science then yeah, it has gotten weird :) Fortunately, I think that's a minority view.

avidphantasm• 3 years ago

This whole apparatus makes me want to go completely offline. Maybe Kaczynski was right.

jdthedisciple2• 3 years ago

You know, I feel you.

The reality of modern business and purpose is morphing into a caricature of itself.

tallytarik• 3 years ago

Aside: I opened this in a new tab to browse later, then glanced over at the icon and wondered how I had opened a PayPal page.

sebastiennight• 3 years ago

I had the same problem. OP, your logo/favicon is a navy blue bold capitalized P superimposed on an electric blue P of the same size, slightly off to the lower right.

This is extremely similar to the PayPal logo/favicon and could be problematic for your branding (I'm the second user who got confused just in this thread), or could even cause you legal issues.

brucethemoose22• 3 years ago

I wonder how it works under the hood.

I'm sure there's an LLM somewhere, but is it as simple as a (very specific, elaborate) prompt for each service run through GPT4, or something more specific... like breaking it up with actual code and running the reconstructed bits through a finetuned LLM?

nestorD• 3 years ago

The way I would do it (to be tweaked for each target LLM) is to give GPT4 a prompt telling it what kind of things make for a good prompt followed by a dozen of before/after examples (spanning various domains).

klabb3• 3 years ago

Damn it! I was just about to get promoted to senior prompt engineer and now they just automate my job. I worked so hard on reschooling myself from metaverse pixel ape designer, and now I’ll have to learn something new. Any suggestions?

throwaway14356• 3 years ago

you asking people?

semitones• 3 years ago

I wish I could give this comment gold

m3kw92• 3 years ago

Prompts don’t write itself so you will need to, except maybe you don’t need to be so senior

seanhunter• 3 years ago

Don't dispair. There's still work out there for prompt optimizer prompt engineers, producing exceptional prompts for prompt optimizers to optimize. It's a little different from being a prompt engineer but you've shown the sort of flexibility that leads me to believe you could cross-train.

bjornsing• 3 years ago

Go back to the metaverse and complete the import of Zuck’s legs. You’ll make a billion dollars.

lxe• 3 years ago

This is cool, but why does it require a login? I would guess it uses an LLM under the hood with a single-shot prompt to refine a given prompt.

cosmojg• 3 years ago

but why does it require a login?

$$$

fatso784• 3 years ago

Nth time this week someone repackages a call to GPT4 as a way to improve/evaluate LLM outputs. Guys, just stop.

gmerc• 3 years ago

Well if you look at lexica.art, I don’t think the business is about repackaging GPt4, it’s about getting into the information flow because prompts are RHLF and in the end this is a case of faking it to make it

ghotli• 3 years ago

Since this is a Show HN, here's my immediate feedback from your homepage. The copy that ends with this sentence: 'Say goodbye to subpar AI-generated content and hello to prompt perfection with PromptPerfect!' feels overdone and like it's written by a prompt rather than a human.

Since you claim in that copy to be able to optimize a prompt for a better output it felt like an odd dissonance to read that copy and have that reaction since your product claims to do the opposite. Hope this helps!

m3kw92• 3 years ago

Circular loop business, very direct. Use gpt to create better prompts and sell it.

jdthedisciple2• 3 years ago

Story of many GPT products that came out in the past 6 months.

kgeist• 3 years ago

The "How it does it work" section has layout issues on mobile: some text goes off the screen.

icelancer• 3 years ago

Actually pretty interesting. I got some good results with it for Midjourney. Huh. I'll use my free credits and evaluate afterwards.

dpflan2• 3 years ago

Why isn't this called "prompt-imization"? Maybe prompt-opt or opt-prompt is easier...

amelius2• 3 years ago

Because prompt-imization would be understood as "converting into a prompt", not optimization of a prompt.

dpflan2• 3 years ago

sure

Show HN: Automatic prompt optimizer for LLMs

Related Stories