Show HN: Roboco-op, a computational blackboard for efficient human/AI collab

https://observablehq.com/@tomlarkworthy/robocoop

by tlarkworthy • 3 years ago

82 27 3 years ago

Once you start editing AI responses you realize how limited and imprecise our UX so far has been. It's like a parent correcting a child during development. It's a huge help in getting it solving hard tasks. You can quickly transfer what you know and then let it generate from there. Huge win.

protokultur• 3 years ago

Stay out of trouble!

all24• 3 years ago

Dead or alive, you're coming with me!

all24• 3 years ago

I guess I'm confused as to what I'm seeing. It appears to an LLM in a Jupiter notebook-esque environment. And that you can modify the LLMs output?

Is there more to it than that?

tlarkworthy8• 3 years ago

You ask a question. It outputs a cell. The cell contains the the answer and the prompt. There is no state, unlike the webapp. When constructing an API call, we use the materialized cells to reconstruct the context. Because of the notebook format, it's easy to edit everything. Because the LLM is speaking code, it's easy to check everything. You can feed the cell value outputs back into the prompt.

Cells can depend on each other, so you can build up. Cells can be prose, so you can plan. You can change the plan halfway. You can edit the AIs plan. You can forget the plan and do something else without affecting what happened after. You can keep the dialogue and context tight and information dense.

When generating a response, the LLM considers it's previous responses, which you can edit. So you get very fine control over its chain-of-thought. No need to use system prompt to train.

It's simple, but the emergent properties enable much more powerful collaboration.

all24• 3 years ago

This makes sense and tells me what the benefits are in an explicit manner. Thank you.

tudorw3• 3 years ago

First impression, oh my, er, what is this, deeper reflection, wow, so I can sort of program this like a smart spreadsheet where cells can contain LLM results which can feed into LLM prompts, and as I alter the contents of a cell, or LLM provides new results, the whole thing is 'recalculated', did I get that about right? Really looking forward to seeing how the UI develops in this space, so many ideas to explore.

d4rkp4ttern• 3 years ago

Intriguing but a simpler example would probably have gotten the idea across much more effectively

happytiger• 3 years ago

That is pretty brilliant. I’m just trying to get my head around the power of chaining all this together.

supermatt2• 3 years ago

Unrelated in all but name, but for those who enjoy the robocop movies and are gamers, the new game (delta city) feels very true to the movies. It's not really a challenge, and the gameplay is pretty linear, but it's an enjoyable ~12hrs if you want to be a bullet sponge cleaning up the criminal slime.

lukeholder• 3 years ago

Rouge city or delta? Can’t find delta.

supermatt2• 3 years ago

rogue city sorry.

runnedrun• 3 years ago

Have you experimented with sending back the evaluated result in some way? How often does it give back broken code?

tlarkworthy8• 3 years ago

The block 'Cell Values to Include' let's you forward values selectively. The v0 had auto feedback but it wasn't robust, quite risky on credits, and not that useful in most cases, so now it's a multi select per prompt opt in.

Havoc2• 3 years ago

This could have done with a slightly gentler and clearer introduction.

Goes from something was trained by ai to markdown to visualisations to react stuff.

AI and notebooks sounds like it could be a powerful concept - just can't tell what's going on here in this article.

Which parts of this is AI generated and what's the user input. maybe I'm looking at llm output? Though the cells seem to contain input and output combined with && somehow. Maybe that's the llm echoing back the input? Is the ai generating the cells or interpreting them? Both?

tlarkworthy8• 3 years ago

I agree but I found people bounce with big intros, so it's a bad attempt at "show don't tell."

The cells were written by AI, but the metadata data that generated the cell is mixed in (thats the {...} && bit).

So now the full chat context can be derived from the notebook. Because the notebook is editable, you can fully edit the chat context in an ergonomic way. Both the prompt, the AI code output. So when the AI makes a mistake (it often does), you can correct it. Your ability to detect mistakes is accelerated by the fact it's a computation based conversation. Because the chat context is derived from the editable state, your corrections are internalised into the chat conversation. It continues as if it answered that for real.

So in the notebook you see that the AI is rapidly become an Observable notebook expert, because I have manually coarse corrected its chain-of-thought. An out-of-the box chatGPT session cannot do what is in that notebook. My expertise have been transferred. Once it knows how to solve something in the chat context, its very easy for it to repeat the pattern for new situations. Having it as a standalone forkable self-enclosed notebook artifact is useful. The existing notebook infra allows you to copy and paste cells across notebook too. So I am thinking we can curate stand alone skill memes. The examples are all over the shop just to see what kinda of things I could get it to solve, I imagine a real use would concentrate on upskilling just enough to solve a specific class of problem.

Havoc2• 3 years ago

Because the chat context is derived from the editable state, your corrections are internalised into the chat conversation.

Ah right. That makes more sense to me. Thanks for explaining

lmeyerov• 3 years ago

I very much believe in this direction both as a practicing data scientist, and working daily with folks who kinda know SQL or Python but are really more experts in other stuff, like security, k8s, supply chain, etc.

For a slightly different take focused more on that kind of user, eg, connected to their operational databases like Splunk, elastic, neo4j/Neptune, & databricks, and want the choice to use directly or embedded in their existing tools & automations, we are getting ready for more cohorts on louie.ai cloud, and would welcome folks curious for their team. Early enterprise & gov teams on self-hosted louie.ai face the same thing every day.. it really changes how I feel about programming.

While a lot more to do, half a year of working with LLM-augmented querying & coding already feels like a weight has been lifted from my shoulders that's been there for decades yet I never realized things can be different. Ex: "Try that last cell again, except this time trim the IP columns to the first 2 octals, and then visualize how servers connect together in a GPU graph network viz." I can in theory code the sql/spl, pandas, and python viz for that... and figure out the precise syntax/lib/code tweaks to get there... Or just write that high-level edit as if I was pairing with any reasonable person!

darkteflon2• 3 years ago

This looks fantastic. Is this something we can try for ourselves, or it relies on a private library?

tlarkworthy8• 3 years ago

its public observablehq notebook under MIT license. You can fork it and off you go! You have to add your OPEN_API_KEY, I am not paying for your fun, lol! But yeah, the code is all public. The burger menu on the left is where you can open/close cells for editing.

darkteflon2• 3 years ago

Thanks! Honestly, this is unbelievably cool. It has that rare quality of making you go: “oh, I must immediately adopt this obviously superior paradigm”. Bravo!

Show HN: Roboco-op, a computational blackboard for efficient human/AI collab

Related Stories