andywalters

Wouldn’t it be a good thing if most world leaders regularly consulted a sufficiently intelligent and grounded model? I would love to see real research on this but I strongly suspect it would universally improve decision quality. If that’s right it’s reasonable to imagine we will enshrine AI into our executive, legislative, and judicial processes when their benefits (impartiality, objectivity, holism, etc.) become clear.

Et voila! A sui generis political actor is born.

This is one of the best papers I've read in a long time. It encapsulates so many of my thoughts on the open internet of AI agents for business that I think most people are sleeping on.

We're at the dawn of a major shift in the internet and there are so many tasty problems to be solved, and so much we need to do to protect and grow this thriving ecosystem.

Honestly, this is a call to arms for us -- we're launching an open source agent framework that's specifically designed to facilitate open, interoperable, A2A compliant agents out of the box.

Thanks! Not currently providing an export but i am working on baking in a redis vector db of all emails to allow for RAG.

out of sheer frustration with the constant deluge of crap in my inboxes i finally broke down and threw an llm at my account. clearmail allows you to: - filter all incoming emails by simple rules you write in english - auto-categorize to any labels you want - never deletes emails, just re-categorizes - runs entirely locally or with openai

feedback welcome

Hi, author here, I do think it's a fair criticism to say the mechanism could be more clearly demonstrated if it formed and evaluated hypotheses clearly outside the domain of any likely RLHF / fine tunes / system prompt text.

I'll work on that.

I don't think the comparison to the image example holds up; while something like a random-ish behavior is at play when it comes to hypothesis generation, the hypothesis evaluation is a different matter. It does seem to reason about its own capabilities intelligently, at least insofar as one can detect coherent reasoning as represented in language.

The program you mention using an array might be a thin version of what I've demonstrated here, but it operates on much the same principle.

Great question about parallels. I suppose the closest biological parallel would be a human brain reasoning about an external agent's character based on behaviors just observed. (But it quickly falls apart past that: unlike human neuroplasticity, because GPT-4 has no mechanism to update its own weights, knowledge about the agent being evaluated is lost at each new inference unless explicitly included in-context.)

Hi I'm Andy, author of the post and WHOA this blew up randomly! Some quick thoughts:

- I certainly agree the model is not updating itself in the sense that its weights remain unchanged. But if a program can evaluate its own behaviors and build a self model from it, even if that model lives only in-context in the form of linguistic representations -- still, isn't that a _form_ of self-awareness? And with the larger context windows available now but not when I wrote the post, it will be interesting to see what's possible.

- To the charge that the program merely maps the shape of the RLHF / fine tuning / system prompt text -- other than in the most trivial sense, I don't actually think this bears out. As you watch the program describe in language its successes or failures testing the generated a hypothesis about itself, it's very hard to argue it's not doing something very much like reasoning about those questions, rather than regurgitating its training data. Of course, there are those who stipulate a priori that these models can't reason, so they'll likely not be convinced. By my lights, insofar as these things can ever reason in-context, the model is in fact reasoning about itself.

- In terms of application, I don't know that there's any direct application to "AGI" but the idea of building a self-model in-context could be extended to building a model in-context of, say, a human with which the program interacts. This could lead to more realistic interactions in say, gaming or customer service, where interactions with an agent include its own assessment of your behavior towards them. In short, the post's outline for "getting to know itself" could become the basis for "getting to know someone else".

- More tangentially, the idea of recursively developing hypotheses and evaluating them could extend to analyzing any dynamical system, like an API or even a static knowledge domain.

- Another possibility would be instead of making updates to an in-context prompt, perhaps it makes updates to a knowledge graph or vector store and these are pulled via RAG when relevant.

Overall I'm overjoyed to learn the piece has sparked some great conversation and hopefully some further ideas or applications! If you'll allow me the tiniest bit of horn-tooting, I do run an AI development shop putting my best thinking on prompt and AI engineering into working code for SaaS companies. The software we've written has now powered over 1M AI interactions and we're always interested in talking to great potential clients :D

Cheers!

I think it's a little more complex than that and I did my best to lay out in detail why in the article. Here's a video I made of it working, you can see how it works and judge for yourself whether it's just pretending. https://youtu.be/430CvR2OgOQ

With some clever prompting, GPT-4 autonomously learns about itself by generating hypotheses which it tests. The self-inquiry process is different every time it runs but it can more or less un-prompted spawn creative works like poetry about the beauty of nature, philosophical inquiry such as a debate about the merits of protectionism, and even explaining its self awareness to a hypothetical researcher.

Heh yeah we haven't baked in much self-knowledge about Lexii.ai yet, working on it!

Founder at https://Lexii.ai here—we’re a GPT-3 based search engine that pulls in real-time legacy search results and cites sources. Would love feedback, we’re just in alpha but we’ve already got ~50,000 searches per month.