lukebuehler

It’s endlessly fascinating to read the AI transcript of an expert who _really_ knows how to cut to the chase. It just shows how much you can potentially squeeze out of these models. I’m also surprised to see that even Terrence Tao seems to use it in a way that resembles, in progression, how I use llms in my area of expertise (emphasis on progression and usage patterns, not absolute skill, obv I don’t match that): short pointed questions that goes all in on the jargon and machinery of the field and steers the llm hard (eg no softballs). I’ve noticed that llms switch their tone and meet you basically more or less on your level.

I’m thinking along my similar lines. Expansion, if it happens, will likely not be on a recognizably human substrate, but rather something else. But currently it’s more of an intuition than a rigorous argument for me. How do would you formulate a more solid argument around this idea?

A self-hostable Claude Tag or OAI Work [0].

A while ago, I realized that most new agent harnesses being built must be hosted on your machine or on a VM--in other words the agent needs a full OS process at all times.

But we do not have good harnesses being built that are multi-tenant, do not use compute while they are paused, but are are still as powerful as, say, Claude Code or Codex, OpenClaw.

So I set out to build one. I realized that the best substrate for these kind of agents are durable workflow engines. I'm currently supporting Temporal. AFAIK much of OAI agent infra is built on Temporal too. My harness is decidedly not just another agent SDK, but rather a battery-included product.

[0] https://github.com/smartcomputer-ai/lightspeed

Make hay while the sun is out.

Very interesting: I wonder if the RL approach is diverging between Anthropic and OAI?

I noticed that Fable uses shell tools almost exclusively (even to search and edit files), compared to previous Anthropic models.

Having run some experiments with 5.6, I notice that it uses built-in file systems and provider native tools much more (not shell tools), compared to previous OAI models.

yes, basically people that want to host powerful, long-running agent runs not on dedicated VMs (although lightspeed can use those too), but on an abstraction layer above.

My thesis is that the right abstraction is durable workflow engines. And AFAIK, OAI also uses Temporal for their complicated hosted agent infra.

I think these kind of semi-coding agents--but hosted--are the future for enterprises. Claude Tag, Claude Cowork, now Work by OAI.

Agents-on-your-machine clearly have their place, but for many workflows this is too unruly. Hence, the "long-running agent running on shared infra" pattern.

I think this is where the ball is headed. I'm building towards an open source version of this[0]. Still just working on the core, but hopefully soon self-hosted versions can be built on top.

[0] https://github.com/smartcomputer-ai/lightspeed

Oh man, I love capitalism spoiling us here. I was just enjoying my extra Fable credits, now I'll switch to using 5.6 this weekend. I was planning to ration my Anthropic credits, I guess now I do not have to. And I was half wondering if exactly this would happen: right when Fable usage credits were starting to kick in for people, OAI swoops in and takes the puck. As much the AI craze is crazy, this play by play part is pretty fun.

getting downvoted for my other answer. I wasn't clear: yes, there is of course a lot of prior art in pi, and pi specifically does not just store the session events, but adds an abstraction for easy branching which is great.

But what I tried to get at is the question what additional events you store to construct more than just the llm session log but also more fine grained events around the entire agent state, which of course depends on what you want out of your agent.

The paper here in question is going even further and is event sourcing a larger state than just the session transcript, specifically additional graph structures that are getting built as part of the session.

in my agent, specifically, I focus on the event sourcing all the stuff that makes an agent work well as part of a deterministic workflow, which again is prerequisite to run agents in durable workflow engines like Temporal.

I write more about my approach here: https://github.com/smartcomputer-ai/lightspeed/blob/main/doc...

most coding harnesses store the exact messages that have been sent to the llm and the messages items that have come back from the llm, not much else. Then, there is also a set of events/commands/etc that are in-memory only. Together they constitute the current state of the agent loop.

In Lightspeed, we store all of them as events, thus can always reconstruct the exact state of the loop (e.g. state of open tool calls, compaction decisions in-flight, etc). This makes it possible to run the agent in a durable workflow engine easily.

Very cool. I settled on the same/similar design in my agent harness.

All relevant events that affect the context window are stored in an event log. Forking agents and sessions is simply setting a pointer to the sequence number of another event log.

So if you want to check an implementation of this pattern see: https://github.com/smartcomputer-ai/lightspeed

I work with large enterprises that _only_ run critical workloads on locally hosted models. Think banks, insurance, etc--businesses that absolutely cannot leak any data. They also have CC and Codex, but their use is extremely restricted; anything of consequence runs on models running on GPU clusters in their own datacenter.

Agent harness for durable workflows, starting with Temporal.

Most agents for durable workflows feel like toy examples. There is no "Codex" or "Claude Code" for, say, Temporal. So I'm building full-featured agent for these runtimes. Why? Because it makes long-running agents easier to operate and scale. Currently, all frontier harnesses need to run inside a guest OS and need a dedicated process, this is quite challenging to orchestrate and maintain.

To make it work, I had to figure out what part to run as deterministic workflow code, and what part to run as I/O or side effects (aka activities). I'm using a CAS for most of the payloads to maintain a lightweight footprint in the workflow code.

Currently supporting skills, MCP, prompts, a virtual file systems, and soon sandboxes.

https://github.com/smartcomputer-ai/lightspeed

Arguably, if your agent needs a lot of custom logic to drive the agent loop it isn't an "agent" at all. At best it is an agentic workflow, or rather a workflow with some LLMs calls in between.

I think that's why agent SDKs feel like the wrong abstraction. If you are writing a workflow, use a workflow engine (Airflow, Temporal, etc), and call some LLMs with a small LLM library. If you need a "real" agent, use a full-featured agent harness, like Codex or CC or Pi or whatever, then load it up with all the tools, skills, mcps that it needs, and let it rip.

Incidentally I've been building a full featured agent harness that runs inside durable workflow engines [0], but it is designed _not_ as an SDK but rather as a standalone, full-featured harness with an API.

[0] https://github.com/smartcomputer-ai/forge

This is why taking investments too early is usually a mistake. It locks you in to one hand. Of course, you can pivot, but it is much harder with investors looking over your shoulder.

Still one of the best critique of AI agents, even today.

I keep saying this is the single most important article to consider when talking about AI assisted software building. Everyone should read it. The question should always be: is a human building a theory of the software, or is does only AI understand it? If it's the latter, it is certainly slop.

(Second, albeit more theoretical, would be A Critique of Cybernetics by Jonas)

See A Canticle for Leibowitz

It's a tradeoff. Technically, you need very few programs, you can let an agent do everything and coordinate everything. But that is also inefficient, it's slow and uses a lot of tokens. So you allow the agent to build tools and coordinate those tools, just like we humans do. However, with agents, the threshold of pain is much higher, we can let agents do thing's "manually" where humans would build automations much sooner.

Agree. It's code all the way down. The key is to give agents a substrate where they can code up new capabilities and then compose them meaningfully and safely.

Larger composition, though, starts to run into typical software design problems, like dependency graphs, shared state, how to upgrade, etc.

I've been working on this front for over two years now too: https://github.com/smartcomputer-ai/agent-os/

Wall-mounted dashboards are a huge life-hack, especially if you have a family. We got a 37-inch touchscreen one, running DAKBoard.

We have several kids and have been organizing our daily todos and calendars on it for several years. We used to drop the ball quite a bit due to a hectic schedule and the dashboard has helped us tremendously. Since it is mounted in the kitchen, being able to pull up recipes is a plus.

I think sandboxes are useful, but not sufficient. The whole agent runtime has to be designed to carefully manage I/O effects--and capability gate them. I'm working on this here [0]. There are some similarities to my project in what IronClaw is doing and many other sandboxes are doing, but i think we really gotta think bigger and broader to make this work.

[0] https://github.com/smartcomputer-ai/agent-os/

The spec is pretty good! Within a day, Codex has written a good chunk of the attractor stack for me: https://github.com/smartcomputer-ai/forge

I started a full implementation of the attractor spec here: https://github.com/smartcomputer-ai/forge

Thanks! NixOS is great at building and configuring systems, while AgentOS is about running and governing long-lived, deterministic agent worlds. They share ideas like immutability and declarative state, but they operate at different layers. I would say if NixOS is about reproducibly constructing a system, AgentOS is about reproducibly operating one: tracking decisions, effects, and evolution over time.

I’m building AgentOS [1], trying to experiment where agent substrates/sandboxes will head next. It's a deterministic, event-sourced runtime where an “agent world” is replayable from its log, heavy logic runs in sandboxed WASM modules, and every real-world side effect (HTTP, LLM calls, code compilations, etc.) is explicitly capability-gated and recorded as signed receipts. It ensures that upgrades and automations are auditable, reversible, and composable. The fun bit is a small typed control-plane intermediate representation (AIR) that lets the system treat its own schemas/modules/plans/policies as data and evolve via a governed loop (propose > shadow-run > approve > apply), kind of “Lisp machine vibes” but aimed at agents that need reliable self-modification rather than ambient scripts.

[1] https://github.com/smartcomputer-ai/agent-os

Excellent article, and I fully agree.

I came to the same realization a while ago and started building an agent runtime designed to ensure all (I/O) effects are capability bound and validated by policies, while also allowing the agent to modify itself.

https://github.com/smartcomputer-ai/agent-os/

Arguably this is already happening with much human-to-human interactions moving to private groups on Signal, WhatsApp, Telegram, etc.

high hanging fruit!

Eternal Vault is interesting. I would for sure use something like this. However, only if there is a strong story how the vault will survive 20+ years, even if your company is defunct. I do see the pieces scattered around the website (backup to Dropbox, etc), but this story needs to be front and center.