zknill

Yeah, with only minimal guidance it's what you get out of claude. The colours and layout are pretty 'default'.

How do you know?

There's a lazy habit from some folks to say something they either disagree with or don't understand was "written by AI" without backing up that statement.

Isn't the point that you no longer have a connection to the client?

So you can be notified by the database, but you can't (with the stateless HTTP + loadbalancer design explained in the article) get that notification back to the client. Because the client isn't connected anymore; so how does the client know that there's new information?

The industry decided a long time ago that sticky sessions was a terrible idea. They only half-solve the problem, while suffering from session loss on server loss and imbalanced load over time.

And once those long running jobs have reported their status back to the database, how will the client find out about that status?

Please, please, please don't say "polling". Because you've clearly missed the entire argument of the article if you say polling.

would be nice to see an example linked

Folks will probably say, 'I just use cmux, or tmux, or tabs, or warp'. But I do appreciate it when the original project makes an attempt to solve the problem.

Cloudflare Sessions API and Anthropic Routines have a really similar model. Where they are hosting the 'session store' for you, and giving you access to it over long-polling (or sometimes websockets).

It's a bit harder to do agent presence ('is the agent still there') with this model without heartbeats, but possible.

It's good to see the industry starting to address the "durable sessions" problem, because it sucks.

This is way too complex!

100% - the argument of the article is that building any feature beyond chat-based-demos on HTTP SSE streaming is super complex. But a lot of folks still want to do it, because that's what their tech stack is. I think it's still a valuable thing to be talking about how you might do that.

AI generated code, where the author doesn't understand the code, shifts the burden of checking quality and function onto the reviewer.

This post says little about that, and suggests some improvements the _reviewer_ can make.

I think that's completely the wrong end of the stick to be tackling. As the burden is still on the reviewer and not the author.

I wrote about "All your agents are going async"[1], and everyone said "Can't you just do this with SSE". So I figured I'd dig into that claim.

[1] https://news.ycombinator.com/item?id=47832720

This is great, I built a manual integration based on JMAP and CalDav cli tooling, but this is neat. Especially:

The OAuth consent screen will give you a choice of three levels of access: read-only (see emails, contacts, calendars), write (update emails, save drafts, edit contacts and events), and send (send emails).

I suspect the answer is that the AI chat-app is built so that the LLM response tokens are sent straight into the HTTP response as a SSE stream, without being stored (in their intermediate state) in a database. BUT the 'full' response _is_ stored in the database once the LLM stream is complete, just not the intermediate tokens.

If you look at the gifs of the Claude UI in this post[1], you can see how the HTTP response is broken on page refresh, but some time later the full response is available again because it's now being served 'in full' from the database.

[1] https://zknill.io/posts/chatbots-worst-enemy-is-page-refresh...

"and which ones are no longer relevant."

This is absolutely the hardest bit.

I guess the short-cut is to include all the chat conversation history, and then if the history contains "do X" followed by "no actually do Y instead", then the LLM can figure that out. But isn't it fairly tricky for the agent harness to figure that out, to work out relevancy, and to work out what context to keep? Perhaps this is why the industry defaults to concatenating messages into a conversation stream?

Assuming LROs are "Long running operations", then you kick off some work with an API request, and get some ID back. Then you poll some endpoint for that ID until the operation is "done". This can work, but when you try and build in token-streaming to this model, you end up having to thread every token through a database (which can work), and increasing the latency experienced by the user as you poll for more tokens/completion status.

Obviously polling works, it's used in lots of systems. But I guess I am arguing that we can do better than polling, both in terms of user experience, and the complexity of what you have to build to make it work.

If your long running operations just have a single simple output, then polling for them might be a great solution. But streaming LLM responses (by nature of being made up of lots of individual tokens) makes the polling design a bit more gross than it really needs to be. Which is where the idea of 'sessions' comes in.

I don't know Kitaru too well, but I do know Temporal a bit.

The pattern I describe in the article of 'channels' works really well for one of the hardest bits of using a durable execution tool like Temporal. If your workflow step is long running, or async, it's often hard to 'signal' the result of the step out to some frontend client. But using channels or sessions like in the article it becomes super easy because you can write the result to the channel and it's sent in realtime to the subscribed client. No HTTP polling for results, or anything like that.

With the approach based on pub/sub channels, this is possible to do if you know the name of the session (i.e. know the name of the channel).

Of course the hard bit then is; how does the client know there's new information from the agent, or a new session?

Generally we'd recommend having a separate kind of 'notification' or 'control' pub/sub channel that clients always subscribe to to be notified of new 'sessions'. Then they can subscribe to the new session based purely on knowing the session name.

I don't think this is quite right. I do work for a pub/sub company that's involved in this space, but this article isn't a commercial sales pitch and we do have a product that exists.

The article is about how agents are getting more and more async features, because that's what makes them useful and interesting. And how the standard HTTP based SSE streaming of response tokens is hard to make work when agents are async.

This is actually great for *claws. When Anthropic changed their T&Cs to disallow using claude code oauth tokens in the Anthropic Agent SDK, you had a choice between violate the terms or pay a lot more for the model inference using an API key from platform.claude.com instead of claude.ai.

With this change, it looks like an officially sanctioned version of *claws. Connecting to whatever "channels" you want via MCP.

Architecturally it's a little different, most *claws would call the Agent SDK from some orchestrator, but with claude channels the claude code binary starts the MCP server used to communicate with the channel. So it's a full inversion of control where Claude code is the driver, instead of your orchestrator code.

I updated my nanoclaw fork to start the claude code binary in a docker container on PID 1, and you can read the docker logs straight from claude code stdout, but with comms directly to/from your channel of choice. It's pretty neat.

I've been using Kagi for ~18months and your description doesn't match my experience at all.

Querying for something like "snowflake json from variant?" in both engines and in google I get a sort-of-right-but-not-really-that-helpful ai summary about "parse_json" function. In Kagi I get an actually useful summary with code examples of parse_json, but also the colon-based syntax for accessing values inside nested objects without needing to parse anything.

I very rarely need to go into a page, I use Kagi quick search summary with the "?" suffix and it almost always gives me a useful answer in one-shot.

You don't need to be committing the prompts you're using. There's a whole bunch of back and forth in the prompts as you refine. That's not useful information.

What you should do, is use the context window that you've got from writing the code and refine that into a commit message using a skill.

https://zknill.io/posts/commit-message-intent/

Why an MCP? dbos already ships a cli that appears to have the same features. Why an MCP over a skill that gives context on using the cli?

https://docs.dbos.dev/python/reference/cli

Yeah for sure, the skill leans on the agent being able to identify the intent. I'd say roughly 30% of the time I have to either prompt it to fix the intent, or it will ask me explicitly what the intent is.

Without the skill, the commit messages are often a list of files changed with a list of bullet points at the end saying:

- Added 8 tests to file.xyz

Pretty useless for a reviewer, they can see that from the diff.

I also thought this was going to be about folks who'd migrated off AWS ECS.

Assuming the traditional stateless routing of requests, say round robin from load balancers; how do you make sure the returning UI client ends up on the same backend server replica that's hosting the conversation?

Or is it that all your tokens go through a DB anyway?

It's fairly easy to keep an agent alive when a client goes away. It's a lot harder to attach the client back to that agents output when the client returns, without stuffing every token though the database.

There's two parts to this article. The scheduler/preemption, and the transport over the network. The article is absolutely right that long-lived request/response over HTTP connections with SSE streamed responses suck.

The article touches very briefly on Phoenix LiveView and Websockets. I wrote about why chatbots hate page refresh[1], and it's not solved by just swapping to Websockets. By far the best mechanism is pub/sub, especially when you can get multi-user/multi-device, conversation hand-off, re-connection, history resumes, and token compaction basically for free from the transport.

1: https://zknill.io/posts/chatbots-worst-enemy-is-page-refresh...

presumably a dupe of: https://news.ycombinator.com/item?id=47046068

It seems like this post is a response to the discussion on this previous HN post[0].

There are two main criticisms of "Event [sourcing|driven]" patterns. First, that storing a full history of events becomes a backwards-compatible schema support hellscape nightmare after 12-18months. Second, that the patterns encourage eventual consistency in areas that could really benefit from strong consistency.

This post does a good job of explaining differences between Event Sourcing and Event driven to address problem 1. But does nothing to help problem 2.

Even the event-driven example in the post uses a shopping application where multiple "OrderPlaced" events could be produced without the necessary inventory to fulfill those orders.

I'm just not convinced that most people need this kind of complexity, when a simple database with strong consistency would get a lot of folks a lot of the way there.

[0] https://news.ycombinator.com/item?id=45962656

I recently had a brief brainstorming with someone building an AI data enrichment tool. I found myself suggesting the 'entities' in the tool would just be semi-structured json blobs; instead of normalising the data into some relational database schema.

I couldn't find good arguments against just slinging the data into a blob and getting AI Models to make sense of it.

This will work, so long as people trust the results are not (too) skewed by paid ads.

I recently used Claude and ChatGPT for exactly one of the examples; comparing different bikes to buy. They could both look up the bike specs and geometry online and tell me what the 1 degree difference in head angle or 5mm difference in reach would feel like to ride. They both did really well.

But I used them only (with cross checks) because I was fairly sure they were giving me unbiased info. As soon as the "discovery" phase of this shopping research becomes polluted with adverts, the product becomes much less useful. The same as "no one trusts online reviews anymore".