Show HN: Pydantic-AI-stream – Structured event streaming for pydantic-AI agents

https://github.com/opale-ai/pydantic-ai-stream

by sbargaoui • 7 months ago

6 3 7 months ago

sbargaoui2• 7 months ago

We've been building agents with pydantic-ai in production. The framework is great for defining agents, but we kept rewriting the same infrastructure to actually show them to users.

The pattern we landed on: agent runs in a background worker, emits events to Redis Streams, frontend consumes via SSE. Multiple clients can listen to the same stream. If the user navigates away or hits stop, we delete a key and the agent aborts gracefully at the next node boundary.

pydantic-ai gives you Agent.iter() and streaming primitives, but wiring this up - structured events, reconnection, history across turns - is a lot of glue.

This library wraps iter() and handles the lifecycle:

begin → [llm-begin → part-deltas → llm-end]+ → end

Each event is typed: text delta, tool call with args, tool return, error. Frontend gets structured JSON it can render directly.

It's thin (~400 LOC) and doesn't patch pydantic-ai internals. You bring your own session storage by implementing load/save.

Feedback welcome, especially on the event protocol.

shikhar• 7 months ago

Very cool! Would you consider making the streaming backend pluggable? s2.dev could make a lot of sense as a serverless option, and a self-hostable OSS implementation of the API is also coming soon. S2 is great for agent session-level streams (https://s2.dev/blog/agent-sessions), and unlike Redis Streams all data is always completely durable on object storage.

sbargaoui2• 7 months ago

Thanks! Pluggable backends are surely on the radar.

The current implementation is deliberately thin: the stream operations are just xadd/xread/set/get/delete. Abstracting that into a protocol wouldn't be hard, and a model based on stream-per-session, durable and serverless fits the use case well.

Redis was the pragmatic choice for v0 since most teams already have it running and the latency is good for real-time streaming to frontends. But durability is a valid concern, especially if the agent run matters (billing, compliance, debugging), you want it persisted properly, not just in Redis memory.

If S2's self-hostable OSS version lands soon, that'd lower the barrier for people to try it.

Would love to hear if there are other backend preferences out there !

Show HN: Pydantic-AI-stream – Structured event streaming for pydantic-AI agents

Related Stories