vincentvandeth

Update: I ended up building this into a full closed-loop pipeline. A PreToolUse hook detects context pressure at 65%, the agent writes a structured handover (task state, files, progress), tmux clears the session, and a rotator script injects the continuation into the fresh session.

The key insight from testing: rotating at 60-70% — before quality degrades — matters more than the rotation mechanism itself. At 80%+ auto-compact kicks in and races with any cleanup you try to do.

Wrote it up as a Show HN if anyone's curious: https://news.ycombinator.com/item?id=47152204

Nice — just checked it out. The interceptor approach makes sense for teams that need policy enforcement across multiple agents.

Interesting difference in philosophy though: Limits enforces rules defined upfront, while what I built learns rules from production receipts. After 1100+ task completions, the dispatch patterns look completely different from what I would have designed on day one.

Probably complementary — you'd want both. Pre-defined guardrails for the dangerous stuff (your approach), and pattern evolution for the quality/efficiency stuff (mine).

Fair point on the time cost — but I'd frame it differently. The 6 months wasn't spent building a governance layer instead of building the agent. The governance layer grew out of the actual project work. Every receipt, every quality rule, every dispatch pattern was a direct response to something that broke in production. Day one I had zero governance and a working agent. By month six I had 1100+ receipts and a system that catches failures before they ship.

The infrastructure approach makes sense for teams who want to skip the learning curve. The trade-off is that pre-built governance rules are generic by definition — they can't know that your specific codebase breaks when tasks exceed 300 lines, or that planning gates without explicit deliverables always need redispatch. That pattern data only comes from running your own agents on your own work.

Curious what you're building — is it the ledger/tracking layer, the quality gates, or the full orchestration?

Context bloat is a real problem — and yes, order matters more than most people realize. Descriptions near the top of the tool list get preferentially selected, especially in long contexts where attention degrades.

Two things I do to fight this:

First, skill scoping per task. Instead of exposing all 20+ skills to every agent, each terminal only sees the 3-5 skills relevant to its current dispatch. The orchestrator decides which skills to load before the agent even starts. Less noise, better selection accuracy.

Second, context rotation to prevent context rot setting in. When an agent's context fills up, the system automatically writes a structured handover, clears the window, and resumes in a fresh context. This is critical because a degraded context doesn't just pick worse tools — it starts ignoring instructions entirely. A fresh context with a good handover outperforms a bloated one every time.

I'm actually testing automatic refresh at 60-70% usage right now — not waiting until the window is nearly full, but rotating early to prevent context rot before it starts. Early results suggest that's the sweet spot: late enough that you've done meaningful work, early enough that the handover quality is still high.

The organic discovery problem you're describing is essentially unsolvable with a flat tool list. The more tools you add, the worse selection gets — it's not linear degradation, it's closer to exponential once you pass ~15-20 tools in context. The only path I've found is hierarchical: a routing layer that narrows the set before the agent sees it.

About 6 months of iterating, but in bursts — I built it while using it on a production project, so the governance layer grew alongside real failure modes rather than being designed upfront.

The maintenance question is the right one. The rules themselves are low-maintenance because they're deliberately simple and deterministic — file size limits, test coverage thresholds, blocker counts. They don't need updating when the model changes because they don't depend on LLM behavior.

What does evolve is the dispatch templates — how I scope tasks and what context I give agents upfront. That's where the ledger pays for itself. After 1100+ receipts, I can see patterns like "tasks scoped above 300 lines fail 3x more often" or "planning gates without explicit deliverables always need redispatch." Those patterns feed back into how I write dispatches, not into the rules themselves.

So the rules stay stable, but the way I use the system keeps improving. The governance layer is the boring part — the interesting part is the feedback loop from receipts to dispatch quality.

Hard-coded checks before every action, plus a governance layer that separates "what the agent wants to do" from "what it's allowed to do." The deeper issue: if your agent decides whether to issue a refund, you're solving the wrong problem with prompt guards. A refund is a deterministic business rule — order exists, within return window, amount matches. That decision shouldn't be made by an LLM at all.

In my setup, agents propose actions and write structured reports. A deterministic quality advisory then runs — no LLM involved — producing a verdict (approve, hold, redispatch) based on pre-registered rules and open items. The agent can hallucinate all it wants inside its context window, but the only way its work reaches production is through a receipt that links output to a specific git commit, with a quality gate in between.

For anything with real consequences (database writes, API calls, refunds), the pattern is: LLM proposes → deterministic validator checks → human approves. The LLM never has direct write access to anything that matters.

"Just hoping for the best" works until it doesn't. We tracked every agent decision in an append-only ledger — after a few hundred entries, you start seeing exactly where and how agents fail. That pattern data is more useful than any prompt guard.

Yes, tmux. The setup is a 2x2 grid:

T0 (orchestrator) | T1 (Track A) T2 (Track B) | T3 (Track C)

When a worker finishes, it writes a structured report to a shared unified_reports/ directory. A file watcher (receipt processor) detects it, parses the report into a structured NDJSON receipt (status, files changed, open items, git ref), and delivers it to T0's pane.

T0 then reviews the receipt, runs a quality advisory (automated pass/warn/hold verdict), and decides: close open items, complete the PR, or redispatch. Everything is filesystem-based — no API, no database, no shared memory between agents. Each terminal has its own context window, its own Claude Code (or Codex/Gemini) session, and the only communication channel is structured files on disk.

The receipt ledger is append-only NDJSON, so you can always trace: which agent did what, when, on which dispatch, with which git commit.

I open-sourced the setup recently if you want to dig into the details.

You're right that no model handles everything in one context window — that's exactly why I built context rotation. Each task runs in a single agent context (one responsibility, clear scope), and when the window fills up, the system automatically rotates: writes a structured handover, clears, and resumes in a fresh window.

The key distinction: sub-agents run within a parent context with shared state (black box). My approach uses independent parallel agents (separate terminals, separate context windows) that report back to an orchestrator. Large tasks get split into smaller dispatches upfront — each scoped to fit a single context window. The orchestrator can dispatch research to 3 agents in parallel, collect their outputs, then dispatch a synthesis task to a single agent that merges the findings.

So it's not "one context window for everything" — it's right-sized tasks with full observability per agent, and a governance layer managing the sequence and merging results.

This approach sounds clean in theory, but in production you're building a black box. When your planning agent hands off to an implementation agent and that hands off to a review agent — where did the bug originate? Which agent's context was polluted? Good luck tracing that. I went the opposite direction: single agent per task, strict quality gates between steps, full execution logs. No sub-agents. Every decision is traceable to one context window. The governance layer (PR gates, staged rollouts, acceptance criteria) does the work that people expect sub-agents to do — but with actual observability.

After 6 months in production and 1100+ learned patterns: fewer moving parts, better debugging, more reliable output. Built a full production crawler this way — 26 extractors, 405 tests — without sub-agents. Orchestrator acts as gatekeeper that redispatches uncompleted work.

Great list. I've been running a multi-agent orchestration system (11 specialized AI agents) in production for 6 months and your #2 and #5 resonate hard.

What I'd add:

6. Confidence without evidence. Agents will report "task complete" with high confidence when the output is plausible but wrong. Without automated validation gates, you won't catch it until production breaks. 7. Context drift in long sessions. After 50+ tool calls, agents start losing track of earlier decisions. They'll contradict their own architecture choices from 20 minutes ago. Session length is an underrated failure vector. 8. The "almost right" problem. Agents rarely fail catastrophically — they fail subtly. Code that passes tests but misses edge cases. Docs that look complete but have wrong cross-references. This is worse than obvious failures because you trust the output.

What fixed most of these for me:

Quality gates between agents — no agent's output moves forward without automated checks (tests, schema validation, consistency checks) Evidence-based confidence scores — not "how sure are you?" but "what specific evidence supports this output?"

Human-in-the-loop at decision points, not everywhere. You can't review everything, so you design the system to surface the right moments for human judgment Small scoped tasks, agents working on 150-300 line PRs with clear acceptance criteria fail way less than agents given open-ended goals

Your #5 (implementation gap) is the one I see most people underestimate. The fix isn't better agents, it's better systems around the agents.

Happy to share more details about the architecture if anyone's interested

Great questions. I've been running ~2,400 multi-agent dispatches across 4 terminals (different AI models) for about 6 months, so I'll share what I've hit in practice rather than theory.

On RFC 3161 vs. multi-witness anchoring: For most production agent systems today, RFC 3161-style timestamping is overkill — and so is multi-witness anchoring. The practical threat model isn't a sophisticated adversary backdating entries. It's your own agents producing self-consistent but wrong output, and you not being able to reconstruct the sequence after the fact. The real defensibility problem is completeness, not tamper-proofing. Can you prove nothing was omitted? That's harder than proving nothing was altered. I use an external watcher process that observes agent output independently — the agent doesn't write to the log, so it can't selectively omit entries. That separation does more for defensibility than any cryptographic anchoring would at my scale.

On replayability: It breaks down the moment you involve external state. An agent that reads a file, calls an API, or queries a database — the inputs to that decision are gone unless you explicitly snapshot them. Git shows what changed, but not what the agent saw when it decided to change it. Chat sessions expire. CoT gets truncated. I capture pre-state and post-state per dispatch, but the reasoning trace between them is still the weakest link. Nobody I've seen solves this cleanly yet. The best workaround I've found: treat the agent's self-reported reasoning as one input, but verify it against an independent quality check (separate process the agent can't influence). You can't replay the reasoning, but you can independently verify the outcome.

On the overengineering threshold: For me it became necessary after about 200 dispatches. One of my agents hallucinated a dependency, a second agent "fixed" the resulting test failures by creating the missing module, and by morning I had three rewritten files with clean commits built on something that should never have existed. The scary part wasn't the mistake — it was that I couldn't reconstruct why it happened. At single-agent scale, you can eyeball diffs. The moment you have agents responding to each other's output, forensic reconstruction stops being optional. I'd say the threshold is: if agent A's output can trigger agent B's action without human review, you need defensible logs. Not because of litigation risk, but because you literally can't debug cascading failures without them.

Interesting approach. Runtime enforcement is the part most people skip — they focus on logging what happened but don't prevent bad actions in the first place. The policy engine + kill switch combination makes sense for that.

I've been running ~2,400 multi-agent dispatches and came at this from the opposite direction: I started with staging gates (propose → human review → execute) as the runtime layer, then realized I also needed a forensic layer for when things slip through or when I need to understand patterns over time.

Curious about a few things:

- How granular are the JSON policies in practice? I found that "agent X can use tool Y" breaks down fast when agents chain tools in unexpected ways. The sequence matters more than individual permissions. - The hash-chained audit trail — how do you handle schema evolution? After a few months of production, what you want to log changes significantly. Hash chains make adding fields tricky without breaking the chain. - What happens when an agent crashes mid-action? With the hash chain, do you risk a corrupted tail entry that invalidates subsequent verification?

The runtime vs. after-the-fact distinction is important. Ideally you want both — prevent what you can, reconstruct what you couldn't prevent. Nice to see someone tackling the prevention side seriously.