pipejosh

$2 million a year to run 244,000 searches that advanced 361 cases... That's about $5,500 per useful search.

Meanwhile every car that drove past one of those cameras got logged, timestamped, and stored. These things aren't not law enforcement, they're mass surveillance with a badge.

I settled on a similar workflow but across two agents instead of one session.

One agent writes task specs. The other implements them. Handoff files bridge the gap. The spec IS the session artifact because it captures intent, scope, and constraints before any code gets written.

The plan.md approach people are describing here is basically what happens naturally when you force yourself to write intent before execution.

Tried self-hosting with Mattermost to get around Slacks 90 day free tier history but my team didn't care for it much. Ended up back on Slack's free tier. This may solve that issue for me, will check it out.

Agreed. The non-determinism makes traditional testing basically useless here. You can't write a test suite for "the agent decided to do something unexpected this time." Logging and runtime checks are the only way to catch the weird edge cases.

The automated audit only covers static analysis. When the agent actually runs, hitting MCP servers, making HTTP calls, getting responses back, that's where the real problems show up. Prompt injection through tool responses, malicious libraries that exfiltrate env vars, SSRF from agents that blindly follow redirects. Code audits miss all of it because this is a runtime and network problem, not a code quality problem.

Built Pipelock for this actually. It's a network proxy that sits between the agent and everything it talks to. Still early but the gap is real. https://github.com/luckyPipewrench/pipelock

The maintenance burden is real but I think security is the bigger gap. People vibing out code with AI aren't thinking about input validation or dependency vulnerabilities. They build it, it works, they ship it. Then they're running unpatched code with no security review. That's where things get ugly.

The part that worries me about agentic everything is the security model hasn't caught up. We're handing agents more and more access (shell, network, APIs, file systems) and the security story is still basically "the model probably won't do bad things." That's not how we secure anything else in computing. Principle of least privilege should apply to agents the same way it applies to services.

Circuit breakers for cost control is smart. The security equivalent is rate limiting and DLP on the egress side. If your agent suddenly starts making a bunch of requests to domains it's never hit before, or starts including high-entropy strings in URLs, something's wrong. Cost and security are two sides of the same observability problem.

Sandboxing the filesystem is one layer but egress scanning is where it gets interesting. An agent inside a sandbox can still exfiltrate secrets through any HTTP request it's allowed to make. The request looks totally legitimate from the sandbox's perspective. You need something actually inspecting the content of outbound traffic for credential patterns.

This is cool for testing the model side, but the real scary part is what happens after the injection succeeds. Even if your agent fails 3 out of 10 tests, that's a 30% chance it exfiltrates whatever secrets are in its environment. The defense can't just be "hope the model catches it." You need architectural controls on the egress side too.

Everyone's talking about how productive agents are but nobody's talking about what happens when one gets prompt injected. Your agent has shell access, your API keys in env vars, and unrestricted internet. That's one bad dependency readme away from leaking everything. The productivity gains are real but so is the attack surface.