Horos

what about mcp cdp ?

my claude drive his own brave autonomously, even for ui ?

right — no variance, question was off target. worth noting though: the sema-bounded WriteAt goroutines are structurally a fan-out over homogeneous units, even if the pipeline feels linear from the blob's perspective. that's probably why the channel adds nothing — no fan-in, no aggregation, just bounded fire-and-forget.

dead link ?

this makes sense for your workload, but may the right primitive be a function of your payload profile and business constraints ?

in my case the problem doesn't arise because control plane and data plane are separated by design — metadata and signals never share a concurrency primitive with chunk writes. the data plane only sees chunks of similar order of magnitude, so a fixed worker pool doesn't overprovision on small payloads or stall on large ones.

curious whether your control and data plane are mixed on the same path, or whether the variance is purely in the blob sizes themselves.

if it's the latter: I wonder if batching sub-1MB payloads upstream would have given you the same result without changing the concurrency primitive. did you have constraints that made that impractical?

had a look ?

fair point on blocking sends — but that's an implementation detail, not a structural one.

From my pov, the worker pool's job isn't to absorb saturation. it's to make capacity explicit so the layer above can route around it. a bounded queue that returns ErrQueueFull immediately is a signal, not a failure — it tells the load balancer to try another instance.

saturation on a single instance isn't a scheduler problem, it's a provisioning signal. the fix is horizontal, not vertical. once you're running N instances behind something that understands queue depth, the "unfair scheduler under contention" scenario stops being reachable in production — by design, not by luck.

the FrankenPHP case looks like a single-instance stress test pushed to the limit, which is a valid benchmark but not how you'd architect for HA.

Isn't a dedicated worker pool with priority queues enough to get predictable P99 without leaving Go?

If you fix N workers and control dispatch order yourself, the scheduler barely gets involved — no stealing, no surprises.

The inter-goroutine handoff is ~50-100ns anyway.

Isn't the real issue using `go f()` per request rather than something in the language itself?

Thanks for sharing — always a pleasure to discover what others are building.

A few thoughts after your answers:

The E2E file sync part has existing solutions, and your access rights system is really a signed append-only log rather than a blockchain (no consensus, no decentralization) — which is fine, but the term might create misleading expectations.

What I'm more curious about is the access model itself. How are access tokens created and transferred? Who consumes them, and how does authorization propagate? Have you considered a salted API where each user carries a unique identifier, so the whole grant/revoke/delegate flow goes through a single unified mechanism regardless of what's being accessed?

The SQL sync layer is what actually caught my eye — I've worked on similar problems for specific use cases, and encrypted database sync between peers is a genuinely hard problem. That feels like your real differentiator.

On that note: does the SQL layer reference the file content or file paths? I'm guessing you built both interfaces because they're correlated — the SQL holds structured data that points to the encrypted files. If so, that's worth making explicit, because right now they look like two unrelated features rather than two sides of the same system.

even thoses replies create common value.

Tomorow datasets.

Keep publishing !

ACID & Idempotent. dataplane / controlplane. dryruns et runbook automations.

llm does not act on production. he build scripts, and you take the greatest care of theses scripts.

Clone you customer data and run evertything blank.

Just uses the llm tool as dangerous tool: considere that it will fail each time it's able to.

even will all theses llm specific habitus, you still get a x100 productivity.

because each of theses advise can ben implemented by llms, for llms, by many way. it's almost free. just plan it.

who believe here that saying no is possible ?

The "job as library" pattern is simple: instead of wiring jobs into main or a framework, you split into 3 things.

Your queue is a struct with New(db) — it knows submit, poll, complete, fail, nothing else.

Your worker is another struct that loops on the queue and dispatches to handlers registered via RegisterHandler("type", fn). Your handlers are pure functions (ctx,payload) → (result, error) carried by a dependency struct.

Main just assembles: open DB, create queue, create worker, register handlers, call worker.Start(ctx). Result: each handler is unit-testable without the worker or network, the worker is reusable across any pipeline, and lifecycle is controlled by a simple context.Cancel().

Bonus: here the queue is a SQLite table with atomic poll (BEGIN IMMEDIATE), zero external infra.

The whole "framework" is 500 lines of readable Go, not an opaque DSL. TL;DR: every service is a library with New() + Start(ctx), the binary is just an assembler.

The "all in connectivity" pattern means every capability in your system — embeddings, document extraction, replication, MCP tools — is called through one interface: router.Call(ctx,"service", payload).

The router looks up a SQLite routes table to decide how to fulfill that call: in-memory function (local), HTTP POST (http), QUIC stream (quic), MCP tool (mcp), vector embedding (embed), DB replication (dbsync), or silent no-op (noop).

You code everything as local function calls — monolith. When you need to split a service out, you UPDATE one row in the routes table, the watcher picks it up via PRAGMA data_version, and the next call goes remote.

Zero code change, zero restart. Built-in circuit breaker, retry with backoff, fallback-to-local on remote failure, SSRF guard.

The caller never knows where the work happens.

That's the "job as library" pattern: the boundary between monolith and microservices is a config row, not an architecture decision.

https://github.com/hazyhaar/pkg/tree/main/connectivity

And to put it plainly: we won't be able to manage LLM-generated contributions without LLMs. It's physically impossible at this scale.

Which means the immune system has to be built from the same substrate as the threat. The question isn't whether to use AI for review — it's whether that review layer will be open, distributed, and community-owned, or closed, centralized, and controlled by whoever gets there first.

But there's a layer above that which is easy to skip over: human supervision.

Not line-by-line review — that's already gone. What remains is supervision of curated logs, at ratios that might look something like 1 in 10^10. The human role is no longer technical production. It's oversight. And that's a genuinely new function that we don't have good tools for yet.

The flow is perpetual. It doesn't stop, it doesn't slow down, it only accelerates. Which means we'll need to build tooling specifically designed to absorb volume, abstract it into supervisable signals, and train us to work at that level of abstraction — where the unit of human attention is no longer a line of code or a PR, but a pattern across millions of automated actions.

Automation isn't the threat to manage. It's the only viable response to production at this frequency. The question is whether we build the abstraction layer deliberately, as a community, before someone builds it for us.

Something worth sitting with, rather than a conclusion:

As PR velocity reaches this scale — 100 per hour, hundreds of thousands of lines a day — I find myself wondering about the collective immune system side of this.

If we're not yet organized around injection and obfuscation at the community level, PR saturation itself becomes a distinguishable attack vector — and not just for backdoors.

Two distinct risks worth separating:

Offensive saturation: flood a competitor or a fast-moving startup with automated PRs. Their human review bandwidth collapses. Real community contributions drown in noise. The project slows, maintainers burn out, momentum dies. No backdoor needed — attrition is enough.

Forced opening: a project overwhelmed by volume lowers its review standards to survive. It merges faster, checks less. The saturation wasn't meant to block — it was meant to open. Once standards drop, real injection becomes trivial.

The unsettling part: this vector requires no particular skill, is already available, and is organically indistinguishable from legitimate viral growth. To envision an open source that survives AI, maybe we need to envision an open source AI that protects open source.

Genuinely curious if others are thinking about this, and whether anyone has seen serious work in this direction already.

I've set a fully async patern. blobs chunked into sqlite shards.

It's a blind fire n forget go worker danse.

wich can be hold as monitoreed or scale as multiple instances if needed by simple parameters.

Basicaly, It's a job as librairy patern.

If you dont need real time, its bulletproof and very llm friendly.

and a good token saver by the batching abilities.

Three real risks in your current approach before anything else. Shared references mutate silently — in JS, objects passed between your "tables" are aliases, not copies, so a mutation in one place propagates everywhere with no transaction and no rollback. No atomicity — Node is single-threaded but async I/O means two callbacks can interleave writes on the same structure with no guarantee a multi-step update lands cleanly. And everything disappears on crash — for socket/port/container state that's probably fine since it's observable from the system anyway, but you have no history.

That said, you may not need to leave your stack at all. V8's native Map is already a key-value store — O(1) reads, no overhead, typed in TypeScript. Your "tables" are just Maps and cross-referencing is composite string keys:

sockets.set(serverId:{serverId}: serverId:{socketId} , socketData). No library, no dependency, no SQL. This covers your use case as described.

If you want ACID transactions and persistence without SQL, look at lmdb-js — a Node binding on LMDB, the fastest embedded KV store in existence, zero-copy reads, used in production for 20 years. Your tables become named databases, your records are typed values, your cross-references are composite keys. Same mental model you're building, with 20 years of correctness guarantees underneath.

What's the actual reason for building from scratch rather than using native Map for the in-memory case?

Quick question before going further: is this an exercise in language internals, or do you have a concrete use case in mind?

Asking because the answer changes the architecture significantly. If you're targeting live in-page data — extracting objects from the DOM as you browse, filtering them reactively — you may not need storage at all.

A Proxy-based observation layer gives you reactive queries without allocating anything new: the objects already exist in the tab's heap, you're just watching them mutate. No GC pressure, no persistence headaches, no query planner needed. That covers most of what you described: "items where price < 50 updates as you browse" is an event subscription with pattern matching, not a database problem.

The cases where you actually need storage — and therefore need to think about heap budgets, GC, serialization, query planning — are narrower:

Cross-session persistence (you want the data after the tab closes) Cross-tab aggregation (comparing prices across multiple open tabs simultaneously) Queries over historical data (not just what's on screen now, but what you saw across 20 pages of browsing)

Those are real storage problems.

But they're also the cases where you're competing with IndexedDB, OPFS, and SQLite WASM — and "I hate SQL" stops being enough of a reason to rebuild from scratch. What's the actual workflow you're trying to support?

Nice try — the local-first distributed pattern worth building on.

Three questions:

Why blockchain for access rights? A signed Merkle structure or a Certificate Transparency-style log would give the same guarantees without the operational complexity. What does the blockchain add here that a simpler append-only signed registry doesn't?

The threat model is unclear. If the blockchain provider controls validation, the "accessible only to end users" guarantee depends on trusting that provider. This is the oracle problem — the chain guarantees integrity of what's inside it, but not the truthfulness of what gets written in. Who runs the chain, and what happens if they're compromised or write false access rights?

Go is listed first in the bindings but the example code is Python. Is the Go binding at feature parity, or is Python the primary target?

yes, using skills. ui slop gonna end soon.

gave a try:got lost :)

Interesting gap between surface scanning (6.6%) and AI deep audit (16.4%).

Two concerns with the AI audit approach. First, the defense LLM is itself an attack surface — we're already seeing payloads crafted specifically to bypass LLM-based guardrails. If the guardian is injectable, you've added a vulnerability to your security stack.

Second, the Mindgard paper from late 2025 tested 12 character injection techniques against 6 guardrails including ProtectAI's DeBERTa, Meta Prompt Guard, Azure Prompt Shield — some hit 100% evasion rate. Homoglyphs, zero-width chars, leet, diacritics. Simple stuff, but the classifiers see raw tokens and can't handle it.

I built a prompt injection detection library that tackles this from the normalisation layer — 10-stage deterministic pipeline (NFKD, confusable fold, leet, base64, zero-width strip, ROT13, escape sequences) that reduces all evasion to canonical form before any matching. The scan itself is not injectable — it's code, not a model.

Where I think this goes next: small encoder-only classifiers (DeBERTa-small, ModernBERT) running on already-normalised text. Post-normalisation, the model only needs to detect the logical intent pattern, not handle evasion — that's the layer below. Too small to be reprogrammed via prompt, too focused to be redirected. One classifier per attack category: override, extraction, jailbreak, etc.

But these classifiers will only be as good as their training data. Right now everyone trains on static datasets (deepset, safeguard). What's missing is a community-maintained corpus fed by real-world incident reports — like antivirus signature databases. The detection engine matters less than the definitions it runs on. ClamAV isn't great because of its scan loop, it's great because thousands of people report samples.

Your foundry example — no payload until the agent writes it — is the genuinely hard case that needs AI. But for everything else, deterministic normalisation + focused micro-classifiers + community-curated signatures is a more defensible architecture than putting another LLM in the path.

https://github.com/hazyhaar/pkg/tree/main/injection

Prompt injection detection library in Go, zero regex.

Most injection evasion works by making text look different to a scanner than to the LLM. Homoglyphs, leet speak, zero-width characters, base64 smuggling, ROT13, Unicode confusables — the LLM reads through all of it, but pattern matchers don't.

The project is two curated layers, not code:

Layer 1 — what attackers say. ~35 canonical intent phrases across 8 categories (override, extraction, jailbreak, delimiter, semantic worm, agent proxy, rendering...), multilingual, normalized.

Layer 2 — how they hide it. Curated tables of Unicode confusables, leet speak mappings, LLM-specific delimiters (<|system|>, [INST], <<SYS>>...), dangerous markup patterns. Each table is a maintained dataset that feeds a normalisation stage.

The engine itself is deliberately simple — a 10-stage normalisation pipeline that reduces evasion to canonical form, then strings.Contains + Levenshtein. Think ClamAV: the scan loop is trivial, the definitions are the product.

Long term I'd like both layers to become community-maintained — one curated corpus of injection intents and one of evasion techniques, consumable by any scanner regardless of language or engine.

Everything ships as go:embed JSON, hot-reloadable without rebuild. No regex (no ReDoS), no API calls, no ML in the loop. Single dependency (golang.org/x/text). Scans both inputs and LLM outputs.

result := injection.Scan(text, injection.DefaultIntents()) if result.Risk == "high" { ... }

https://github.com/hazyhaar/pkg/tree/main/injection

Great writeup. I hit the same problems on a 590-file Go monorepo and ended up building two "skill files" that address several of your friction points directly.

Your BODY.PEEK bug is the canonical case of "the agent doesn't know what it doesn't know." My approach: CLAUDE:WARN annotations directly above functions in source code.

The agent sees the trap at the moment it touches the function, not after the bug ships.

If email.py had a CLAUDE:WARN saying "messages marked Seen before processing — use BODY.PEEK or data loss on failure", the agent would have caught it while writing the code, not after production.

Same for architectural drift. Your slash command duplication across cli.py, email.py, telegram.py — locally reasonable, globally messy. A per-directory ASCII schema (skill 2) makes the intended architecture explicit.

An agent reading the dispatch schema sees that all channels must route through commands.py. Without it, it only sees the file it's editing.

The sub-agent blank context problem: each local CLAUDE.md starts with a protocol pointer to root + 3 mandatory grep commands. Sub-agents inherit the research protocol structurally instead of by luck.

The deeper difference from what you describe: your CLAUDE.md is a flat file that grows after each bug. In my pattern, annotations live distributed in the code (CLAUDE:WARN, CLAUDE:SUMMARY, CLAUDE:DEPENDS), an audit skill maintains them periodically, and the per-directory CLAUDE.md stays a 50-line manifest — not a catch-all.

One more pattern I'd add to your workflow: a prod bug is a test failure first. No fix ships without a red test that reproduces it, goes green, and gets logged. ( claude does it autonomously very well. )

The agent is great at writing the fix, but left to itself it will patch the code and move on. Force the cycle: red test → fix → green test → commit.

Your BODY.PEEK bug becomes a regression test that protects you forever, not just a rule in CLAUDE.md that the agent might or might not read next session.

A/B on the same prompt, same codebase: 58K tokens with the full doc system vs 73K without, 2 minutes vs 8 minutes, zero false positives vs at least one.

https://github.com/hazyhaar/GenAI_patterns

Prompt injection detection library in Go, zero regex.

Most injection evasion works by making text look different to a scanner than to the LLM. Homoglyphs, leet speak, zero-width characters, base64 smuggling, ROT13, Unicode confusables — the LLM reads through all of it, but pattern matchers don't.

The project is two curated layers, not code:

Layer 1 — what attackers say. ~35 canonical intent phrases across 8 categories (override, extraction, jailbreak, delimiter, semantic worm, agent proxy, rendering...), multilingual, normalized.

Layer 2 — how they hide it. Curated tables of Unicode confusables, leet speak mappings, LLM-specific delimiters (<|system|>, [INST], <<SYS>>...), dangerous markup patterns. Each table is a maintained dataset that feeds a normalisation stage.

The engine itself is deliberately simple — a 10-stage normalisation pipeline that reduces evasion to canonical form, then strings.Contains + Levenshtein. Think ClamAV: the scan loop is trivial, the definitions are the product.

Long term I'd like both layers to become community-maintained — one curated corpus of injection intents and one of evasion techniques, consumable by any scanner regardless of language or engine.

Everything ships as go:embed JSON, hot-reloadable without rebuild. No regex (no ReDoS), no API calls, no ML in the loop. Single dependency (golang.org/x/text). Scans both inputs and LLM outputs.

result := injection.Scan(text, injection.DefaultIntents()) if result.Risk == "high" { ... }

https://github.com/hazyhaar/pkg/tree/main/injection

Interesting project. A few questions that came to mind: How do you handle GC pressure at scale? V8's hidden classes make homogeneous object arrays fast, but the per-object overhead adds up — 100K entries is already 6-8 MB of metadata alone, and major GC pauses become unpredictable. What's the persistence story? The moment you serialize to IndexedDB or OPFS, the "native structures" advantage disappears. Have you looked at columnar formats to keep it fast? How do you handle compound queries without a planner? Something like "age > 30 AND city = 'Paris' ORDER BY name" needs index selection strategy, otherwise you're back to full scans. The part I find most compelling is reactive queries — define a filter, then as objects land in the store (from DOM extraction, a WebSocket, whatever), results update incrementally via Proxy interception. No re-scan. That's not really a database, it's a live dataflow layer. Concrete example: a browser extension that extracts product data from whatever page you're on. Each page dumps heterogeneous objects into the store. A reactive query like "items where price < 50 and source contains 'amazon'" updates in real time as you browse. No server, no SQL, just JS objects flowing through live filters. That would be genuinely useful and hard to do well with existing tools.

Thanks for the pointer to asyncmachine! Let me clarify HOROS architecture since there's some confusion.

HOROS uses time slots for orchestrator clones on a SINGLE machine by default. Not distributed - 5 Go processes share the same kernel clock:

Runner-0: T=0s, 10s, 20s... (slot 0) Runner-1: T=2s, 12s, 22s... (slot 1) Runner-2: T=4s, 14s, 24s... (slot 2)

Zero network, zero clock drift. Just local time.Sleep().

Your approach (logical clocks) solves event ordering in distributed systems. HOROS solves periodic polling - workers can be idle for hours with no events to increment a logical clock. Wall-clock fires regardless.

Different primitives: - Logical clocks: "Event A before Event B?" (causality) - TDMA timers: "Is it your turn?" (time-slicing)

For cross-machine workflows, we use SQLite state bridges:

Machine-Paris Machine-Virginia ┌─────────────┐ ┌──────────────┐ │ Worker-StepA│ │ Worker-StepC │ │ completes │ │ waits │ │ ↓ │ │ ↑ │ │ output.db │ │ input.db │ └──────┬──────┘ └──────▲───────┘ │ │ └──→ bridge.db ←─────────────────┘ (Litestream replication)

bridge.db = shared SQLite with state transitions StepBridger daemon polls bridge.db, moves data between steps

State machines communicate through data writes, not RPC. Each node stays single-machine internally (local TDMA).

Re: formatting - which results were unclear? Happy to improve.

TDMA schedules the orchestrators (lightweight checks), not the workers (heavy jobs).

Orchestrators: Active 1/n of time (~10ms to check state) Workers: Run continuously for hours once started

T=0s: Orchestrator-0 checks → starts job (runs 2 hours) T=2s: Orchestrator-1 checks → job still running T=10s: Orchestrator-0 checks again → job still running

Think: traffic lights (TDMA) vs cars (drive continuously).

Work throughput is unchanged. TDMA only coordinates who checks when.

Gemma Term of uses ?