Claude built a system in 3 rounds, latent bugs from round 1 exploded in round 3

https://github.com/mycelium-clj/mycelium/blob/main/benchmarks/SCALING.md

by yogthos • 5 months ago

33 75 5 months ago

reedf17• 5 months ago

It's starting to become obvious that if you can't effectively use AI to build systems it is a skill issue. At the moment it is a mysterious, occasionally fickle, tool - but if you provide the correct feedback mechanisms and provide small tweaks and context at idiosyncrasies, it's possible to get agents to reliably build very complex systems.

Zafira• 5 months ago

At the moment it is a mysterious, occasionally fickle, tool - but if you provide the correct feedback mechanisms and provide small tweaks and context at idiosyncrasies, it's possible to get agents to reliably build very complex.

This sounds like arguing you can use these models to beat a game of whack-a-mole if you just know all the unknown unknowns and prompt it correctly about them.

This is an assertion that is impossible to prove or disprove.

rafaelmn2• 5 months ago

No it's more like if you knew how to build it before - LLM agents help you build it faster. There's really no useful analogy I can think of, but it fits my current role perfectly because my work is constantly interrupted by prod support, coordination, planning, context switching between issues etc.

I rarely have blocks of "flow time" to do focused work. With LLMs I can keep progressing in parallel and then when I get to the block of time where I can actually dive deep it's review and guidance again - focus on high impact stuff instead of the noise.

I don't think I'm any faster with this than my theoretical speed (LLMs spend a lot of time rebuilding context between steps, I have a feeling current level of agents is terrible at maintaining context for larger tasks, and also I'm guessing the model context length is white a lie - they might support working with 100k tokens but agents keep reloading stuff to context because old stuff is ignored).

In practice I can get more done because I can get into the flow and back onto the task a lot faster. Will see how this pans out long term, but in current role I don't think there are alternatives, my performance would be shit otherwise.

reedf17• 5 months ago

The same is true with human engineers - isn't this just what engineering is?

threethirtytwo5• 5 months ago

This is an assertion that is impossible to prove or disprove.

This is a joke right? There are complex systems that exist today that are built exclusively via AI. Is that not obvious?

The existence of such complex systems IS proof. I don't understand how people walk around claiming there's no proof? Really?

stavros7• 5 months ago

I'd agree, I've been building a personal assistant (https://github.com/skorokithakis/stavrobot) and I'm amazed that, for the first time ever, LLMs manage to build reliably, with much fewer bugs than I'd expect from a human, and without the repo devolving to unmaintainability after a few cycles.

It's really amazing, we've crossed a threshold, and I don't know what that means for our jobs.

Grimblewald16• 5 months ago

No bugs means nothing if bugs get hidden and llms are great at hiding bugs and will absolutely fail to find some fairly critical ones. Your own repo, which is slop at best, fails to meet its core premise

Another AI agent. This one is awesome, though, and very secure.

it isn't secure. It took me less than three minutes to find a vulnerability. Start engaging with your own code, it isn't as good as you think it is.

edit: i had kimi "red team" it out of curiosity, it found the main critical vulnerability i did and several others

Severity - Count - Categories

Critical - 2 - SQL Injection, Path Traversal

High - 4 - SSRF, Auth Bypass, Privilege Escalation, Secret Exposure

Medium - 3 - DoS, Information Disclosure, Injection

You need to sit down and really think about what people who do know what they're doing are saying. You're going to get yourself into deep trouble with this. I'm not a security specialist, i take a recreational interest in security, and llm's are by no means expert. A human with skill and intent would, i would gamble, be able fuck your shit up in a major way.

Grimblewald16• 5 months ago

Say i buy into your mysticism based take, is it a useful tool if it blows up in damn well near every professionals face?

lets say i accept you and you alone have the deep majiks required to use this tool correctly, when major platform devs could not so far, what makes this tool useful? Billions of dollars and environment ruining levels of worth it?

I'd say the only real use for these tools to date has been mass surveillance, and sometimes semi useful boilerplate.

reedf17• 5 months ago

Honestly people are in such a weird place with this shit. I'm not saying don't read the fucking code - but I managed to get my setup to write 100k lines of indistinguishable SWE code in a week or so. The main limitation was my reading speed. This is something like a 10x speedup for me.

mikkupikku4• 5 months ago

is it a useful tool if it blows up in damn well near every professionals face

It doesn't, that's ego-preserving cope. Saying that this stuff doesn't work for "damn well near every professional" because it doesn't work for you is like a thief saying "Everybody else steals, why are you picking on me"? It's not true, it's something you believe to protect your own self-image.

croes2• 5 months ago

if you provide the correct feedback

And how do you define correct feedback? If the output is correct?

reedf17• 5 months ago

I don't know if you deliberately cut-off the full point, but for the benefit of those with tired eyes I said 'feedback mechanisms', i.e. feedback in the control system sense.

zihotki2• 5 months ago

According to the https://blog.katanaquant.com/p/your-llm-doesnt-write-correct... previously discussed on HN, it may be at least partially true:

The vibes are not enough. Define what correct means. Then measure.

mikkupikku4• 5 months ago

Truth Nuke

bartread• 5 months ago

It's starting to become obvious that if you can't effectively use AI to build systems it is a skill issue.

I think it's fair to say that you can get a long way with Claude very quickly if you're an individual or part of a very small team working on a greenfield project. Certainly at project sizes up to around 100k lines of code, it's pretty great.

But I've been working startups off and on since 2024.

My last "big" job was with a company that had a codebase well into the millions of lines of code. And whilst I keep in contact with a bunch of the team there, and I know they do use Claude and other similar tools, I don't get the vibe it's having quite the same impact. And these are very talented engineers, so I don't think it's a skill either.

I think it's entirely possible that Claude is a great tool for bootstrapping and/or for solo devs or very small teams, but becomes considerably less effective when scaled across very large codebases, multiple teams, etc.

For me, on that last point, the jury is out. Hopefully the company I'm working with now grows to a point where that becomes a problem I need to worry about but, in the meantime, Claude is doing great for us.

Greed• 5 months ago

Would definitely tend to agree. Whenever I read complaints about accuracy of LLMs with complex systems, it has generally been from those that aren't thinking very critically about how they're using them in the first place. If you were to replace that LLM with a real human junior, would you really walk away for a few weeks and then assume the solution given was correct by default when you got it back? Obviously not. So you identify and gatekeep the most critical parts ahead of time, make error correction part of the process, and chunk the Giant, Complex Thing into Smaller, Achievable, Verifiable Things.

LLMs are proving to be very much force multipliers of the kind of developer you already are, and of those who report a 10x increase in productivity they're probably all being genuine. Whether that 10x is of careful, thoughtful choices or reckless rough-shod slop though is really an artifact of the developers themselves. I've been saying from the beginning that your effectiveness with LLMs is roughly equivalent to your ability to get effective results out of a real team of human contractors.

yogthos6• 4 months ago

The point is that we can make life easier for both ourselves and the agents by structuring code in a way that's conducive towards their strengths.

NotGMan• 5 months ago

How is that different compared to a human dev that would miss bugs?

Grimblewald16• 5 months ago

The ability to plan long term and anticipate flow on errors.

onion2k2• 5 months ago

You chosen AI agent can and will plan as far ahead as you tell it to plan. If you tell it to do things in iteration 1 that might block it in iteration 3 then you should have told it more about where you want to go back in iteration 1.

nomercy400• 5 months ago

Yes, this is simply 'technical debt'.

They should try to fix technical debt before going to the next round. Of course Claude can probably also do this.

duskdozer2• 5 months ago

The people using these programs are generating massive amounts of code, and you won't convince me they're actually carefully understanding most of it, if they even read it all. And it bypasses the first verification step where you are actively typing in the code that will be run.

kleiba• 5 months ago

I didn't downvote you, but I've noticed recently a trend on HN that just asking a normal question (possibly to start a discussion) gets downvoted for no apparent reason and without any explanation. Not good etiquette, in my opinion.

threethirtytwo5• 5 months ago

I think it's all the angry people who were wrong about AI. Every week they come on HN and say AI is utter crap and useless, and every week AI becomes more and more part of the developer work flow.

I would be pissed off too if I was a hypocrite who was so sure AI was total garbage and was now at the same time needing to use claude on a daily basis.

A lot of developers are going through an identity crisis where their skills are becoming more and more useless and they need to attack comments like the above in a desperate but futile attempt to make themselves matter.

yogthos6• 4 months ago

It's a similar problem in the human context, but I think the reason stuff like workflow agents haven't caught on is because humans don't really like to work this say. Writing a conditional and calling a function keeps you in your flow, but having to jump between an orchestration layer and your code with implementation details breaks that. But LLMs don't have this problem. In fact, they benefit from having all the additional information that's expressed in the graph layer.

karel-3d2• 5 months ago

I have no idea what am I reading

Mycelium structures applications as directed graphs of pure data transformations. Each node (cell) has explicit input/output schemas. Cells are developed and tested in complete isolation, then composed into workflows that are validated at compile time. Routing between cells is determined by dispatch predicates defined at the workflow level — handlers compute data, the graph decides where it goes.

No still don't understand

Mycelium uses Maestro state machines and Malli contracts to define "The Law of the Graph," providing a high-integrity environment where humans architect and AI agents implement.

Nope, still don't

IshKebab• 5 months ago

Yeah it reads very Time Cube...

harperlee• 5 months ago

I don't understand why the poster (which is the author) links us to a slop report of a test for their library. It would be much more effective to cover part of this info into the README where we get the context of what they want to achieve (where there is a very clear "Why?" section), and then link to it instead. I have flagged it as AI slop.

karel-3d2• 5 months ago

I don't understand LISP or Clojure, but it seems to be some kind of library for making web services out of LISP, which has some separate components that are somehow well defined. And somehow it's all related to AI.

Again I don't know much about Clojure and I am too slow for functional programming in general.

yogthos6• 4 months ago

The whole point of the framework is to see what LLM oriented framework would look like. My argument is that the way code is normally structured is not conducive towards LLMs because context grows in unbounded way, and they end up getting lost.

The whole point of the 'slop' report is to have the LLM try implementing the features using both the traditional approach and the framework and then reflect on how it fared with each approach.

_flux• 5 months ago

The top-level README gives a bit better idea. Armed with that the explanation might sound a bit more understandable.

I'm not familiar with the project (or Clojure), but let me try to explain!

Mycelium structures applications as directed graphs of pure data transformations.

There is a graph that describes how the data flows in the system. `fn(x) -> x + 1` in a hypothetical language would be a node that takes in a value and outputs a value. The graph would then arrange that function to be called as a result of a previous node computing the parameter x for it.

Each node (cell) has explicit input/output schemas.

Input and output of a node must comply to a defined schema, which I presume is checked at runtime, as Clojure is a dynamically typed language. So functions (aka nodes) have input and output types and presumably they should try to be pure. My guess is there should be nodes dedicated for side effects.

Cells are developed and tested in complete isolation, then composed into workflows that are validated at compile time.

Sounds like they are pure functions. Workflows are validated at compile time, even if the nodes themselves are in Clojure.

Routing between cells is determined by dispatch predicates defined at the workflow level — handlers compute data, the graph decides where it goes.

When the graph is built, you don't just need to travel all outgoing edges from a node to the next, but you can place predicates on those edges. The aforementioned nodes do not have these predicates, so I suppose suppose the predicates would be their own small pure-ish functions with the same as input data as a node would get, but their output value is only a boolean.

Mycelium uses Maestro state machines and

Maestro is a Clojure library for Finite State Machines: https://github.com/yogthos/maestro

Malli contracts

Malli looks like a parsing/data structure specification EDSL for Clojure: https://github.com/metosin/malli

to define "The Law of the Graph," providing a high-integrity environment where humans architect and AI agents implement.

Well, beats me. I don't know what is "The Law of the Graph" and Internet doesn't seem to know either. I suppose it tries to say how you can from the processing graph to see that given input data to the ingress of the graph you have high confidence that you will get expected data at the final egress from the graph.

I do think these kind of guardrails can be beneficial for AI agents developing code. I feel that for that application some additional level of redundancy can improve code quality, even if the guards are generated by the AI code to begin with.

yogthos6• 4 months ago

That's mostly correct, one small correction is that cells don't have to be pure. They just have to focus on doing a single task with some hard boundaries.

And what I meant with the law of the graph was simply that the graph defines the actual business logic, and then each cell is a context free component that can be plugged into it. I guess I was just trying to be clever there.

The key benefit I'm finding is that cells can be reasoned about in isolation because they know nothing about one another. You don't have implicit coupling happening the way you do in normal programs that's embedded in the call graph.

My approach is to use inversion of control where the cell gets some context and resources like a db connection, does some work, and produces the result. That gets passed on to the graph layer which inspects the result, and decides what cell to call next.

With this approach you can develop and tests these cells as if they were completely independent programs. The context stays bounded, and the agent doesn't need to know anything about the rest of the application when working on it.

The cells also become reusable, since you arrange them like Lego pieces, and snap them together into different configurations as needed.

flowerbreeze• 5 months ago

I have the same problem. The "What It Is" section starts with "Mycelium is a Clojure workflow framework built on Maestro" and that's a bit generic. Maybe something to test some AI generated code and then test if the tests are tested enough using Closure, but I'm not entirely sure.

The main question that is not obvious, is what should I use it for?

yogthos6• 4 months ago

I'm talking about expressing the application as a state machine and then implementing each step in the state graph as an independent subprogram. The cells accept a state, do some work, and produce a new state. Then the graph orchestrator inspects the state and dispatches to the next appropriate cell.

flowerthoughts• 5 months ago

I've used the analogy of a circular saw before ("it's not really sawing, because you can't feel the wood...")

It's easy to just slab on a Skil saw, cut through the beam and it'll be somewhat straight. But when every manual stroke counts, there's enough time on a human time scale to correct every little mistake. It's definitely possible to become skilled at using the circular saw, but it takes effort that it feels like you don't need at first.

This is similar. LLMs are so powerful for writing code that it's easy to become complacent and forget your role as the engineer using the tool: guaranteeing correctness, security, safety and performance of the end result. When you're not invested in every if-statement, forgetting to check edge cases is really easy to do. And as much as I like Claude writing test cases for me, I also have to ensure the coverage is decent, that the implicit assumptions made about external library code is correct, etc. It takes a lot of effort to do it right. I don't know why Mycelium thinks they invented interfaces for module boundaries, but I'm pretty sure they are still as suceptible to that "0" not behaving as you'd expect, or the empty string being interpreted as "missing." Or the CSG algorithm working, except if your hole edges are incident with some boundary edges.

Edit: spelling

zihotki2• 5 months ago

Your analogy with a Skil saw is genius! You can cut much faster but it's also much more dangerous. Just like the AI indeed.

4b11b4• 5 months ago

I've been thinking like a CNC. It can be made, but the edges are rough, needs sanding. OR - you accept that the edges are borderline dangerous. Also, some complex joints can't be built with a CNC

yogthos6• 4 months ago

I agree with everything you're saying here, and I'm not arguing the approach eliminates the need for the human to be in the loop. That's not the goal.

What I'm trying to do is to create hard context boundaries which help both the human and the agent understand what the code is doing. With Mycelium, you have a graph expressing the top level logic, so you have a declarative workflow that you can review. With traditional code this is mixed in together with the implementation details, and you have to tease out the business logic by reading through the code.

Mycelium creates a hard boundary between implementation details which live in the cells, and the high level business logic of the application. You can set up a workflow manifest which declares what you're doing. Then you can go an implement each step in form of a cell. And then you can review it and test it in isolation without having to consider the entire application.

This is the part that reduces the cognitive load and makes it easier to ensure that the code is doing what's intended.

As a note, I'm not arguing that I invented anything fundamentally new here. Workflow engines have been around for a while. I'm simply applying the idea directly in the context of coding agents.

josefritzishere• 5 months ago

"Too Many requests" message. Hug of death?

Claude built a system in 3 rounds, latent bugs from round 1 exploded in round 3

Related Stories