logicprog

This seems like a really really great debut model for a new lab. I'm happy

No, they don't. They're more interested in pathologizing and talking down to and about me.

"It's not an issue of usual vs unusual, I'm saying the models are way better at writing and maintaining django or react code bases than your own hand rolled architecture that you define in some docs that it has to learn and keep in context. All of the models do an amazing job making local edits and working in small greenfield projects but once you get to full production systems with close to a million lines things start to rot. The code still works and the models are able to make progress but all of a sudden you have 3-4 different versions of your concepts sprinkled in random corners of your code base.

My second point is that the models are way better at things like Rust or Lua than Python or JS, because the average person producing code in those languages has way more programming experience"

Its hard not to feel insulted here. "Projects using AI like this don't exist, and if they do, they're broken and the people who made them have psychosis." However, although I haven't tested my projects cross machine, or cross platform yet (my editor only works on MacOS because it uses kqueue), I would be happy to take an unedited video demonstration of both, I assure you they both run.

Feel free to peruse the repos! I follow compression-oriented programming, so there are places where the code is "insufficiently" DRY intentionally, as I feel out the precise use cases, and I explicitly made some other decisions (like a God editor object) for meta design reasons (I want it to be easy to modify the editor state from anywhere, just like in Emacs), so all I ask is that you review these without being excessively punative to prove your point.

https://github.com/alexispurslane/cursed/

https://github.com/alexispurslane/industrialworld/

Feel free to look at my past projects, prior to me starting to use AI (which started with gemini-ocr) to assess my baseline skill level.

The GP was just talking about not using common languages like TypeScript and frameworks like React, which is a bar of unusual that seems arguably far lower than mine, especially since I literally defined special custom DSLs with unusual semantics, and used libraries like SDL_gpu with little to no pre-existing training data. So this seems like shifting the goalposts.

Also, what would be highly unusual to you?

That's fair, I think my point is there's a nuanced theory behind my choice to be rude, which is essentially "you need to experiment more, be more open minded, really work at it, and then you'll see what others are seeing, it isn't impossible." It's like Dark Souls, nobody can beat it for you, or teach you how to use dodge rolls, you've gotta build that skill yourself, and blaming the tools won't help.

In my experience, because as you say, the flexibility of these tools and how you engage with them is so broad, and how much the nuances of thay engagement can often matter, the only way to teach someone how to do what you do with them is to sit down and pair a program with them. Barring that, any instructions you could give would massively under-determined what they actually try as a result of your instructions, and so it ends up being an infinite loop of them just coming back and saying "it doesn't work." So in the end, people really just have to discover how these things work for themselves.

I haven't hit nearly a million lines, that's true, these projects are "only" 20k and 45k lines of code or so, respectively; however, I do have directed acyclic graphs of worker agents, managed asynchronously by the agent I chat directly with, do huge architectural refactors (the most recent, was to update everything in the codebase that used the sychronous Lua standard library `io` functions to use the asynchronous, multithreaded IO system I created for the editor — at least, everything that could use the new library, which I kept deliberately simple in a few ways, and everything that wasn't already temporary, but you know) and that's turned out quite well, and at least for me, 45k sloc is nothing to sneeze at, especially since with a green field project there's less examples to keep them on track. I do regularly check in to make sure it's using my design to the fullest, as well.

I do highly unusual, off the beaten path projects with AI.

For instance, a game engine written entirely in LuaJIT, but allocating almost all data structures using CFFI, a very unusual, custom object-orientation DSL I designed myself, and that uses SDL3's SDL_gpu library to do all rendering, a ton of GLSL 4.6 compute shaders cross compiled to SPIR-V and Metal (which constraints what GLSL can be written), and is a hybrid tile-oriented and 3D engine.

Another example is my Emacs-like live-modifiable image-based editor, again in LuaJIT+CFFI, with a kqueue+atomic lock-free ring buffer+pthread based actor model, each running a separate LuaJIT interpreter, with a custom advicing system, event bus, async/await implementation riding on the event bus, a custom process management and IO library, and testing via Tmux.

Even DeepSeek V4 Flash mostly does fine with this, once it's built a few memories in the memory system. Let alone something like DSv4-Pro or GLM 5.2. All I can say is skill issue.

Do you have any evidence for Valkey "widely replacing" Redis, instead of just rapidly gaining adoption as well? Additionally, it's important to clarify that the replacement was 100% over licensing issues — iirc predating the vibe coding entirely — not the use of AI to code, and that furthermore, no one has to my knowledge pointed to any flaws in the Redis codebase even correlated with AI use. So this is a disingenuous framing.

I thought arenas were one of the simplest and most easy to deal with and conceptualize memory management strategies around. Arguably easy, even easier to understand and just as easy to manage as GC. Did they do something special?

DSv4 is nearly in the 2t range, but yes you're generally right

I'm saying Claude didn't introduce any more bugs than the human maintainers of rsync had in the past, not that it introduced no bugs at all. That "arbitrary amount" is the historical amount of bugs. That's why I'm confused. You're completely missing my point and talking past me. Yes, Claude may have introduced a bug. That doesn't change that it doesn't really matter.

Additionally, I quoted Tridge in response to a comment about an increase in changes to rsync, not in response to the person pointing at one bug Claude introduced. If you actually looked at the thread, you'd see that. I didn't deny the Claude introduced bug at all.

The data literally shows there aren't, there have been worse releases before. In what way did I manipulate the data?

That one was on me. I always mess that up.

Whether commits decrease the sev/10c depends on if there are a lot of small commits increasing the demoninator. In reality, we have the opposite: the post-Claude releases have way fewer commits than the pre-Claude ones.

Thus, if anything their sev/10c is inflated. If I changed it to lines of code changed, the relative bug ratios would be much smaller, and the conclusion wouldn't change. In fact, the conclusion would look "better" for Claude; if I was using "mental gymnastics" to come to this conclusion, I would have already used a metric other than adjusting per commits!

What different metric would you suggest that would change the conclusion?

Showing "humility", as you so moralistically and condescendingly put it, would require being wrong first.

I'll do that when I get a chance

The statistical methodology I used is mine. As is the interpretation. Completely. To the degree that I misunderstood statistics (and it is under debate even in the thread you link, and the people accusing me of misunderstanding statistics there are universally misrepresenting my point, which is to point out a total absence of evidence for any difference, not to prove the null hypothesis) that's on me

That doesn't make any sense, what?

Okay, so let me get this straight. Because I used AI to, among other things, write the prose of the original draft of this article, all of the days of effort researching and carefully thinking through the metrics I would use, and the methodologies to analyze them, and rewriting the entire analysis multiple times from scratch based on specifically asking people in my life who are qualified in statistics what I should do and trying to come up with the fairest analysis I could with the little data available doesn't matter at all? The post just have some sort of essential AI nature that makes it low effort just because one aspect of it didn't have sufficient effort put into it for you personally?

And once it's originally posted, it doesn't matter the great extent I go to address metrics and methodological critiques in order to ensure that the data is as robust and helpful as possible. And the effort in writing and refining my prose and the organization of the report in response to people's complaints and criticisms because I do value their time. And when people told me the AI prose was bad, I spent two hours to to make sure that it was something people would want to read, that doesn't matter at all? It's only the original intention that matters. So you just have this arbitrary cutoff point for what counts towards my intentions in the post and my character. No allowance for learning or adaptation, and the fact that I'm clearly committed to putting a lot of effort into making this something that is useful and pleasant to read for people, I just didn't do it for the first draft originally, doesn't matter, only the original version matters?

And more than that, you're not going to actually deal with the substance of the issue, the actual calculations and methodology and conclusions that I came to, instead, the only semi-substantive critique you're going to make of the post is to tone police me and dance around the real issues, as if you're afraid of ever touching them?

The best argument you could make that my bias actually influenced my conclusions would be to point into the methodology and metrics where I did that. I made it all extremely open and transparent and auditable both by describing it in extreme detail in the post and by providing all of my source code and the ability to build the database it runs on from scratch. If there was an actual flaw or bias that my intentions going into this created your biggest possible Smackdown, your best weapon in your arsenal would be to actually point that out. But instead, again, you're just tone policing me. but a polemical style in the presentation of an objective statistical analysis does not in the least undercut its accuracy. Have you considered that my polemic became so fiery, in fact, precisely because I ran the tests and found how non-existent the evidence was for this outrage and that's what made me angry? No, you didn't because you saw some words that hurt your feelings and now you won't listen to facts.

Feel free to run it and find out. I don't think it would produce very much useful information though

In what way did it create more bugs? It literally doesn't show up in the data. What are you talking about?

If they're the statistical methods and metrics hold up, or they don't. Also, if you don't want to read my opinion on things, then just grab the GitHub repo and run the end-to-end replication and look at the output data yourself.

Why was v3.4.1 the most buggy, right before the Claude commits? Why did "nobody notice"? It's way to strange to just say welp, it must be human error.

Why wouldn't it be except question begging priors assuming it couldn't be?

Why does v3.4.2 have 0 bugs, or 0 bug score. And why was such an outlier (no other commit seemingly has this??) allowed to mix into aggregate statistics and bring all the "is Claude buggy?" scores down.

My original metrics which didn't filter out feature requests and questions had it at four bugs and prior to that it was even higher and it didn't make much of a difference to the overall analysis (fell well within the IQR, the lower end of it too). Also, removing one outlier just because it looks kind of funny to you, especially when we only have two Claude releases at all, would be worse in my opinion and more arbitrary.

Tridge already explains this:

"Like many developers of open source packages I’ve been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis).

As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much more thorough test suites, code coverage analysis, CI testing on a lot more platforms, deliberate and thorough scanning for possible security issues (so I find at least some of them before other people!) and the addition of a whole lot of defence-in-depth hardening techniques. This is all a huge amount of work. "

https://medium.com/@tridge60/rsync-and-outrage-d9849599e5a0

Also.

This statement is honestly so ridiculous that I felt it didn't warrant a direct response, but here's one anyway: AI enthusiasts have been proudly proclaiming for literal years that AI makes them 10x as productive based on cherry-picked anecdotes with zero empirical evidence to back it up.

Let's go back to remedial classes on this one.

"I have found that [tool] has made me more effective" is what we call lived experience. It is an "i" statement communicating something about the person’s life. It does not require evidence by default, and you are a crazy person if you call bullshit without good reason, because many "I" statements are epistemically justified in ways that can't be empirically demonstrated or require tacit knowledge.

"[tool] has been buggier since [change]" is a falsifiable claim; you need to actually provide evidence for believing it, and what I'm showing is literally that there isn't any.

This statement is honestly so ridiculous that I felt it didn't warrant a direct response, but here's one anyway: AI enthusiasts have been proudly proclaiming for literal years that AI makes them 10x as productive based on cherry-picked anecdotes with zero empirical evidence to back it up. It's way, way too late to claim hypocrisy here. As I stated under the original submission about this topic, irrational anti-AI behavior is usually just an equal and opposite reaction to irrational pro-AI behavior.

I'm talking about the double standard on the anti-AI side about what evidence should count, not some vague industry-wide epistemic standard, whatever that means. I'm aware LinkedIn Lunatics and Steve Yegge are also being crazy. And it seems to me that even your response here is engaging in a bit of a double standard, or something akin to it, in that you think the irrational anti-AI behavior should be given a pass — and the conclusions perhaps even taken seriously — just because pro-AI people did it too.

And that doesn't help. If anything, editing the AI output to make it read less like blatant slop just comes off as deceptive, like you're trying to hide the fact that the analysis was AI generated.

Okay, so, if I don't spend the time to write everything myself, that's bad because it's AI slop. If I do rewrite everything myself, then it's evidence of deceptiveness... despite being asked by multiple people to do that, and being extremely explicit about my methods and process and the commit history being (as you've shown), very public.

Also, the AI-generatedness of the text doesn't mean the analysis is AI generated, in terms of what was actually done. That's a category error.

Looking at the commits, you were adding more AI generated text less than 2 hours ago[0] before quickly editing out one of the most blatantly sloppy sentences I've ever read[1].

The second commit literally says that that was my prose it was fucking with by adding slop. It's just that me adding my prose, and it adding slop to it, were in the same previous commit. Additionally, my process is often giving it exactly what I want to say, more or less, and having it HTML-format it and insert the templated numbers and UI widgets around that text.

But again, even if I'm spending the time to read through and edit everything it's writing to de-slop it, then I'm clearly also reading it through enough to make sure the analysis makes sense, and is accurate; how is that not enough "effort" for you, if effort is supposed to be a proxy for verification?

Even if we ignore the bias clearly on display there, the premise alone is enough to dismiss the entire thing as heavily biased and chasing a pre-determined conclusion - of course someone who is so dependent and trustful of AI that they decide such an analysis on the bugginess of AI code should itself be written by AI is going to steer the conclusion towards "actually AI code is good and you luddites are overreacting".

That's not ignoring the bias, that's literally restating that you think the bias is there. But if you really think that my bias meaningfully "steered the results," then show me how that happened. Tell me how you would've proven the Claude releases were meaningfully worse, or unusual, at all, or how the methods I chose biased the data against that result, or literally anything except shifting the goalposts and using accusations of "bias" as a get-out-of-jail-free-card.

The entire concept is so tone-deaf that failing to notice it or predict the criticism before publishing is enough to prove the bias.

And you're so committed to your preconceived notions that anything made with AI must be bad, wrong, or not worth your time, that you'll spend your entire time begging the question ("it's made with AI, therefore it's wrong") and shifting the goalposts instead of engaging meaningfully.

Also, I certainly predicted the criticism (in general, anyway, to the fact that it was made with AI; not the prose being AI) but I made it this way anyway, because if someone is so AI-blinded that they can't read and evaluate the actual metrics, methodology, and provide meaningful criticism to it, and instead can only see that it was made with AI, and they're so it doesn't matter.

Nothing you have said makes the analysis wrong. At this point, you're essentially just resorting to ad homenem and begging the question.

Edited that claim, and made several clarifications elsewhere. The whole point of this analysis is that outrage is unjustified on the basis of two totally statistically unremarkable releases that no one would have remarked on pre-AI (my further proof of this is that there was a pre-AI remarkably broken release, and no one did comment!) and zero positive evidence outside cherry-picked anecdotes for any negative impact. We should wait for outrage and version pinning and cancelation until there is evidence, no? I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently; I'm not trying to build any kind of predictive model for future Claude releases to say anything grander than "these specific releases are fine, what are we freaking out about?", not some claim about what Claude-exposed releases will look like or trend like in the future or in general.

Fair point. Let me edit (if I still can) to tone it down.