agentultra

You should check out Borges’, The Library of Babel.

Basically if you have an infinite string of characters in some language then you have every possible book, tweet, script, scribble, and scrap of everything ever written. You just have to find it.

Thats already a large universe to cover. Programs? Much bigger set.

This seems to be grasping for analogies to make sense of the work this person was doing.

A compiler does a lot more than source code translation.

There are specifications that tell us the de jure specifications of the language, if there is one, and then we have to recognize the de facto implementations of said language. The often disagree and leave much on the table. Some times on purpose, such as implementation details, and other times by omission.

Users of this compiler expect a deterministic compilation of the source text into the target code but it’s rarely 1:1. There are optimization passes, inlining, barriers, etc.

And then there are the run-time effects of executing a program!

I think the analogy gets a little weak because natural language is not a precise enough language to specify discrete systems.

What it sounds like the author is doing is bypassing decision points with other people and delaying making decisions themselves until the LLM agent forces them to? Which is a fine approach but I don’t think the analogy with a compiler is necessary.

People seem to have a hard enough time understanding branch prediction and thread barriers.

That will depend on your life goals.

There are cabinet makers who still make pieces by hand with hand tools. They charge whatever they want and do well. Not everything is Ikea.

It's not hard to observe what the tools and systems can do. It's impressive. But I suspect for some the trade-off is not worth it based on their principles. No amount of disbelief or encouragement will change that.

You still need an expert human to drive an LLM. And even then, with C, you need to be very careful. LLM’s generate lots of code. Code that has no obvious errors in it. The kind humans aren’t good at picking up on their own. You need other tooling in place to support human experts to prevent this kind of stuff as you would in a non-LLM contribution.

I always have to ask people, correct with respect to what?

The reason being that most software developers aren’t very good at specifying systems precisely enough that they can confidently determine whether their program does what they think it does.

We don’t need to specify systems to the rigour of mathematical proofs every time, of course. The point is that spoken language isn’t precise enough to measure correctness against.

When you suggest that the difference in correctness is huge, I can’t really understand what you mean. Correct with respect to how you think good software should be written? Correct with respect to informal specifications written in prose? How?

I know that the frontier models have large contexts and can compress a lot more tokens from their training sets. But how does that make it able to read your mind and make formal what is ambiguous or informal?

Wild idea: just write code.

I know. It’s an unbelievable concept in this AI era. Write code? Isn’t that what dinosaurs did?

If you expect that a human will need to read and maintain that code you might as well write it for them. You’ll get annoyed by having to read overly-verbose copy-pasted code. So will they. So write the code yourself and bringo: you’ll fix things yourself and write things in a way that makes sense for other humans to maintain.

Or you can come up with a convoluted web of markdown files to try and coax your agents and loops to understand what future human maintainers will expect the code to look like.

I’m not sure what path will be easier in the long run. Anyone inherit a loop-based agent-driven code base yet and have to try to understand it?

I learned a while ago that eMacs is more like a programming environment that has a text editor built in. Sort of like some SmallTalk images. When you think of it this way it’s pretty neat.

Coding does not inherently have the “precision” of mathematics. If you’ve ever tried to formally verify even a basic algorithm you’ll know how “loose” programs are with their specifications.

If you end up down this route you’ll gain an appreciation for a branch of mathematics that has spent most of its history maligned by the wider community.

I too finally started grokking trigonometry and calculus in high school thanks much due to programming.

I don’t think it’s the best entry point for mathematics though. Us programmers tend to bias its effect on our learning and appreciation. For most people programming is tedious, cryptic, and frustrating. It doesn’t aid understanding mathematics if you can’t even use it.

Maths is beautiful on its own. And so is programming.

I think it’s still worthwhile because for all that these systems can do it still takes an experienced human to drive them. Any positive results are due to a human understanding the training data, the system, and importantly the output. You can’t one-shot a production grade C compiler or OS with these tools and never will be able to without over fitting the model. You need to know what a production grade C compiler requires in order to generate one using tools like this.

So keep learning. Mainly because it suits you, benefits you, and you like doing it.

Well the other thing is that agricultural uses are generally on a different scale and use different sources. And at least you can eat almonds and alfalfa. Probably more useful than asking an LLM to autocomplete your homework. Depending on your needs.

Even amongst municipal water usage, it seems hard to figure out how much DC usage factors in to other uses because often they’re not required, and therefore don’t, disclose their usage.

All I’m saying is that it’s not a closed case and not everyone complaining about DC water usage is a Russian propaganda bot.

Breaking it down this way is a great way to minimize the numbers so that it appears reasonable.

See? Middle-Eastern investors are growing alfalfa in the western desert using legal allotments of water! That is so much worse than what we’re doing! Go after them!

They can both be using an egregious amount of water for silly purposes.

The other part of the water debate is also the pollution different systems create. Many data centres went in with the promise of closed-loop systems but changed half-way through construction and couldn’t be stopped.

I think it’s more complicated than, “they’re wrong, it’s just hype.”

“The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads.”

I disagree. If you ask a model for a manual and it regurgitates that manual from its training data, it’s over-fitted. It will regurgitate something that looks like a training manual. Or whatever fits your query about training manuals.

You still have to push back on them sometimes when you spot an error. And you can only spot them if you already know what you’re looking for and should expect. Otherwise you have to ignore the output and just get the links which… could be outdated or made up as well. You’ll never know until you verify the results.

And this degrades with compression and time.

There’s no royal road. I agree that trying and getting frustrated and having to take the effort to understand something pays off in spades. I just think it’s still worth it and vastly under appreciated in this era of “everything fast, now.”

PBT is great and often good enough.

I use model checking mostly in the design phase of a project.

Some much bigger projects at MS and Amazon use it continuously throughout their process.

If you are willing to relax the restrictions, and you probably should, model checking is probably worth its weight in gold for these scenarios.

You won’t get proofs but you will spell out your logic in a formal language[0] and each run of the checker will exhaustively check your invariants[1].

[0] Useful because often you will learn something you hadn’t considered.

[1] A proof will guarantee your statements hold over quantifiers that are much too large for a model checker to check exhaustively. But, you can say that for a model of size N, property Y is guaranteed to hold. The “small model theorem,” posits that if there is an error in your specification, it is more likely to show up in a small model. You sacrifice the completeness of proofs but this trade-off has been worth it to me.

You can’t vibe code a production capable C compiler if you’ve never written one.

Sure, companies are asking for LLM experience. But whether they know it or not they are also hiring someone who knows what they’re doing with the output.

Projects like this are still worth doing by hand. I’d dare say it’s even enjoyable.

Alienation is inherent to the system.

For sure. Just because the studies are incomplete or difficult doesn’t mean they’re useless. We still do unit testing and type systems continue to get more sophisticated and spread further because we believe they have an effect on quality and productivity regardless of the lack of evidence.

However it takes some taste in engineering and perhaps some mathematical sophistication to figure these things out. “Just use AI,” is not a very convincing argument either.

It’ll take time to sort out, I wager.

A lot of people believe that. Not a lot of evidence on the table for it (it’s not agent developers’ fault; empirical studies are expensive and rarely live up to scrutiny). Not sure it’s worth forcing people unless you like malicious compliance.

Doubt. People brought in all kinds of web applications in the early Web 2.0 era because corporate IT was being too stingy (for a lot of reasons). People will find efficiencies on their job on their own. No need to denigrate them.

* whether you’re protected from hackers or data breaches*

Not a matter of if, but when, a breach happens.

Doubt that. If you couldn’t write it yourself you can’t get an LLM to do it for you. So unless you’re attracting the 50 best software developers you’re getting middling to miserable results.

You can’t beat thermodynamics no matter how hard you try.

This shouldn’t be a surprised to the majority of people.

Exciting stuff, especially the tree-sitter changes.

Emacs is a fine weapon. And one that grows with you over the years of honing it to perfection. Nothing else compares.

Every line of code is a liability. A potential security breach. An error waiting to corrupt customer data. A maintenance burden.

I’ve had to use one of these to book something with a service company. It was horrible. It’s like talking to Pinnochio… there’s nothing there. And it’s trying to sound human and conversational. It’s creepy and annoying.

Just give me a human being or a plain voice menu.

Businesses just don’t want to pay people if they can help it. Some things are inefficient. Get over it.

Can’t we value both?

If we consider that “attractive code,” to many means simple, elegant… and therefore easy to extend, maintain, few errors, performant, etc.

Again, they didn’t smash the machines because they hated the technology or wanted everyone to be making lace by hand.

They were arguing for basic human rights in the workplace. Things like child labour were still super common and were among the practices the Luddites wanted to abolish. Along with the workhouses they wanted to replace with protection for workers (they didn’t have the word for it but they wanted a social security system).

They smashed the machines for leverage. There was little labour law at the time. Most of it was written by the capital holders with the help of the constabulary. Things like showing up to work on time or no pay, etc. Violence, controlled violence, was the tool they used to try and get the capital holders to the table and negotiate.

It failed, as we know, and it was a bloody failure. People were executed and jailed. The movement became a pejorative for someone who is backwards and against technology and progress.

I feel like the navel-gazing-ivory-tower programmer is almost a straw man used by commenters and bloggers to make themselves sound pragmatic. Summoned only be be torn down. Never to be found on an existing software team.

I have come across the architecture astronaut before. But I feel like they’re the result of the culture of the ecosystem the language. The Java and C# programmers whose language requires you to juggle weak types with visibility keywords and null ability. They can be forgiven for not being able to implement a priority queue without a committee and a class hierarchy deeper than the Mariana Trench.

But the perfectionist that never ships anything useful and only ever tweaks interfaces and types? Never met one.

Most people are just trying to balance progress with practical concerns.

I dunno about everyone, but I do. It drives me up the wall when software is slow, stutters, etc. There’s almost no good excuse for it. But it happens anyway because of some combination of, “nobody cares,” and, “money.”

I derive joy out of people using a thing i helped design and they never get frustrated with it. It works, it’s fast, it solves the problem and it’s rarely a pain to maintain or extend for me or my team. Thats good, solid stuff right there.