SKILNER

Thanks for providing some actual facts.

So many people flailing around to justify their feelings rather than thinking rationally.

It is time to give up arguments without evidence.

Disappointing - not a very insightful list

If you've ever undertaken the task of documenting entire workflows, then you know that you quickly put up the white flag at the word "entire".

When you actually talk to people about what they do there are often many, many nuances, micro-events, micro-decisions and micro-actions in their work. This is why it can take days/weeks/months to completely train a new person for a job.

This level of detail is barely documented - anywhere. There is a huge amount of information buried in workflows that AI has barely had access to for training. A lot of this is more in the realm of world models, rather than LLMs.

So imagine trying to use AI to improve these workflows it knows so little about. Then imagine AI trying to reinvent them across an organization.

We find these use cases where AI provides great value - totally true - but these barely scratch the surface of what goes on.

This strikes me as a very solid methodology for improving the results of all AI coding tools. I hope Anthropic, etc take this up.

Rather than converging on optimal code (Occam's Razor for both maintainability and performance) they are just spewing code all over the scene. I've noticed that myself, of course, but this technique helps to magnify and highlight the problem areas.

It makes you wonder how much training material was/is available for code optimization relative to training material for just coding to meet functional requirements. And therefore, what's the relative weight of optimizing code baked into the LLMs.

"According to the study, AI still struggles with several crucial facets of coding: sweeping scopes involving huge codebases, the extended context lengths of millions of lines of code, higher levels of logical complexity, and long-horizon or long-term planning about the structure and design of code to maintain code quality."

uhhh, not sure even the best people or teams are very good at this either. Condemning AI for not being capable of something we're not capable of, ok...

“If it takes longer to explain to the system all the things you want to do and all the details of what you want to do, then all you have is just programming by another name.”

This is called the specification process, which hopefully is already occurring today.

There's so much self-serving bias in articles like this, as well as the comments on HN, Reddit, etc. It's good to critique AI, but that self-serving line is frequently crossed by many people.

> It reeks of cope.

haha, well said, I've got to remember that one. HN is a smelly place when it comes to AI coping.

That's an excellent job of teaching - thank you!

You get to keep your job instead of being replaced by someone willing to use the latest tools.

It's a good idea to think of a converse situation, but this is a bad example. The constraint was not about technology but about budget, perceived benefits and political will.

Let's just imagine we're critiquing cars or planes in about 1910. They're so flawed. No one can say with certainty whether or how they will improve.

Side note of interest, from Wikipedia: "Flying Machines Which Do Not Fly" is an editorial published in the New York Times on October 9, 1903. The article incorrectly predicted it would take one to ten million years for humanity to develop an operating flying machine.

There's a lot of resistance to AI amongst the people in this discussion, which is probably to be expected.

A chunk of the objections indicate people trying to shoehorn in their old way of thinking and working.

I think you have to experiment and develop some new approaches to remove the friction and get the benefit.

Is the UX of the name any indication of the UX of the product?

I think this was more comprehensively described by The (Eight) Laws of Software Evolution: https://en.wikipedia.org/wiki/Lehman%27s_laws_of_software_ev...

This subject always reminds me of something someone said, possibly Professor Alain April, "software is the only system where maintenance tends to degrade it."

Great idea, thanks for doing this.

But - why is the most important information, the title, in such a light, hard to read, font? The title should stand out, not the comments count etc. See... Hackernews! :-)

I don't understand the glee so many people have over this. I love being able to use Generative AI tools. How is it different than if I asked a person to draw these pictures for me? I know someone will gleefully clobber this question with a legal answer, but God, let's move forward, hunh?

I gave a presentation many years ago to a software process group in Silicon Valley entitled, "Unsolved Problems of Software Maintenance". The problems are all still there.

Having done lots of both greenfield and maintenance work over four decades, I can unquestionably say that software maintenance is way, way more difficult.

I heard this first from Joel Spolsky, "it's easier to write a program than read it."

Which led me to write an article on the subject, "How To Read A Program," about the cognitive aspects of trying to learn unfamiliar code. https://www.itjungle.com/2009/01/28/fhg012809-story01/

The article uses AS400 applications as an example, but is general enough that it applies to applications in most languages.

This is a field begging for help from AI.

That's an awesome site. What tools did you use?

What this completely overlooks is the time developers spend trying to understand existing code. For greenfield development that is close to zero. For maintenance work it is over half of their time and arguably the most difficult work in software engineering. Maintenance work in complex systems is far more challenging than developing new complex systems.

Some people, not many, take software maintenance seriously and conduct studies on it - in the study at the link below they point out that comprehending existing code took on average 57% of developer's time. Actual editing consumed 5%.

And software maintenance is no different - think again.

https://www.researchgate.net/publication/318811113_Measuring...

Gray goo - "Self-replicating nanobots are going to cover the world in gray goo." This was the prediction about nanotechnology that has so far not come to pass.

However, these generative AI tools are surely on the path to cover our digital world in gray goo.

The direction seems to be that virtually everything digital will collapse to a zero value.

pearl clutching

Says it all

Haha, me too, I read that sentence and closed the tab.

Headline: Too many Americans living where they choose to live.

"It's so crowded no one goes there any more."

Attributed to old baseball player, Yogi Berra.

I.e., Too many Americans living where they choose to live.

I developed a dynamic analysis/IDE tool for old IBM systems. I was told by VC's that IT management never spends significant money on programmer productivity. I think they're mostly right. Maybe because no one knows how to actually measure it.

Great post - the one word missing, "maintenance." Most programming work is done maintaining/enhancing existing code. The greenfield work is a piece of cake by comparison.

You want to do the hard stuff? Maintain existing code you're not familiar with.

The problems around maintaining unfamiliar code are huge, largely unsolved, expensive and risky. There's a little branch of computer science called Program Comprehension and no one pays any attention. Though most programming money is spent on maintenance.

I think the author is identifying problems. Problems most people just take for granted. I give the author credit for that.

Regular people have their regular jobs to do. They want to use tools, not build them.

I'm all into PowerApps and all that, but no-code for anything the least bit sizable or complex is a fantasy from citizen developers.

Someone else mentioned Strunk and White's Elements of Style, but in the true spirit of that book let me just cut to the chase and say, "omit needless words."

Our brains only have so much space in them so you only get to cram a small number of words into someone else's brain; make them the most productive words possible.

When writing, it's harder to remove words than write them.