Programming AIs worry me

https://buttondown.email/hillelwayne/archive/programming-ais-worry-me/

by azhenley • 3 years ago

137 181 3 years ago

buttondown.email

fassssst• 3 years ago

Train a separate AI to write tests against the other AI’s code, that’s rewarded for finding defects :p

suzzer992• 3 years ago

AI: "This is a bug"

Human: "No it's not"

AI: "I won't hurt you unless you hurt me first."

buildbot• 3 years ago

Humans may he bad at proofreading code, but we’ve had a lot of experience mitigating that too - what else are tests for? Maybe coding AIs will push everyone into test driven development - write a spec/tests of whatever then just wait awhile for it to crunch through code until something works.

rom-antics• 3 years ago

That's an interesting idea, it sounds almost like using a fuzzer for generating new code instead of finding bugs in existing code.

nathanwh2• 3 years ago

Until someone makes the AI write the test too, I don't think there's going to be any red-tape style solutions to this problem that isn't just "no AI written code allowed".

sharemywin• 3 years ago

Actually, it probably could write really good test cases. they tend to be boring and verbose.

It's only test data but you get the point

Sure, here are some examples of phone numbers in various formats:

(123) 456-7890 123-456-7890 +1 (123) 456-7890 1-800-123-4567 555.123.4567 123.456.7890 1234567890 (123) 456-7890 ext. 123 +44 123 456 7890 (international format) 011 44 123 456 7890 (international access code format) 123/456-7890 123\456-7890 (123) 456-7890 x1234 (with extension) 123-456-7890 x1234 (with extension) 1 (123) 456-7890 (with country code)

jart2• 3 years ago

What's more likely to happen is "no human written code allowed". I could easily imagine a future where people are socially ostracized for writing code manually because it's dangerous, unsafe, and insecure. In that future, Rust developers will get a taste of how they made C / C++ programmers feel.

candiddevmike• 3 years ago

Iteration via adding more test cases... Sounds like hell, I'd find a new job.

kibwen• 3 years ago

This still assumes that the root of our modern ills as a profession is "we don't have enough code", when in fact the root of our ills is that "the code we have is all crap". Being able to generate code that is worse than ever, faster than ever, will not result in any improvement at all to the state of software reliability.

sublinear• 3 years ago

Extremely well put!

abudabi123• 3 years ago

Frog march the code through a valley of death such that only reliable software survives at the other end. Use Ai to automate the simulation.

baby5• 3 years ago

heh, this was true for stackoverflow or for googling pretty much anything. Nothing new. The tools are not doing our job, they're just making the job easier.

fragsworth• 3 years ago

Your head is in the sand. Some time soon, they can and will actually do the whole job.

jarjoura4• 3 years ago

Nah, never the whole job. Software engineering isn't 100% writing code, so even if an AI could write 100% of the code, there's still all of the other things around it that need to get done.

I am actually super excited for better AI tools to help write so much of the boilerplate crap I have to write. I'm here because I want to solve problems, not write code, and writing less code makes me happy.

ResearchCode• 3 years ago

It looks like it will be able to do every non-programming office job first.

l33t2333723• 3 years ago

By then they should also be able to do or make redundant 90% of jobs in general.

If the AI can make the entire product then it can probably market the product, provision resources, handle accounting, user support, etc.

At that point we’ve cracked capitalism and our ability to create has far surpassed our ability to consume.

whartung2• 3 years ago

Human beings struggle to communicate system requirements to other human beings. We're supposed to be "smart", communication is supposed to be something we're good at.

Getting requirements to an AI is going to be grotesquely challenging, and even when we're able to get that far, testing the work is going to be an ever greater challenge.

"Exactly what I asked for, but not what I wanted" is going to rise to an entirely new level.

goatlover2• 3 years ago

This claim has been made many times before.

jejeyyy77• 3 years ago

as someone who has used copilot/chatgpt since they became available, I must say we are still a ways off.

Mainly, as others have pointed out, only a small % of coding is typing out code.

ostenning• 3 years ago

They will allow people to increase efficiency and thereby lowering the amount of time and cost for a developer. Soon complex web platform development will be like building a wordpress website, you’ll have 10 concurrent clients or projects rather than just one.

baby5• 3 years ago

In the book zero to one thiel makes the case that the next boom is not in replacing humans, it’s in assisting humans. IMO he was right a long time ago (not sure when this book was written) and continues to be right to this day. I’m not saying AGI is not coming, but this isn’t it just yet.

imtringued4• 3 years ago

Your head is in the sand.

You know what "whole job" means, right?

pixl972• 3 years ago

Heh, I should develop a tools searches for the first SO response with code and then check to see how often that response contains a security flaw.

brundolf2• 3 years ago

I think the economic reality is that for a lot of software, especially user-facing software, "correct enough that it passed manual QA" is the threshold. It already is; even pre-AI. So I think AI + proofreading will be successful there, and it may not even make things worse than they currently are, especially in the long run

While critical logic, that really really has to be correct, to a point where you should maybe have it all planned out before even writing code, will still need to be hand-programmed for a long time

aardvark179• 3 years ago

I think the problem is that code that passes manual QA testing is fine until it suddenly isn’t. That may be due to the complexity of the code base, the turn around time on doing those tests, the accuracy of people following scripts, or just the number of tests and the size of the code base.

Similarly integration tests are fine until they suddenly aren’t, and you really need things like unit testing.

Unit testing also only works up to a point without tools to reason about the entire codebase and how it interfaces with others.

Organizations flourish or flounder based on recognizing when they are approaching those points and dealing with their underlying issues. Maybe AIs can write code that is good enough in a lot of cases, but unless people can understand that code and how to work with it and refactor it as things grow then they may not be a good idea.

brundolf2• 3 years ago

All of those things are true, and yet, that's how a lot of software is built in practice because it still ends up "close enough". The market has shown that this is acceptable in a whole lot of domains

I'm not saying I like it, I'm saying it's the reality

jarjoura4• 3 years ago

Just like a human, don't use these new AI tools to read your mind.

I've found ChatGPT to be impressively helpful at knowing where to continue to look for something. I can write a sentence and explain what I'm looking for, and maybe even paste in a code snippet, and 10 seconds later I get enough information to go back to Google and continue digging deeper.

For me it's become a bridge tool. Instead of spending an hour trying to get Google to understand my confusion and endless ads thrown my way, I get a very concise answer with full definitions in case I needed to know. It understands me, which is the most important part.

The code examples it spits out are meh, and likely always going to be error prone, but I cannot imagine it won't improve over time.

pixl972• 3 years ago

Google to understand my confusion and endless ads thrown my way,

Another reason why Google is terrified here. Search is per query cheap. GPT isn't.

dybber• 3 years ago

I guess many companies will pay for Github Co-pilot, ChatGPT style things.

No one will pay for access to a search engine.

fragmede3• 3 years ago

If it were down to cost per query, they wouldn't have implemented search-as-you-type, which is probably 5x as expensive as waiting for the user to hit the enter button.

moremetadata• 3 years ago

ChatGPT/Bing is a superior rival to Google's I'm feeling Lucky button, thats a threat to Google's search engine dominance.

kitsunesoba• 3 years ago

It's like the friendly, patient forum poster who answers your questions and explains things in a "how to make a peanut butter sandwich" manner we all wished we had but are hard to come across.

chasd00• 3 years ago

I like that! A “friendly, patient forum poster” is a great analogy.

baby5• 3 years ago

This makes me think that what’s really missing from these new AI is a response that asks for more context, instead of directly attempting to give you a correct answer. Often my questions don’t have enough contexts to be answered correctly but the AI never tells me that

hoot• 3 years ago

Tell ChatGPT that you want it to ask for more context when it's needed and it will do so. I've done this before and it will continue to ask more probing questions until it's satisfied. You kind of have to guide it a little bit but it definitely works.

AuryGlenz• 3 years ago

I was asking it some medical related questions today and I had the same thought. “What are some medical conditions that could cause constant fatigue, a weak immune system, and circulation issues resembling Reynauld’s syndrome?”

To which is could reply “Does the patient have skin lesions or joint pain?” to eliminate possibilities.

ilaksh• 3 years ago

I'm working on an "AI Programmer" website and have attempted to do a requirements interview like that to get more context. Hoping to get the new release out within a week.

gonzo41• 3 years ago

This is exactly how I use it now. It's like a custom SO helper. ChatGPT has read all the doc's. If you can ask the right question it'll give you a good boost.

But the building, that falls back on the engineer.

chaosbolt• 3 years ago

Yeah it's great as an assistant to a critical thinking human, people who are criticizing it are either doing it for clickbait aka money or expected it go think for them.

Like it showed me how to split an audio file into parts with ffmpeg, I'm sure I knew it at some point but it took me less time than reading the -h or looking for an answer on stackoverflow, especially with google's greed that it stopped showing stackoverflow first and sometimes leaves it to the 2nd page and fills the first with garbage, started using yandex as an alternative to google a year ago, now using it by default and google from time to time, it's sad that the best product is behind us.

grugagag2• 3 years ago

I think people are not criticising the good uses but the potentially really bad ones. While Im welcoming a human helper artificial agent I am uneasy about the upcoming general downgrading of human labor. Big money will benefit from this more than anybody else

imtringued4• 3 years ago

Yeah it's great as an assistant to a critical thinking human, people who are criticizing it are either doing it for clickbait aka money or expected it go think for them.

You forgot the hyped up people who blindly tell you the latter and thus force you to state its limitations loudly. Every time I see these trolls I worry that I am too stupid to enter the right prompts but no, it then always behaves exactly how I think it should.

bakuninsbart• 3 years ago

When you have two choices and don't really know which one to pick, there's an old trick to throw a coin. Go with the coin, unless you really dislike the outcome, which tells you that the other choice was the right one all along.

I use ChatGPT in a similar manner, for example when writing documentation, I'll ask it: "What are the important topics that should be covered in a documentation on X?" and "Can you give me a suggestion for a structure of such a documentation?". Now I can easily follow along, unless I feel like it is a bad approach.

With coding, it is almost like an improved rubber duck. I don't trust the replies, but the engagement is helping me structure my thoughts.

andrewstuart3• 3 years ago

Relax.

AI assisted programming is brand new.

It's just the start. So if it gets things wrong, is missing features, annoys people, remain cool, gigantic leaps are coming far beyond your wildest dreams.

And there will be lots of people around right now showing how smart they are by making AI do wrong or silly things. Ignore them - none of these prove that AI is the wrong direction.

AI programming is the single most exciting thing to ever happen to programming - a permanent mentor and pair programmer at your side levels up every programmer in the world.

Even more exciting is innovations that are a direct outcome of AI assisted programming.

When the world wide web first came along, everyone made a website that was like a company brochure. That was Web 1.0 and that's all people had the imagination for. In time, the usages of the web started to arrive that leveraged the core values of the technology, such as interconnection between people. That's where AI programming is now.

We are at AI 1.0

Programming models will come out that are primarily oriented to the strengths of AI programming - but first someone needs to think them up. And they will not only be surprising but absolutely mind blowing. It's entirely possible that in the future the entire internet has an AI sublayer that does all the work of systems talking to systems and people simply don't have anything to do with it. It's possible that the nocode movement is the foundation of the next great innovations in programming. Existing programming models are like the company brochures of web 1.0 - our old, hard, human centric way of doing things.

Programming literally should be as easy as talking to the computer and telling it how to shapoe and change the software and it should happen right there in real time on your screen. It should be an iterative, interactive process that anyone can do. It should be more like painting and less like arcane wizardy that can be performed only by the programmer priests who have spent years learning. Even the most advanced software should be capable of being built this way.

In the 1980's I worked for a startup company run by a super smart guy Peter Dunai, who once said "a revolution in programming is coming". I thought I'd seen that revolution already with the Internet, modern programming languages and StackOverflow, but now I realise none of that many anything compared to what the true revolution is going to look like.

AI 2.0 will come and I can't wait.

bombolo• 3 years ago

AI 2.0 will come and I can't wait.

It might happen in a decade or more.

jarjoura4• 3 years ago

I agree for the most part, but I am also intrigued how this will play out on the regulation side of things.

Hollywood loves to treat AI as something that can go rogue, but these tools will have all kinds of safety locks on it and will only give you answers to things its allowed to. Plus, it will be logged and tracked just like everything else.

You never want to openly allow anyone access to an AI bot that can spit out encryption cracking algorithms with ease. Or worse, something that can solve infrastructure bricking paths that could take down important systems. Or an AI bot that can interfere with real-life things, for example, find someone that doesn't want to be found.

So yea, I cannot wait for this to get better and remove all the boilerplate I have to deal with, and let me focus on the parts of the job I actually love.

Fellshard• 3 years ago

How will you know when it's actually working as intended? The issue is /strongly/ epistemological.

nineteen999• 3 years ago

I think this is the main thing that sticks out to me. I think Hillel Wayne is right to point out that it moves the main task of a developer from writing code to proofreading it (to a small degree now, perhaps to a larger degree in the future). Most programmers prefer to write their new code than to read someone else's. So if we are delegating our favourite parts to AI, how long before we ask it to write the unit tests as well? Who's going to validate that the unit tests it writes are correct? I forsee a bunch of frustrated QA teams at least in the (near) future being asked to test sloppy code which has passed a bunch of sloppy unit tests, but has had only minimal review from the developer(s) responsible for it. Perhaps I'm overthinking it.

fleddr• 3 years ago

Nothing you said relaxes me. I like my arcane wizard status.

andrewstuart3• 3 years ago

Me too, but the future is coming to push the druids and the old gods into the shadows.

brobdingnagians• 3 years ago

web started to arrive that leveraged the core values of the technology, such as interconnection between people

A lot of people blame the advanced social media products on the internet for a decline in mental well being and for increasing isolation and lack of human connection in society. That might not be the best example of something we want the AI path to emulate.

notshift• 3 years ago

AI development will not be using no-code technologies. Text is the best form of compression (of meaning/intent/information) for 99% of software and that isn't going to change.

What AI will accomplish is creating an entire new package ecosystem from scratch, written at a low-level so that it can be extremely performant, while also being highly extensible, abstract, intuitive, interoperable with all languages, etc.

Then the code will be able to take human prompts and translate them into business logic code that uses all those packages. For fuzzy domains with low correctness requirements, the AI will be able to make all the changes to the codebase without verification. For domains with higher correctness requirements, humans will review all the code, or you could have the AI first generate a formal specification based on the prompt, have the humans review that, and then have the AI generate code along with a proof of that code's correctness in conformance to the spec.

goatlover2• 3 years ago

Programming literally should be as easy as talking to the computer and telling it how to shapoe and change the software and it should happen right there in real time on your screen.

Kind of like what the intention of Smalltalk was, minus the verbal part. You modify a live image like sculpting clay, except now you have an AI generating code for you. I wonder what Alan Kay thinks of this approach.

rsaarsoo• 3 years ago

[programming] should be an iterative, interactive process that anyone can do. It should be more like painting...

You mean like taking years of rigurous practice to master, requirering specialized equipment and expensive materials plus an education in color theory, composition and art history :)

YeGoblynQueenne• 3 years ago

> AI assisted programming is brand new.

The field that studies the automatic creation of computer programs is called "program synthesis". It is a field that straddles multiple sub-fields of computer science, including AI (where research is usually on inductive program synthesis, from examples).

https://en.wikipedia.org/wiki/Program_synthesis

It is also a very old idea, as old as computer science itself- or even older. Here's wikipedia, again:

*During the Summer Institute of Symbolic Logic at Cornell University in 1957, Alonzo Church defined the problem to synthesize a circuit from mathematical requirements.[2] Even though the work only refers to circuits and not programs, the work is considered to be one of the earliest descriptions of program synthesis and some researchers refer to program synthesis as "Church's Problem".

https://en.wikipedia.org/wiki/Program_synthesis#Origin

Even program synthesis from a natural language specification is not brand new. For example, here is the first example of research work on the subject that I could find on duckduckgo in ~1 second:

https://www.microsoft.com/en-us/research/wp-content/uploads/...

The fact that most people have never heard of all this before is not new, either. What is new is that OpenAI and Microsoft are marketing a program synthesis tool.

andrewstuart3• 3 years ago

Yes.

mr_toad• 3 years ago

It should be more like painting

So, something that takes natural talent, years of practice and a lot of time and patience?

At least programming pays well.

mjb• 3 years ago

It's interesting that Hillel, a huge formal methods and specification advocate, sees this as a proofreading problem and not a specification one. I can't say I agree.

hwayne5• 3 years ago

I'd be interested in hearing more about your disagreement! If I had to reconcile this post with my overall shameless FM promotion, I'd say that "proofreading is hard" is the human limit that is amplified by AI use, and more specification is a potential solution. But also, historically, an expensive one, so I don't know if that's going to adopted enough to fully offset the drawbacks of mass AI adoption.

adversaryIdiot• 3 years ago

I haven't had much luck using these AI's past surface level stuff.

hwayne5• 3 years ago

They are godsent at getting a TikZ drawing jumpstarted.

twawaaay• 3 years ago

Anybody worrying that programmers loose jobs stop worrying.

All the automation in the world does not replace the need for people to debug problems.

Even more interestingly, the more something is automated the more time is spent on solving the issues when they happen. Finding a simple mistake in a very complex automation can be a challenging project.

But... if you have been programming Cobol your entire life you might want to start learning some newer technology.

bryanrasmussen2• 3 years ago

You think the programming AIs are going to be trained on Cobol?

agentwiggles• 3 years ago

I mean, I have no ability to check it for correctness, but ChatGPT is certainly capable of writing Cobol. Just asked it for an implementation of Dijkstra's algorithm and it looks reasonable enough to me.

Why _wouldn't_ the programming AIs be trained on Cobol?

bambax• 3 years ago

Might as well share a wishlist. [1/] A ChatGPT-style AI that can only reply with links to reference docs, libraries, or wikipedia articles

That already exists. It's called an effing search engine. But for some reason people have decided, as of ten days ago, that they're not good enough anymore.

As for why? I really have no clue. But they still work extremely well.

Bjartr3• 3 years ago

As for why? I really have no clue. But they still work extremely well.

I've found Google to be significantly less useful than it used to be. Largely because I have to jump through hoops to convince it to show me what I'm actually looking for. ChatGPT is already much better often enough that it can turn 20+ minutes of faffing about with Google into one minute or less of just asking for what I want and getting it.

Yeah, it's wrong sometimes, but it still manages to point me in the right direction often enough to be worth the need for extra vigilance

mordae• 3 years ago

Hahaha, try asking GPT-3 about electronic engineering.

---

Transcript of IRC channel #electronics:

Mordae: Hi. So I was designing a transformerless LED bulb and I was wondering about the filter capacitor. What happens when the load (one of the LEDs) fail?

Rampant: Hi Mordae. It really depends on the design, but the general idea is that the filter capacitor will help prevent spikes and other issues caused by the failing LED. It can also help with voltage regulation.

---

Actual answer: Light bulb needs a bulky electrolytic cap. Those are usually not 400V tolerant. You are looking at 60V at best at the required sizes. Losing the load will swiftly charge up the capacitor to 325V. But well before that happens, there are going to be fireworks: https://www.youtube.com/watch?v=6WUxgmMDts4

I believe that there are eventually going to be useful mentors for bunch of fields, but right now it's crazy dangerous to blindly follow its recommendations.

whartung2• 3 years ago

I'm sorry, is there enough context in the original question as presented to come up with your "actual answer"? I mean, I don't know, I'm not an EE, lucky I can make a light switch work. But given the dearth of details in the query, is that AI answer grotesquely wrong? Just seems like generic advice.

Dylan16807• 3 years ago

I'm sorry, is there enough context in the original question as presented to come up with your "actual answer"?

I'd say there is. "Transformerless" implies a specific design, especially when you mention there's a filter capacitor.

is that AI answer grotesquely wrong?

Yes. It's fine to say the filter cap helps with "spikes and other issues", but not ones caused by a failing LED. It took half a true sentence and made it into a false sentence. In this design the filter cap is something you give protection to much more than something that provides protection.

Bjartr3• 3 years ago

ChatGPT does a much better job. Not perfect, but at least identifies overcharging the capacitor as a danger (though not fireworks) and provides some possible mitigations to look into (NOT take at face value as definitely correct though, to be sure)

---------------------

I'm designing a transformerless LED bulb and I was wondering about the filter capacitor. What happens when the load (one of the LEDs) fail?

---------------------

When a load (such as one of the LEDs in your transformerless LED bulb) fails or becomes open-circuited, the current through the filter capacitor will drop to zero, and the capacitor will start to charge up to the peak value of the AC voltage from the mains supply.

This can be dangerous because the voltage across the capacitor can quickly rise to a high level, depending on the capacitance value and the AC voltage level. If a person were to touch the exposed conductors of the failed LED bulb, they could receive a potentially lethal electric shock from the charged capacitor.

To mitigate this risk, it's important to include some form of overvoltage protection in your design, such as a discharge resistor or a metal oxide varistor (MOV). The discharge resistor will discharge the capacitor when the load is disconnected, while the MOV will clamp the voltage across the capacitor to a safe level by diverting the excess energy to ground.

It's also important to provide adequate insulation between the live components and any exposed conductors or casings to prevent accidental contact with live parts. Additionally, it's recommended to follow established safety standards and guidelines when designing any electrical product to ensure its safety for users

Tade0• 3 years ago

Out of curiosity I looked up the circuit for that.

Designing a LED power supply as anything else than a current source is asking for trouble.

jmull• 3 years ago

I read this but actually not sure what the author is worried about.

I guess they feel proofreading AI-written code is harder than writing code?

That seems like a leap to me. I don't expect AI-assisted coding to be a panacea, and I'm sure there are ineffective ways to do it. But it seems like there are effective ways too.

So, sure, there is work to do to figure out how to use these tools in the best way. But that's not anything to worry about, IMO. It's opportunity.

seletskiy• 3 years ago

With the new advancements in AI, it's expected that there will be more code being generated by programs rather than written by programmers. Making sure these programs deliver accurate and error free code is a difficult task. The challenge here lies in effectively proofreading AIs so that any bugs are caught before they reach end users. Good tools such as unit tests can help but may not be enough to make sure AI generated code is flawless. As the use of AI increases, we must look to develop better methods for ensuring quality assurance on these programs if they are to truly be useful.

avdempsey• 3 years ago

In my career I've seen resources pulled away from QA teams (sometimes completely disbanding). It would be interesting to see things swing the other way.

zokier• 3 years ago

it's expected that there will be more code being generated by programs rather than written by programmers.

That has probably been case since the 80s; nearly all code running on your CPU is generated by a program ("compiler") from a textual description ("source")

vitiral2• 3 years ago

That's not what is meant. Programming languages are basically huge macro systems that expand to assembly, but the code is still written by a human.

Layers of abstraction are a problem too, but they are a different problem.

Panzer04• 3 years ago

This feels GPT generated.

pphysch2• 3 years ago

The author begins by giving a number of test cases, like f(35)==6, for which the AI provides a candidate f, but the author doesn't actually implement the test cases, is surprised when the AI is wrong, and laments the resulting burden of proofreading.

Just implement those tests, for heaven's sake!

majormajor• 3 years ago

Implementing a correct and comprehensive set of test cases for code is more time consuming and often more difficult than writing the code itself. (The comprehensive part especially - how do you prove to yourself you've covered all the necessary edge cases?)

"Reading code is harder than writing it"

"Debugging code is harder than writing it"

These are pretty common statements, going back years.

They should make the lack of a silver bullet here very obvious.

(Yes, most software is already broken in various ways - my worry is that this will just make it more broken, e.g. non-technical management saying "why do we need to do as much QA, we have AI writing the code now!?")

pphysch2• 3 years ago

Yes, writing good tests is hard.

My point is that he already has the relevant test cases. The hard work is done.

fragmede3• 3 years ago

even easier: ask the LLM for those test cases!

hwayne5• 3 years ago

To clarify, Bertrand Meyer is the one who gave the test cases and accepted the wrong answer without verifying them. I'm the one who lamented the resulting burden of proofreading.

yen223• 3 years ago

Programming AIs worry me because I think they can get better at programming faster than I can get better at programming

qualudeheart• 3 years ago

I once coined an aphorism: “Deepmind can train a new model faster than you can go back to grad school.”

guluarte• 3 years ago

it's like alexa, siri, etc. I'd rather do the stuff myself than talking nonsense with an "ai"

tonyhb• 3 years ago

A ChatGPT-style AI that can only reply with links to reference docs, libraries, or wikipedia articles

This is exactly what we've built at Inngest so that you can learn our SDK: https://www.inngest.com/ai-personalized-documentation.

It can only reply with SDK examples, an explanation, and links to relevant guides. Soon, we're revamping our docs to highlight sections relevant only to your use case - you'll be able to type what you want to do with Inngest and be pointed straight to the relevant guides and docs.

We trust developers to write the code, and only use GPT to assist and point the right way. It's a good compromise, and it's the most immediately applicable use that I can see!

hwayne5• 3 years ago

I know this is getting downvoted for being an ad but this is exactly what I was talking about, so thank you I will look into it!

aib• 3 years ago

What worries me are the low-grade programmers. By delegating their work to tools like ChatGPT (and doing so always a level or two beyond the current abilities of the tools) they will manage to take us to newer lows.

Existenceblinks• 3 years ago

Underrated. It already happened in frontend space. Job market is already rotten by incompetent tech leaders who aren't immune to all the bullshit and bought all the hype from VC-backed bigco.

layer85• 3 years ago

The “almost correct” aspect will be especially fun when AIs are used to refactor code. I was thinking about how a future legacy codebase might look like after a decade of having AI-generated code added to it, and then of course you would want the AI to apply some refactorings from time to time.

One might argue that it won’t be a problem with proper tests. But do coders want to spend most of their time proofreading AI-generated code and taking care of proper test coverage? That doesn’t sound very appealing.

plasticchris• 3 years ago

Why can’t the AI write tests too? Then you can have 100% coverage of your (probably) correct code…

saulpw• 3 years ago

That's even worse, because now the tests are wrong too! I asked it for a list of test cases for a particular function I wanted to write, and every single one of the cases it presented was incorrect. So it was worse than useless, now I had to check its work and then do the work myself anyway.

layer85• 3 years ago

It can, but you have to proofread those too.

noobermin• 3 years ago

Why not generate an AI that can take any code and decide whether it has any errors or not?

ParetoOptimal• 3 years ago

The “almost correct” aspect will be especially fun when AIs are used to refactor code.

Hope your function has all invariants covered with one or more property tests!

layer85• 3 years ago

I’m sure the AI has added all necessary tests like I told it to!

valcron1000• 3 years ago

If reading is harder than writing code, why bother with tools that deal with the later instead of the former?

vitiral2• 3 years ago

My worry is code complexity explosion. Once we start using AI to add features to infrastructure (OS's, compilers, data serialization, etc) it will quickly run away from our ability to understand it. It has already run away, it will now be fucking _flying_.

Verdex3• 3 years ago

I expect that before my career is over that I will be given more than one project to fix that the AI stopped being able to deliver features for.

I imagine it's going to be one of those laugh so you don't cry affairs.

AnimalMuppet• 3 years ago

Maybe laugh because you're going to get paid for a really long time.

(And then, yeah, cry because of what you're going to have to do to earn the money...)

turtleyacht2• 3 years ago

I like it. We're generating "COBOL" debt for our future brethren as we solve present problems with the most modern tools we had.

layer85• 3 years ago

The code might start to resemble biological systems, which are a mess of complexity.

ModernMech• 3 years ago

In the future, programs will develop genetic disorders, and will need to be diagnosed by programmers with the equivalent of medical degrees.

bcrl• 3 years ago

My worry is that AIs reduce the cost for bad actors to engage in harmful behaviour. The companies building these AIs need to take ethics into account. Making it easier to create fake images, video and audio of real people causes harm and should be considered unethical.

We need to strike a balance between the freedom to create new things and the externalization of the cost of the harm these tools inflict upon society. Law makers don't seem to have a clue about the consequences of what's happening, so my expectations are pretty low at this point. Overall, I no longer believe that many new technologies are a net good for society.

vineyardmike2• 3 years ago

There’s a theory I’ve seen in occasional security related things where you publicly publish your private keys when you’re done with them. That way later on you gain some plausible deniability related to any communications and data found later.

When Facebook was new there were stories about people failing interviews because some employer say a picture of them at a bar on Facebook (as if most people don’t go to bars when they’re young). Obviously that doesn’t really happen anymore, but I consider a lot of AI content generation to be the “releasing private keys” for your digital footprint. Was I at that bar or was it stable diffusion?

autoexec• 3 years ago

Making it easier to create fake images, video and audio of real people causes harm and should be considered unethical.

Is photoshop unethical? What should we do to balance the harm it causes to society? Should it be outlawed? What about MS paint, or even word processors? Were harms caused to society by the printing press? Was it all a terrible idea?

AI could create tools that end up doing more harm than good. AI could design literal malware. It's unclear if they'd do a better job of it than we do ourselves, and either way it seems likely that it'll take a human to use the code AI produces (or at the very least to direct the AI to use it against other humans)

The technology is rarely the problem. It's almost always the people who choose to use that technology for evil. However involved AI becomes in what gets created, until AI becomes actually intelligent and the singularity begins it's people, not technology, we need to worry about.

I'm skeptical that faked video and audio will be the end of us. We've had doctored images and impersonators for ages. We've had hoaxes and liars. We'll get smarter and learn to put less trust in random things we find on the internet, just like we're gradually learning not to believe everything we see on TV or that nigerian prices aren't going to make us rich beyond our wildest dreams.

UncleMeat• 3 years ago

Yep. Every line of code in your codebase increases the cost of things like refactors and bug investigations. If developers can write code 10x as fast but only refactor or understand it 2x as fast... well then we have an interesting problem.

tomxor• 3 years ago

100%, Programming is all about trying to keep control, trying to prevent it from becoming chaos, trying to ensure you still understand the program you wrote at the end of the day.

With out doing any of this it's easy, easy not to care... sometimes I do this with throw away code and suddenly it feels like I can fly, I can write thousands of lines so fast, progress is lightning fast, and then it all turns to shit really quickly and the friction of chaos grinds it to a halt - at which point I only validate my normal slow mode of plodding through things carefully and minimally.

bdw5204• 3 years ago

I think AI can replace the "copy everything from Stack Overflow" type of developer but I doubt it is capable of replacing the competent developer. The currently popular form of AI is likely to hit a local maximum just like machine translation has where it is reasonably close most of the time on easy or intermediate problems but sometimes breaks in bizarre ways and doesn't work at all for the hard problems.

Since speeding up the work of incompetent developers by a factor of 100 is a good way to end up with an unmaintainable disaster that will break in bizarre ways that are very hard to fix, I think we probably are going to see complexity balloon way beyond human comprehension in every codebase foolish enough to use it. Just like those incompetent developers create more work for competent developers, AI will create more work for human developers.

BoorishBears• 3 years ago

Have you actually tried using an LM for programming?

ChatGPT catches me off guard sometimes even when working on "boring" problems. For example, I was consuming structs from a UNIX domain socket in Java and had some tedious translation from C-types to ByteBuffer calls.

It's brain dead easy thing to do, the structs were defined as packed so not alignment issues, ByteBuffers handle endianness, there's a little bit of trickiness with unsigned types but nothing terrible... but with 6 types and writing tests for all 6 definitions, I wondered if ChatGPT could do it and save me a couple of minutes.

What impressed me wasn't that it managed to generate the near identical code to my example definition,it was the fact it recognized one of the fields as padding and pre-emptively discarded its value.

The padding wasn't in any way shape or form covered by typing, yet the fact the comment in the C definition mentioned padding caused ChatGPT to generate code that skipped over the padding instead of trying to needlessly capture its value and commented that was doing so.

And when I had it generate unit tests, it managed to maintained context on the padding and added a helpful comment pointing it out.

It's small corner cases like that which really make me see LMs as being the natural productivity multiplier for programmers.

There's a ton of nuance in some problems that by the very nature of an LM, you'd never imagine it'd be able to capture. Yet when you press on it, it actually does better on those than it does on "copy everything from Stack Overflow" problems.

amelius2• 3 years ago

I think we might get AI that can help in understanding systems. We could run Copilot in reverse and generate documentation.

(Some people are already trying this with generation of commit messages)

agentultra3• 3 years ago

It’s a bit of a nihilistic view that programmers are merely slow boiler plate auto completion systems and our jobs will be replaced by AI.

We already write more code than we understand given how lowly the industry regards formal methods and how divided it is even on testing.

It is not surprising to me that Bertrand Meyer could miss that error in the code! A former colleague of mine challenged his grad students to implement an error-free version of binary search. He gave them the specification and verified each one: the majority of the submissions had errors. It’s a marvel that computers and software work so well.

And instead of tackling the problem of programming, as Dijkstra put it, we’re going to automate writing more code. Great.

See you in a few years when we can’t even keep up with the deluge of error prone programs.

Although we have gotten this far with very few proofs and models. Who knows, maybe we’ll just tolerate it?

It would be nice to have code synthesis work on larger examples from deep specifications. And models that understand proof tactics and can speed up the process of formalizing proofs. That would be nice.

sigstoat• 3 years ago

A former colleague of mine challenged his grad students to implement an error-free version of binary search. He gave them the specification and verified each one: the majority of the submissions had errors.

I was in a class on Isabelle where we had to write and prove correct a binary search function. As I recall, everyone admitted that their first function didn’t work.

renewiltord• 3 years ago

Java's was broken till '06

jasfi3• 3 years ago

AI will be a major productivity enhancer. The things that make programming interesting will be what programmers focus on in the future. There's so much potential software that could be written that I don't think that the number of programmers will decrease at all.

mbork_pl• 3 years ago

This may be true, and it worries me. I have a (completely unproven) conjecture that things that make programming boring (like writing boilerplate code) are more or less things that let us avoid burnout.

How many hours per day can you stay focused on difficult (or at least non-boring) problems? And for how many years?

Whenever someone says "AI = productivity enhancer", I hear "AI = burnout accelerator".

pydry2• 3 years ago

If programmer numbers decrease it won't be as a result of running out of things to code or AI, it'll be as a result of market consolidation within a few, extremely powerful firms who can squash most competitors like bugs.

Like the US auto market - a once thriving ecosystem of fiercely competing innovative small businesses consolidated into 3 vertically integrated firms who now think innovation is building a slightly bigger truck.

Of course, they'll probably blame it on AI coz a story about the inevitable march of technological progress invites less government oversight of the abuse of market power, and we will no doubt be plied with plausible sounding stories to that effect.

agentultra3• 3 years ago

AI will be a major productivity enhancer.

This worries me. The problem of programming isn't that we're too slow at writing code. It's that we're not thinking critically enough about the problems we're solving. And the way we approach programming is to not fully understand what we're doing and derive joy from fixing the result of our neglect later on.

TFA touches on this a little but doesn't dive in deep: how do you know that your software does what you think it does? In such a small example, a world renown researcher missed an error! Several people did. In their current state, LLM's will gleefully give you wrong answers as if they're 100% correct and it is now your job to verify they are correct. Most programmers do not know how to write a proof or other sort of formally verifiable specification. How will they know that the code they are about to commit is correct?

For now, LLM's aren't capable of mathematical reasoning. You can't prompt it for a piece of code and then ask it to demonstrate it's reasoning or prove to you why the change is correct with regards to your specifications.

And to make matters worse, humans are terrible at finding errors in code. Empirical studies on large scale informal code review suggest that humans can only reduce error rates when reviewing somewhere in the realm of 200LOC every couple of hours. We're not effective at finding errors to begin with and our abilities drop precipitously if we read too much code. Now we're going to have an assistant that generates possibly correct code and we're going to have to proof-read hundreds of lines?

This does not seem like a good use of time or enhancing my productivity.

I think we need to wait until we can get ML models that understand proof tactics and kernels to help automate proofs. I would be much more willing to accept code from a bot that can demonstrate its reasoning and isn't simply generating code based off of statistical models. Computers are nefariously discrete machines and "almost right" isn't good enough in many, many cases.

What will enhance productivity, in my humble opinion, are better languages that let us write more precise specifications and can derive the code for us.

anonzzzies• 3 years ago

See you in a few years when we can’t even keep up with the deluge of error prone programs.

I think the worst case of this situation will pop up before that; not writing code at all but just hook the AI up, ‘prompt engineered’ directly like so;

     <body>
     <script>doInsuranceForm();</script>
     </body>

Which will AI generate a html insurance form every time the page is requested and handle the user input etc. And it’ll be subtly wrong most of the time; it’s insurance, who knows if the current programmed forms are correct anyway (I know of at least one insurer that rounds off $0.01 wrong almost every policy because of using float for money on the web side).

Why have the AI make code we can read? Just let it do the entire thing /s

It’ll happen though and it will be used seriously by companies. The quality of the AI, legislation and public/consumer complaint fallout will determine if we get back to code that humans check. Being sceptical about humans wanting quality over paying as little as possible, I am going to say no.

foooobaba• 3 years ago

I think this idea of having formal methods integrated with AI code gen will be really powerful. If we can write verifiable specs, or even a way to automate approximate verification, then it can be used a s a scoring function to train an machine learning algorithm that performs codegen.

stevage• 3 years ago

These are really solid concerns. This shift from "creator" to "supervisor" is happening in lots of AI areas - look even at how self-driving cars are currently expecting their human drivers to act as supervisors, which they're bad at.

Also, on this point:

We can make proofreading easier with better tooling. Unit tests are a means of “proofreading”: we can catch AI errors automatically with tests.

The unit tests will definitely be written with AI as well. If that AI and the coding AI are wrong in the same way, we will have very subtle and hard to catch errors.

gqcwwjtg• 3 years ago

That problem sounds just like when I write tests for my own code. I don’t know if it’s totally avoidable.

NateEag• 3 years ago

It just now occurred to me that a natural solution to this problem could be to move past code review to "other teammate writes the tests for your PR."

You owe them an additional beer at the next happy hour for each test they write you wind up pushing a fix for.

Waterluvian• 3 years ago

Oh wow. That IS what I want. An AI that can offer a code review and suggest things.

thebigspacefuck• 3 years ago

To be fair, I’ve seen humans do worse.

amelius2• 3 years ago

Before we have AIs that can program, we will have AIs that can do computer administration tasks, like solving dependency problems with software installations etc. and compiling and installing stuff.

So before we have a tool that can do that, I'm not worried.

zshrdlu2• 3 years ago

It would be fun to see one attempt a Gentoo install.

jongjong• 3 years ago

Of course, AI models are often wrong. Even if they were right in the sense that they gave you exactly what you asked for, they may still be wrong in the sense that their interpretation of your requirements may not be what you intended; they are not great at filling the gaps if there is any context missing. The same amount of time it takes to provide an AI with all the necessary, nuanced context to produce an optimal solution could have been used to actually produce the desired optimal solution yourself. The hard part of software development is not coding, it's the amount of critical thinking and synthesizing required to produce optimal solutions.

I'm optimistic about AIs because they may finally prove once and for all that software engineers cannot be commodified as easily and thoroughly as most other white collar roles. It may be easier to commodify executives who make a small number of abstract high level decisions than it is to commodify developers who make many extremely nuanced, detailed decisions every day.

For sure, AI will replace certain kinds of developers, but not all kinds, at least not for some time. I think senior developers and architects will be among the last people in the firing line.

sramam• 3 years ago

I think senior developers and architects will be among the last people in the firing line.

And soon there will be no junior developers left to rise up to their ranks... So "last" in more ways than one!

m3kw92• 3 years ago

Would AIs that write unit test help test their own code, they will write code till it passes and link all the modules together

m3kw92• 3 years ago

They already knew this so I didn’t start the AI coding apocalypse

ccvannorman• 3 years ago

<< Laughs >>

Oh wait, you're serioous?!

<< Laughs harder >>

AI is great for making syntactically correct code; and so is copy-pasting the first result that works after a low quality search.

It still takes real minds to make code that is worth anything, and there will always be buffoons making code that doesn't work. Except the HN website, of course. ;-)

gmt2027• 3 years ago

The work of a master craftsman using the most basic hand tools is still leagues ahead of any amateur with advanced tools, automation, fancy CNC, and lasers. Mass production is where the tech would favour that amateur. We are surrounded by cheap mass-produced disposable products and are well aware of their low quality. No gourmet chef should be threatened by the efficiency of McDonald's. Fortunately, our software systems are custom artifacts with zero marginal cost of reproduction. AI will make competent programmers more productive and take expert programmers to new levels.

One transformation will come when a single developer can do the work of a team of ten today. Like solo content creators who no longer need large studios to finance and organise massive production teams and expensive equipment, we'll see more independent developers and small teams creating innovative software with some artistic character. Without the drag of management and upfront financing, the threshold for success will be much lower.

Another will be a market for lemons. Employers will have to find ways to establish competence at building high quality, understandable and maintainable systems. AI can already pass Leetcode-style tests so these are clearly not the right measures.

This is my best optimistic take on things. The rest of the time I'm worried the AI is coming for our jobs.

10g1k• 3 years ago

Prediction: An inflection/tipping point will occur when an AI has the ability to programme/create another AI with capabilities or coding superior to its own. That's the moment. It will snowball from there. A begets B which is better than A; B begets C which is better than B; and so on. After that point it will be limited only by processing capability and network connectivity.

turtleyacht2• 3 years ago

Insightful! Of all the creative prompt engineering I've read, I haven't found that yet: the equivalent of "fork a new process and while true."

Hey, ChatGPT. Ask this question at this URL when I talk about farms. But while you do that, ask it to also think on farms. Every few seconds, append its responses to every answer you give me.

Programming AIs worry me

Related Stories