Ask HN: Does ChatGPT/AI enable anyone to code?

https://news.ycombinator.com/item?id=40925163

by FinnLobsien • 2 years ago

12 58 2 years ago

news.ycombinator.com

I'm non-technical and have never written code. There's a lot of talk on Twitter how AI means anyone can now easily code and build products and stuff.

Since a lot (most?) of HN is about technical stuff, I'd love to find out from y'all - how true is that?

Do you think that software engineering will go to a place where anyone can just say "help me build an app that does these three things" and I can just deploy that?

Or will a baseline of skill always be required?

borisk• 2 years ago

Give it a try :)

I think it's early days and whyle AI can write code to deliver features it's not perfect. So someone who knows how to write code, deploy software has a basic understanding of software architecture and cyber security will have a lot more luck than someone who has to run everything through the AI.

It's similar to how computers in chess worked. At one point (around the 90s and 00s) computers could play very good chess, but they had weak spots. So the best chess player in the world was a team of humans and computers. But eventually computers got better and no longer needed any human help. I think a similar scenario may play with software development, and we're still not at the place where an AI can everything without technical input from a human.

FinnLobsien5• 2 years ago

Yeah I've tried a few times, but never as an actual project - mostly just to see if I could make something that actually runs.

But when something didn't work I instantly ran into the issue of not knowing if I did something wrong or if the code was bad lol

solardev2• 2 years ago

FWIW, you can often feed the generated output back into the AI and ask it to troubleshoot, like "when I ran this code, I got ____ error" or "it's missing the ____, how do I fix that" and it will usually apologize and try again. It takes a few tries. But it usually does with humans too.

giuliomagnifico2• 2 years ago

Baseline skill will be always required, you can ask ChatGPT to improve or correct your code, but if you have no idea where to start you can’t rely 100% on ChatGPT.

Sometimes the hallucinations it has can be dangerous, you can’t copy-paste all, you have to give a look to the code to avoid some nonsense operations.

This can be dangerous also because if ChatGPT writes an entire app and the app has a bug in the code, ChatGPT will replicate the same bug on all the other apps that rely on that code. And if one guy found a vulnerability in ChatGPT coding operations, it can be exploited on every app created with ChatGPT.

However it’s a super useful tool, especially when you have to write ripetitive code or change something like a variable, you can ask ChatGPT to do these repetitive tasks and it’s good at it.

FinnLobsien5• 2 years ago

so sounds like your take is the "it doesn't replace the humans, but augments them" argument, right?

giuliomagnifico2• 2 years ago

Yea it’s like having a personal helper/apprentice, it’s very useful but you can’t alwa rely 100% on it.

oxfordmale2• 2 years ago

Have you ever done DIY? Things like replacing a tap could be done very quickly using YouTube. Why do people still call plumbers? Because reality is never as clean-cut as YouTube videos. Sometimes, you find the connection parts have completely corroded, and you can't undo them with your standard DIY tools; other times, you discover you have odd-sized connections (metrics versus non-metrics and vice versa). The real skill of a plumber is to be able to handle real-life problems.

It is the same for coding. Yes, you can ask it to generate basic code, and it will work. But my job as a software engineer isn't to write code. It is to deal with messy humans. Sure, ChatGPT can write an app that does three things. Does it work reliably on all different mobile phone devices? And do you only need three things, or do you want 16? Most clients know precisely what they want; they can't express it accurately in words in a single iteration. When you have delivered those three things, they will often ask for more. They will often ask for a hexagon-shaped room after you have built them a perfect circular-based one and then complain heavily if you quote it will take several weeks to refactor.

mnky9800n• 2 years ago

you also have no way of verifying that the youtube video is telling you to do the right thing. even if it works, it still could not be up to code so suddenly you have expensive changes when you have to sell your house.

same with chatgpt. when i ask it to do things i know a lot about, i am often time shocked how bad the code is that it generates and often does different stuff. makes me wonder about the code that i ask it to generate to do things i dont know about. haha.

FinnLobsien5• 2 years ago

typical Gell-Mann amnesia. I write a lot so I see this when I've used AI to generate writing.

If I ask it to generate something that I understand: "Wow this is the most surface level garbage anyone could create by reading one SEO-optimized article"

When it's about something I don't know anything about: "Wow it's impressive how far AI has come"

What I do find AI super useful for is finding stuff like metaphors.

"I want to find a metaphor between product-led vs. sales-led growth strategies for startups. Please give me a list of 10 i could use"

Those prompts are incredibly useful. I reckon there's some type of analogy with coding here.

atrettel• 2 years ago

I've done the same thing for the subject that I did my PhD in, and oh boy is the result weird. It is somewhat right at a high level but most of the low-level details are wrong at worst or confused at best. The explanation is incredibly shallow and misses so many important aspects that matter when implementing the idea in practice. It repeatedly refers to Mark Morkovin as "Nikolai Morkovin". And it even makes up a citation as a reference, though knowing this subject deeply I recognize that it seems to have combined two different references from 1962 and 1977 into a single nonexistent reference.

FinnLobsien5• 2 years ago

Interesting. It sounds like there are two parts to this:

a) Product sense becoming more important i.e. talking to users and figuring out what they want and how to build it rather than the pure ability of using code to make the computer do something

b) Having the machine but no idea how to repair/augment it only gets you so far. Like you might be able to run a lawn care business but if your lawnmower breaks and you can't repair it your business is dead.

Is that about right?

oxfordmale2• 2 years ago

Yes, that is about right.

It is also about knowing when to mow the lawn. Generally, you shouldn't mow the lawn when it is too wet, as it is too heavy on your mower. You also want to avoid mowing it when it has been too sunny, as the grass will turn brown. That requires experience beyond buying a mower.

solardev2• 2 years ago

(Not the original poster)

My experience with ChatGPT for coding has actually been the opposite: It was better at understanding and guessing underlying needs than the typical programmer I've worked with, who usually needed everything spelled out in technical bullet point requirements. Clients don't think like that (because they don't know how software works), but programmers need that (or need to come up with it on their own) to write the code. ChatGPT on the other hand seems pretty good at using its vast training data to guess at your intent and needs, as long as it's a relatively common use case.

I would say that's one of its great strengths (its ability to combine your prompt and its vast natural language data to give you an answer that effectively summarizes what your good options are). It's a better product owner than most programmers, in other words.

On the other hand, its coding ability isn't very impressive. It has the ability to generate code and it will run it and debug it if you ask it to, but it doesn't always do that by itself, and it's not great at doing "sanity tests" along the way to make sure each unit of code it's producing is actually doing what it should. For the most part it's doing the same sort of statistical modeling of language for coding as it does for natural languages, but code is a lot more precise and nuanced and can't just be a popularity contest of most statistical likely outcomes.

As a programmer, even a beginner one, you can usually spot these mistakes or at least ask it to explain or change some part of the generated code. However, for someone with zero coding experience, it might be harder to even spot these mistakes to begin with. You might be able to deploy the code but it might not actually do the thing you think it's doing, or maybe only do it under some circumstances but not others.

To be fair, though, the same things can happen when you work with human programmers. There's a lot of back and forth there too and trial and error. And many of us aren't great communicators, so ChatGPT has an advantage there. If you're patient with it, it might be a nice long term partner. But it's not quite at the "make me a _____ app and deploy it" one-prompt stage yet. Probably soon! I have no doubt one day soon it will replace developers like me, but for the time being, it works more effectively when paired with a human (and vice versa).

FloorEgg• 2 years ago

ChatGPT will allow you to start making progress 10x faster than before. It will help you be productive from the start without programming experience.

However as your project grows you will be forced to learn programming principles in order to progress. ChatGPT will help you learn these principles 10x faster.

My suggestion is don't think so binary about it. ChatGPT+GitHub copilot will make it way easier to code, and to learn how to code, and you will get stuck far less often (than you would have before LLMs) but you should still expect to have to learn a lot along the way.

LLMs are to programming what a power drill is to carpentry. Way easier to use and faster than a screw driver, but you still have to use it...

therobinhood• 2 years ago

Absolutely, AI makes coding accessible to a broader audience. It allows anyone to start coding, but it's like driving a car without fully understanding how the engine works. You might find yourself constantly troubleshooting and revising your code. For example, building a simple website is quite straightforward with AI assistance, but when you dive into more complex, low-level programming, a deeper understanding becomes crucial to avoid major pitfalls.

FinnLobsien5• 2 years ago

I think the car analogy is interesting because I'd say 90+% of drivers don't really know how the engine works besides gasoline being burned to make the car move lol.

hpen14• 2 years ago

A better analogy: "it's like (engineering) a car without fully understanding how the engine works."

reyml• 2 years ago

definitely, I did not know how to code, but thanks to ChatGPT/Claude i have been able to create several websites with a lot of backend and API manipulation.

i still don't know how many things work, like some functions, and I struggle a lot when the code gets complicated - it starts to lose the overall picture. To address this, i have to start reading the code line by line to understand what's happening.

So far, I have used Node.js with Express in the backend mostly. My advice would be to go ahead and try it, don't be afraid, as it helps a lot. it makes you feel like you can build a lot of complicated stuff, though, I'm having a hard time finding ideas lately

muzani• 2 years ago

It's not that easy. I mentor at hackathons. Early on, even with GPT-4, lots of people thought the key was thinking up good ideas and getting LLMs to build them. Most still didn't get their apps up by day 2. There's definitely a baseline of skill.

It certainly helps with deployment though, especially things like why import vs require, where to find the API keys, which class to look inside, and so on. I would say we're twice as productive now. If it took you 3 days to do hello world, it would take maybe 1 now.

You definitely still need to read the docs. AI is just great at telling you which docs to read. But it may lead you astray if you choose not to.

Another gotcha is that LLM is the average of the above average skill. It sounds funky but basically GPT still recommends a 2018 stack for Android, which is far outdated, even though it knows the 2024 stuff. We see people trying it in interviews, and many end up giving feedback that they wish they didn't use it.

ogou• 2 years ago

"easily code and build products and stuff"

Those are three different things. Most of these AI agents can produce working code in different languages. Whether that matches tightly defined business needs is a different matter. Writing code that makes a P5.js sketch that animates a cat is a neat trick, but not something you can build a product around. Building products is a broad enterprise and can be very difficult, even with a full team of highly skilled engineers. Google and Facebook have access to the best engineers on the market and even they make products that bomb or go way over budget and deadline. The "and stuff" part is usually the part that makes money. Logistics, financial infrastructure, marketing, marketplace differentiation, and much more go into successful business built on software. An AI agent can describe those things in plausible terms, but can't manifest any of it.

What these platforms CAN do is help prototype experiments quickly. Then you could take it to a Dev team and have it built properly. I think that is an interesting future and will allow more people to participate in the early stages of some projects.

krapp• 2 years ago

The thing is, you have to know how to code well enough to know how useful the "code" ChatGPT generates is. And I put code in quotes because it's important to realize that AI doesn't know what "code" is, it just knows what syntactically likely strings are. So it might make up libraries that don't exist or write functions that don't work or don't even follow the semantics of the language, it's just spitting out something that "looks like code."

Much of the time, and for simple enough tasks for which there exists enough examples in the training data, what it produces might be useful. But it can also be wrong in subtle, non-obvious ways. But if you don't know anything about coding and just don't want to pay people for their time and skill, nothing is obvious.

enceladus06• 2 years ago

What LLMs help with the most is getting the first half of the project or script outlined. Then you go in and read it, make sure ChatGPT didn't spit out something crazy. But it is super helpfull for writing short scripts and starting projects. But it still does halucinate.

rsynnott• 2 years ago

There's a lot of talk on Twitter how AI means anyone can now easily code and build products and stuff.

There's a lot of talk about all sorts of nonsense on Twitter. No, this isn't true.

nubinetwork• 2 years ago

I was able to get an LLM chatbot to write another chatbot, with the idea of giving it filesystem access and the ability to run git. I got half of it working, but I got stumped around function call LLM models, so I kindof gave up.

But a better answer would be, given a large enough context size (32k or higher), it should be reasonable to write code as long as it doesn't try to do something stupid like using the wrong json parser format for a function that doesn't even exist in that library.

hpen14• 2 years ago

I think ChatGPT is just the next level search engine for coding. It doesn’t fundamentally change anything about building complex systems, just another tool.

I would never ship a production app built with ChatGPT. I need to know the intricacies of my code base. I wanna fix bugs in my head based on the architecture and specific implementation details. I need to know how fragile or not my system is and its weaknesses

hpen14• 2 years ago

But honestly I hope everyone keeps pushing this “AI is gonna replace programmers” stuff. Will keep people out of the field and my technical chops will have higher market value.

Spooky23• 2 years ago

I agree.

I’m a management dope now and lost most of my coding mojo, but I can much more quickly conjure up small scripts or programs to do small tasks or demonstrate a point.

When I started my career, I was the “kid” who introduced Google to the office! ChatGPT feels like that!

qarl11• 2 years ago

Yes.

Last week I wanted to interface with a bluetooth device in Python on a Mac.

I asked ChatGPT for a solution - it told me about a Python-ObjectiveC bridge to the MacOS Bluetooth API. It coded a stub to search for the device, connect to it, query for services, and write data to a specific service.

I've never used bluetooth before. I have minimal experience with Python. With ChatGPT this entire prototype took about 10 minutes to implement.

hpen14• 2 years ago

Ok now ask ChatGPT to produce a git patch file for your production repo and see how far that gets you. Not far.

qarl11• 2 years ago

HEH. Your point is that ChatGPT is useless because it doesn't (yet) do everything?

FYI - it can't turn lead to gold either.

JSDevOps2• 2 years ago

That’s one example. So the answer isn’t yes. Come back when it’s stated literally making up names of functions and see how frustrated it gets you. THEN and only then can you be classed as a software engineer.

naffenuf• 2 years ago

Getting software built with ChatGPT depends quite a bit on your level of curiosity and ability to formulate your requests clearly. Tell it what you're trying to do as clearly as possible, with images if you have them, and start asking questions: "I'm not a programmer, but can we work together to implement this? Where would we start?" Ask it to write code in english (pseudocode) so you can read it. Break your application into the smallest pieces and build them one at a time so they work together. Once you make sure it's going to do what you want, ask what language to use and what tools to install. If it says you need a python script, ask it to write it. If you don't know how to run that on your computer, ask. If the interface isn't what you intended, show and tell it how to fix it. Get it? This process is slower than if you knew how to code yourself, but it's also a hell of a lot of fun and the LLM does a lot of the tedious stuff.

JSDevOps2• 2 years ago

Give it ago and let us know how you get on ;-) … at this point I’d be more worried about if your a project manager and your sole job is shuffling tickets around a Jira board or using a spreadsheet.

Ask HN: Does ChatGPT/AI enable anyone to code?

Related Stories