Poor code quality due to AI assistants GitHub Copilot and ChatGPT

https://blobstreaming.org/poor-code-quality-due-to-ai-assistants-github-copilot-and-chatgpt/

by thunderbong • 2 years ago

21 10 2 years ago

captn3m0• 2 years ago

It’s unclear if this is based on public source code or if they had more than that?

rdegges• 2 years ago

Poor code quality is not a unique problem to AI coding assistants -- it's also a pervasive problem in the wild, regardless of who or what is writing code.

My thoughts around code quality are generally that if you're writing code for yourself (as a hobby, for open source, whatever) -- then write code however you want, as it's for fun. Obviously, others can contribute fixes and help make things better, but there's no professional "responsibility" in these cases.

On the other hand, if you're writing software meant to be used by others (including companies you work for), it's your responsibility as a developer to ensure the code you're pushing is:

- In compliance with style conventions

- Properly reviewed for security issues (tools like Snyk and other SAST/SCA tools work well for this)

- Makes architectural sense given the rest of your codebase

- Is documented so others can easily contribute/review without a lot of handholding

The rules above are true regardless of how the code is being generated.

genman2• 2 years ago

No, it is not, but this article is about trends - code getting worse instead of maintaining a steady poor quality.

ado__dev• 2 years ago

Fully agree here. The report also states that jr. devs are more likely to be using coding assistants, thus may not spot poor code generations and be much more likely to commit bad code.

We're still in the early days of AI and there are certain things that code assistants excel at today, and there are things they do very poorly at currently.

riedel• 2 years ago

genman2• 2 years ago

Interesting. I have not used AI generated code in professional settings (due to the institutional reasons) but I have thought about this and had postulated that there is high risk that the code will become more bloated and will lose architectural cohesion. It is interesting to have this backed by some data.

Now this is not necessarily a bad thing. I have seen a lot of code that is trying to follow the DRY principles and the result is horribly tangled mess where you have no idea what gets when executed by just looking at the code.

There is also hope that AI summarizing ability could help to gain a better systematic understanding of the code. Perhaps it's generalization capabilities could even help to refactor the code and build up a good test suit.

But this is to be seen to happen and better awareness is an advantage.

rickydroll• 2 years ago

I experienced the same thing when I first started using AI as a coding assistant. As my skill with the tool improved and I developed a better process for working with AI coding assistance, those problems disappeared.

I only knew to be aware of and correct these issues because I have decades of experience writing software. Modern business practices won't give junior developers the time to learn good practices and they won't get much mentoring because the senior people disappear after a couple of years.

thrill• 2 years ago

It's a poor carpenter who blames his tools.

ed• 2 years ago

In your analogy the carpenter isn’t complaining. It’s the person maintaining the carpenter’s work.

h335ian• 2 years ago

Having recently participated in a of review GitHub Copilot to determine if we might allow it at work, I can say it appears that most of the utility is in tab-completion of what you were roughly intending to express, in very short form: variable completion, expression completion and function/method/class/object/value signature completion.

I personally didn’t see suggestions that would have significant impact on code quality - simply helping one more quickly express initial concepts including all the language necessities. Intellisense++ish.

I have only seen it assist developers who were going down a bad path get there more quickly. I have not observed anything that would encourage/discourage any design pattern or developer choice.

My gut tells me that the fact that devs can more quickly deliver a “solution” (bad/good) means that they deliver their fat PR and … well … it’s fat - so it’s not scrutinized as well as it might have been had the dev taken some time and another set of eyes hit their branch before they got too far along. It’s often much harder to get the proper attention and review on a large-ish PR than it is to peek in along the way.

I personally couldn’t find anything that I feel would inherently lead to lower quality code. I do see how devs could more quickly deliver their low quality code and it can be a challenge to deny that code.

I’m left on the fence. I’ve found it valuable for autocompletion of what I’m about to express, but I also have no illusion that it’s going to tell me that what I’m about to express is stupid and/or of low quality.

Those things about quality are also quite language specific. Not so much about guessing what you might be about to type for tab-completion.

Also note - it’s about efficiency - as GitHub has stated. Duh. Notice copy/paste declined? Tab-completion. How many jr devs have you seen that won’t just write out a function declaration, they copy paste an entire function, gut it, and write their code. Odd to me, super slow and lame, but even I have seen the utility of tab-completing empty function blocks that are 90% of what I wanted.

Sooo. I’m inclined to say it’s interesting - but in no way at all diagnostic, deterministic or a very good study when it comes to writing conclusions. These are hyperbolic statements. Nearly link-bait.

Poor code quality due to AI assistants GitHub Copilot and ChatGPT

Related Stories