'Students who use AI as a crutch don't learn anything'

https://english.elpais.com/technology/2024-10-03/ethan-mollick-analyst-students-who-use-ai-as-a-crutch-dont-learn-anything.html

by belter • 2 years ago

111 150 2 years ago

english.elpais.com

_tk_• 2 years ago

Very misleading title. From the article:

„The crutch is a dangerous approach because if we use a crutch, we stop thinking. Students who use AI as a crutch don’t learn anything. It prevents them from thinking. Instead, using AI as co-intelligence is important because it increases your capabilities and also keeps you in the loop.“

fgbnfghf• 2 years ago

I use AI to ask questions when I am not totally sure what the question is, and it is very helpful for narrowing that down. It can be powerful as a tool to help get your foot in the door on new knowledge. Just like google search there is a correct way to use it and an incorrect way.

Another thing to consider is the motivation of companies like OpenAI. Their products are designed to be used as a crutch. Their money is in total reliance on the product.

ksd482• 2 years ago

I feel like this is exactly what the title is conveying. What’s misleading about it?

iamleppert• 2 years ago

That’s what they said when the calculator was invented. Out with the old, in with the new! Sorry but not sorry life was so hard before but we got AI to do the work for us now.

giantg2• 2 years ago

Not the same at all. The results of calculators are verifiable. The results of AI can be dangerously wrong without easy validation. Calculators don't eliminate the application of concepts from learning, but AI does.

parpfish• 2 years ago

Verifiability is part of it, but the other part is that calculators don’t provide a full end-to-end solution (unless you’re doing worksheets for homework).

Each calculation is just one step, it’s up to the user to figure out which steps to take and how to chain them together. they might even learn that the whole thing would be faster if they could do some of those calculations their head.

like if you’re trying to figure out how much wood to buy for a deck, you’d still need to break the big problem down into those individual computations to do in the calculator. Unlike an llm where you could just ask it and it’d jump straight to a final answer

rthrth45y• 2 years ago

Good point but also not a new problem. Humans use the same mechanics to assign variable weights and biases in order to validate information. If you look at a banana, you can determine what it is based on the knowledge you contain, and that process triggers a similar cascade of weights. the difference is brains are much more capable of this, but we still misremember things and recite wrong information. The game of telephone is a great example.

I don't think AI eliminated the application of concepts from learning. I think that has been eliminated enough due to the erosion of our public education systems. If we are not capable of critical thought with the information our own peers present us, why would it be any different when we seek it from AI?

ben_w8• 2 years ago

The results of a calculator are only easy to verify with either someone who can do the same arithmetic or who has another calculator.

I had this happen to me once while shopping, where I could immediately tell that three items costing less than £1 each should not come to a total of more than £3, but the cashier needed that explained to them.

(And that's aside from anything about asking LLMs to output in a format suitable for mechanical validation, which they can generally do).

Izkata3• 2 years ago

Calculators don't eliminate the application of concepts from learning

On their own they would. We just don't see it very much because of external forces managing it - like teachers not allowing students to use them early in school.

StefanBatory• 2 years ago

No, I can't agree with you here.

I'm software engineering student. I had a phase year ago where I was using ChatGPT a lot, a lot more than I ever should have.

And it messed up with my brain a lot. I felt I became utterly lazy; to the point where quick fixes that should have taken me like, 10-15 seconds (?) I had to do with AI, which often took a very long time.

And the point of studying is to learn. You won't learn anything if you have someone else write your software for you.

givemeethekeys• 2 years ago

My friends in a top tier high school had no intuition about numbers because they used calculators for everything, including basic addition and subtraction.

Some of them had customer service jobs where they became utterly confused if the change was 99 cents and the customer gave them an additional penny.

mistrial9• 2 years ago

except that the typical case is ... the charge is $1.01, I give you a $5 and a penny. A penny from the client to the house on top of a charge of $0.99 by the house, does xxxxxxxxx ... (edit) as pointed out below, a penny plus a $0.99 charge means that the cashier can return a whole number of bills, avoiding any coins..

pclmulqdq5• 2 years ago

Everyone crying "calculator" about ChatGPT has me convinced that ChatGPT is bad for your education. Learning to do mental math as a child sucked, but now that I can do it, my brain is so much more free to think about stuff that matters. There's no intervening step of "let me pull out a calculator to see what that is," I just know the answer. The thoughts can just flow freely.

blibble• 2 years ago

the difference is I still understand everything the calculator can do, and can do it by hand on paper

the AI generation is not going to know how to do anything other than type into chatgpt

at which point human progress ends and we start going backwards

SoftTalker2• 2 years ago

And worse, the people who control the AIs now have ultimate power to rewrite history and mold opinion. If they want everyone to think the earth is flat they can do it.

nonrandomstring5• 2 years ago

<passive aggressive> life was so hard before but we got AI to do the work for us now.

Aggression against what? Yourself?

I think you show a tragic misunderstanding of technology and what it is doing in the world. It's not the work that it's doing for you. It's the living. Is it really your life you want a machine to take?

Nobody wants to "work". Henry David Thoreau said ,"There is no more fatal blunderer than he who consumes the greater part of his life getting his living." All good, no? But that's not what "AI", in the hands of exploiters (or even yourself, as a self-exploiter) is going to do to you. Technology is more "productive" but creates more, not less labour.

Better to heed Max Frisch who said, "Technology is the knack of so arranging the world that we don't have to experience it." Would you employ a machine to enjoy a music concert for you? To have sex for you or play games for you so you're not troubled by the effort?

Dalewyn3• 2 years ago

To have sex for you or play games for you so you're not troubled by the effort?

Two of the three games I play on a daily basis largely play themselves, so... yes, actually. I still have plenty of fun watching them.

s0ss• 2 years ago

Lots of nuance that you’re not addressing, IMO. Here’s some more nuance:

Steroids. It’s not a perfect metaphor, But I think it’s useful. Two people are trying to gain muscle mass. They both have an ideal starting point. First person has a healthy diet, lots of exercise, and sleep. The second person has all of the same things the first person however they also taking growth hormones.

Lots of folks look at the two results and will see lots of different things. Beauty is in the eye of the beholder I suppose. If you think the end results of the work should yield sculpted bodies with larger than normal muscles… you might opt to use hormones. However, if you think sculpted bodies with larger than normal muscles looks unrealistic or just not your style/goal… you would probably opt for a more natural approach.

Both have their merits and could be described as “fit” despite their differences. folks may value one over the other. people might fantasize about looking like thor, but if everyone actually looked like thor, things would be weird. My two cents: Thor is fiction, and while we need fiction. Im not going to pretend that anyone should look like thor in order to be in shape or to be described as fit. If we allow ourselves to be fooled into thinking that it can be normal to look like thor, then we are doing something wrong. Fiction should not become reality.

wredue3• 2 years ago

A new study also shows people using AI produce code with 41% more bugs. And that’s just what the users missed!

Calculators arent giving you “kind of correct” answers.

aspenmayer3• 2 years ago

A new study also shows people using AI produce code with 41% more bugs. And that’s just what the users missed!

Do you have a link or more info? Without further context, the 41% doesn't tell us the whole story; all we have is a numerator lacking a denominator. Did bugs per line of code go up, or down? Did # LOC produced after using AI go up/down? For all we know, the increase in productivity caused average bugs per line to go down, rather than up, which is contrary to the argument you're making.

lawn• 2 years ago

That’s what they said when the calculator was invented.

And it's beneficial to ban calculators for learning, which is the point of the article?

TrackerFF• 2 years ago

I'm guilty here. For years I've been meaning to learn modern webdev, but every time I've sat down to read the docs, tutorials, books, and what have you - I just give up after a couple of hours. Getting seemingly easy stuff done is just a drag.

The other day I decide to try ChatGPT 4o with canvas. For a solid year, I've planned to create some easy membership registration and booking system for this small club I'm part of - just simple stuff to book rooms in a building.

Well, to my absolute amazement - I had a working product up and running after 4 hours of working with ChatGPT. One block at a time, one function at a time. After a day I had built on a bunch of functionality.

So while I'm not completely clueless on back-end programming, my front-end skills are solidly beginner. But it felt like a breeze working with ChatGPT. I think I manually modified at tops 10 lines during all this, everything else was just copy/paste and upload source files to ChatGPT.

Any errors I'd get, I'd either copy/paste, or provide a screenshot.

I actually tried doing something similar when GPT3.5 came out almost two years ago, but it was just too cumbersome then. What I experienced the other day felt lightyears beyond that.

So, did I learn anything ? No - not really. But did it solve a problem for me? yes.

EDIT: But I will add, it did provide solid explanations to any questions I had. Dunno how well it would have worked if my 70 year old mom had tried the same thing, but a gamechanger for people like me.

aimazon3• 2 years ago

The point missed here is that you didn’t need to write any code at all, with or without ChatGPT. ChatGPT helped you with busy work: you reinvented something that already exists, instead of using a mature and established membership platform, you built your own. The reason this has parallels to education is because that’s what education traditionally is: busy work.

You did learn something, by the way: you learned how to use modern tools. You didn’t do things most efficiently but it was more efficient than writing code without the help of ChatGPT.

pclmulqdq5• 2 years ago

When you are learning something, that busy work helps. What you think of as busy work when you are a professional is actually often sort of novel to learners and is a simple example of how to do stuff.

stackghost2• 2 years ago

The reason this has parallels to education is because that’s what education traditionally is: busy work.

Busy work is work that is assigned merely for the purpose of occupying one's time.

That's not the same thing as practice. We drill children in arithmetic not to keep them busy but because it turns out repeatedly solving multiplication problems is an effective way to teach children their times tables.

whimsicalism2• 2 years ago

I love that AI negativity on HN is so strong that we reclassify whatever work AI can do into “busy work” as soon as it is possible.

JTyQZSnP3cQGa8B• 2 years ago

you reinvented something that already exists

Since AI can’t invent new stuff, who will do that? Juniors who haven’t learned anything because of those tools? Or seniors who will disappear one day because they are retiring or are being replaced by AIs?

I already work with juniors who use ChatGPT and cannot explain what they wrote. They have a fucking engineers degree and don’t know anything. It’s catastrophic and may increase in the future. What will happen if it continues like this?

tempodox• 2 years ago

You happened to use an LLM for something that is most prominent in its training data. Do something off its beaten path and correcting all the hallucinations will be more work than just plain old learning it.

banku_brougham• 2 years ago

yep, this is what have found so far.

whimsicalism2• 2 years ago

not true in my experience - of course you have to work within its capabilities but i find it to be a capable partner across large segments of tasks

ants_everywhere• 2 years ago

This is my experience too. Even with pretty common problems in popular languages like Python. The code generated by ChatGPT 4o is full of bugs. If you give it feedback it just thrashes instead of trying to locate and correct the underlying problems.

Even if you ask it to think about and correct the underlying problems, it still generates buggy code, often with the same problems it was pretty decent at reasoning about.

furyofantares• 2 years ago

You also didn't learn modern webdev for the years you'd been meaning to.

You're actually better poised to learn it now if you care to, now that you have a component you care about that already works that you can work from. Of course maybe you won't, maybe having GPT there will indeed prevent you from ever learning it, I don't know.

swatcoder3• 2 years ago

So, did I learn anything ? No - not really. But did it solve a problem for me? yes.

And this is exactly the concern.

The tools are genuinely useful for some tasks. But unlike club organizers getting to DIY some hobby project for their club, students aren't yet being tasked to produce useful things in the best way possible. They're being tasked to do fairly rudimentary things so that they can learn some fundamentals by way of practice.

And likewise, in trades like ours, juniors are tasked to do useful things, but they're given affordance to deliver those things in ways that help them learn some fundamentals by way of practice.

Students and juniors who skip the practice are basically just trading their future expertise and readiness to accomplish trivial things that either don't or barely matter. Some of them may become the first generation of expert prompt engineers, accomplishing things in totally new ways in what amounts to a novel trade, but many of them are just going to be shooting themselves in the foot.

echelon2• 2 years ago

This is experience though, and people do learn this way.

This is the exact same exercise as my first time slapping Dynamic Drive scripts together to customize EzBoard back in the 2000s. I didn't understand any of it at the time.

This style of learning is hands on. You learn a little bit about the shape of the problem before you sit down and learn the theory.

Not everyone learns by opening the book first. Some people like to get their hands wet. Introduction through practical osmosis can lead to a fertile appreciation for the theory.

valval4• 2 years ago

I’m at least twice the programmer I was before LLMs, and I spend maybe a tenth of the time reading docs I then did.

treflop• 2 years ago

If it’s not your daily job, I don’t see the harm. Sometimes I use ChatGPT for something I absolutely don’t care about.

But if it is, then I think you’re trading it for a career of trial and error.

Regularly I watch people at work spend a week trying to solve a problem, but because I learned the fundamentals at some point in my past life, I am able to break down the problem, identify the root cause, and solve it quickly.

sibeliuss• 2 years ago

As someone who already has skills in backend / frontend, AI tooling has made me fearless in terms of new material. I couldn't type it out by hand, but by getting something working through a (much faster) trial and error process, I'm learning so much! I suspect this is your case as well. There's a lot of learning going on underneath, which will only improve your abilities in ways that will come back as astonishingly beneficial if you keep working on your project.

bdlowery• 2 years ago

You skipped all the hard parts, all the struggling, and now you have a working product without a mental model and can't level up to doing harder things on your own. Struggling IS learning. You didn't try different paths, piece different info together, and then eventually create a mental model. You just used ChatGPT to skip to the end result.

It's like enrolling for a Calc 2, cheating on all the homework to get an A, and saying "did i learn anything? No, but it solved all of these annoying homework problems for me!" Now when you have to take the 1st exam you're screwed because you didn't learn anything.

jessekv• 2 years ago

To dig into your metaphor, what would you say is the equivalent of the first exam in a programming context?

simonw4• 2 years ago

I’ve long believed that the best way to learn anything in tech is to attempt to try to build something with it.

My hunch is that people who use the process you are describing will still get a massive leg-up in learning skills like web development.

Often it isn’t a choice between using AI-assistance to get some working vs spending 20 hours figuring it out from scratch: it’s a choice between getting somewhere with AI or not doing the project at all, because life is full of things to do that are more rewarding than those 20 hours of frustration.

Anecdotally, I’ve heard from a bunch of people who always wanted to learn software development skills but were put off by the steep initial learning curve before you see any concrete progress… and who are now building useful things and getting curious about learning more.

lovethevoid• 2 years ago

I agree on building to learn, but disagree on the massive leg-up. It's like believing that using a template off GitHub is going to teach you much (outside of specific use cases). This is also why a lot of GitHub issues now submitted by users don't even follow very basic processes you may have outlined on narrowing down problems. Here's this long copied code from terminal, fix it.

What makes the most difference in building to learn is the tiny steps you take to build. Printing hello world for the first time, changing it and seeing something else, using inputs for the first time to print hello, [variable], getting that image to animate across the screen. Each step becoming a great foundation for further curiosity, rather than turning your project into a black box.

In contrast, I've heard from a bunch of people who wanted to learn software development, but now don't see a point since AI can do it. Same with drawing. There's a large growing apathy towards learning skills I've noticed.

This is why most advocates for it don't do it from the perspective of learning. They do it from the perspective of building fast in the hands of those who already grasp the foundations.

hammock• 2 years ago

The whole thing reminds me of how we used to check out BASIC books from the library and manually type in complete programs, games etc from the book and run them. I wouldn’t say I learned NOTHING, far from it, but it definitely wasn’t a path to become fluent in BASIC

drdeca2• 2 years ago

“In the Phaedrus, writing is the pharmakon that the trickster god Theuth offers, the toxin and remedy in one. With writing, man will no longer forget; but he will also no longer think.”

When I have to navigate to somewhere I haven’t been before, I generally do not read a map, but follow instructions from some navigation software. As a consequence, I often don’t really know where places are, just the route I take to get to a destination. With GPS navigation, I do not get lost, but neither do I have much awareness of how locations are spatially arranged.

Such technologies seem to always be like this.

A potion which removes a difficult task, but also dulls the ability to do such tasks oneself.

It is like that one SMBC comic https://www.smbc-comics.com/comic/identity “ Humans offloaded memory to books, then thought to computers. Now, we're offloading our desires to the network. All that remains are basic bodily functions, which well offload in another generation or two. At that point, well just merge into one united entity so, it all works out.”

illwrks• 2 years ago

Not to put a downer but have you also perhaps created a liability for your club? If you don’t fully grasp what is going on with the code have you potentially left the door open to exploits?

dmurray• 2 years ago

A reasonable thing to be concerned about, but he described having a solid understanding of the backend, which is the more likely place to introduce a security hole.

analog31• 2 years ago

>> So, did I learn anything ? No - not really. But did it solve a problem for me? yes.

In my old age (60), I've gotten a little bit philosophical about this issue. I'm old enough to have pored through entire textbooks and manuals, e.g., BASIC, HyperCard, Turbo Pascal, MS-DOS (to name a few). But I can still ask myself at the end of the day:

So, did I learn anything?

Those things are all flawed, temporary creations of some individual, and are no longer useful. On the other hand, there are certain things that I've learned, and consider to be "fundamental," such as math, physics, and admittedly, music. Now a philosopher might correct me and point out that my choice of "fundamental" is arbitrary, but if nothing else, those things are long-lasting. The laws of physics that I'm capable of grasping haven't changed in my lifetime, nor has the technique of playing the double bass without injury.

Perhaps a thing you could do is sit down and decide what things you consider to be fundamental enough (relative to your interests) to learn on a deep level, and what things you can interact with on a superficial basis by letting AI take care of them for you.

iknownthing• 2 years ago

I did something very similar to this, except I would say I did learn something. It felt like studying by looking up the answers - not the best way to study but if you look up the answers to enough questions you are bound to learn something by osmosis.

QuantumGood• 2 years ago

Many years ago I skim-studied a couple webdev books after trying to choose useful ones. The deeper I looked, the nuttier they seemed, which later I realized was due to bandwagon bias. Bias of any kind is of course hard to subtract, but seems a particularly strong problem for new coders looking for learning resources.

xigoi2• 2 years ago

May I hazard a guess that the website you created is at teast 10× slower than it needs to be?

pluc• 2 years ago

Technology that renders effort and research pointless makes people lazy and stupid, story at 11

gomerspiles• 2 years ago

I suppose a headline that doesn't have much to do with the content is also such a technology?

throwaway918299• 2 years ago

I’m sure there were people that said the same thing about Google. I’m pretty sure they even said the same thing about the written word.

yoyoyo1122• 2 years ago

It's such a "Back in my day..." mentality.

"Farmers today are much less skilled and knowledge than farmers 50 years ago!"

fnordpiglet• 2 years ago

If you use a crutch you won’t learn anything is an age old truism. I don’t know why we would think a new tool somehow changes that dynamic.

My daughter is 10 and she is learning factoring, long division, and other things that a calculator does very well with. But she’s not allowed to use it at this stage because she can’t learn while using a crutch.

She’s also learning to write essays. She writes her essays then puts them into ChatGPT and asks for analysis, feedback, explanatory revisions. Then she revises the essay on her own without being able to refer back to the advice. This is using AI as a complement to learning and it’s been remarkably powerful. She can get feedback immediately, it’s high quality and impartial, and she can do it as many time as she finds useful. So, the fundamentals of learning don’t change no matter how powerful or different the tools become. But ignoring the tools because they can be used in place of learning if used in place of learning is dumb.

ryandrake• 2 years ago

My daughter is 10 and she is learning factoring, long division, and other things that a calculator does very well with. But she’s not allowed to use it at this stage because she can’t learn while using a crutch.

Which seems silly to me, but what do I know, I'm not a teacher. Nobody does long division in real life after K-12 school. It is not a useful skill to have, and it is not a useful concept to know. If I have to divide two numbers I just use a calculator like 99% of the humans on the planet.

Knowing what division is, and what it means to divide one number by another is valuable, but can you just teach that without teaching the mechanics of "divide the partial dividend by the divisor, then multiply the partial quotient by the divisor, and subtract from the partial dividend, extending to the next blah blah blah blah"? Are we really training the next generation for a world without electricity?

ben_w8• 2 years ago

Nobody does long division in real life after K-12 school. It is not a useful skill to have, and it is not a useful concept to know.

To my surprise, when I did pure maths at A level I found the same ideas applied to dividing one polynomial by another.

Of course as a mere memorisable algorithm a computer can also do this, so I'm not sure how useful it is even to pure mathematics, but there is (or was, 20 years ago) some use to the idea.

nasmorn• 2 years ago

Maybe learning to follow a simple algorithm helps some children to structure their thinking. Divisions are not hard to do. Doing a lot of them has little benefit though.

Izkata3• 2 years ago

Intuitively knowing how it works means being able to immediately catch 110 is wrong when you tried to enter 55 / 2 (and mis-typed / as *), instead of blindly trusting it.

This was a problem even 20 years ago. I remember one instance from my physics class in highschool, one student's result had gravity going in the wrong direction. The teacher used it as an example for why should understand what we're doing and not to blindly trust our calculators.

hyperG• 2 years ago

It wouldn't be shocking to me that in 30 years, your daughter would be tasked with writing a whole book and not just an essay. Part of the learning process is learning to leverage technology. Something I think we do a really bad job at teaching kids.

Of course, leveraging technology to do the exact same thing you would do without the technology is a terrible lesson on many levels.

nonrandomstring5• 2 years ago

don't learn anything

I don't think this is true. We learn a lot: Deference. Dependency. Entitlement. Impatience. Conformity. Distraction. Overconfidence. Intemperance...

If something "does the thinking for you" it has a much deeper effect than simply being a "crutch for the mind". It changes our relation to the world, to knowledge, motives, ambition, self-control...

"AI" is going to change our minds, but from what I've seen so far the outcome is a really quite awful kind of person, a net burden to society rather than a creative and productive asset.

visarga• 2 years ago

Maybe we need AIs that take care of our growth as well. Let's say a school or company mandates one hour chatting with an AI that will ask you questions and probe your knowledge, and give in explanations as well for things you don't know. The more time you spend with it, the better, it acts like a tutor not like a subordinate that hides away complexity from you.

There is no reason AI can't work like a tutor, the current crop is just a first take on the human-AI interaction problem. The motivation part can be solved by gamification and constraints - you need to earn a number of points by chatting the AI, and those points are reported to your teacher/manager. So a triad of student+AI+human coach would solve the motivation part.

nonrandomstring5• 2 years ago

Those are called parents.

pier253• 2 years ago

The less we learn the more stupid we'll be. The brain, like any muscle, atrophies if you don't use it.

This is already happening with genz in college.

https://www.theatlantic.com/magazine/archive/2024/11/the-eli...

pavel_lishin• 2 years ago

I'd wager you can find a copy of an article with that premise about every generation going back to a generation after the invention of writing.

pier253• 2 years ago

What about IQ declining after many generations of growth?

Or what about the attention span crisis?

Or the lack of technical skills in genz?

I seriously doubt this is a generational thing as you seem to be arguing.

Smartphones and social media didn't exist until 15-20 years ago and we're now seeing the consequences.

Jensson3• 2 years ago

People are much weaker and fatter today than 50 years ago, sometimes the next generation actually is way worse than the previous.

vunderba2• 2 years ago

Whether or not there is any truth to pithy sounding sayings (that have been parroted since the dawn of time aside), the brain is NOT a muscle.

https://www.hopkinsmedicine.org/health/conditions-and-diseas...

pier253• 2 years ago

It's obviously not a muscle. Wasn't the metaphor obvious?

aithrowawaycomm2• 2 years ago

This condescension is very common and very irritating:

Q. You say that the best experts of the future will be those who make the most use of AI. Are people who are waiting to use AI making a mistake?
A. I get it, it’s an unnerving technology. People are freaking out. They’re getting a sense of three sleepless nights and running away screaming. It feels like an essential threat to a lot of careers. I think if you’re a good journalist, the first time you think, “oh no.” But then you start to see how this could help you do things better than before.

There are a lot of white-collar jobs where LLMs do more harm than good because a 1/4 hallucination rate means you waste too much time on wild goose chases. I briefly thought GPT-4 was useful for finding papers given a description of the results - I “kicked the tires” with some AI research and was very impressed. But when I tried to find papers on animal cognition, about 75% of the results were fictional, though supposedly authored by real animal cognition experts. And GPT-4o is even worse! The tools are just not good enough for my use case; Google Scholar is far more reliable.

I just don’t understand the childish motivated reasoning behind assuming the skeptics are scared. Maybe if I spent “three sleepless nights” talking to ChatGPT I would be more enlightened.

simonw4• 2 years ago

I briefly thought GPT-4 was useful for finding papers given a description of the results.

That’s one of the many poorly documented traps of LLMs: trying to use them to find papers like that is a fast-track to worthless hallucinations. If that was one of your first experiments I can’t blame you for thinking this tech is “more harm than good”.

LLMs are terrible search engines… except for the times when they are great search engines!

Learning when and what to use them for continues to be a significantly under-appreciated challenge.

aithrowawaycomm2• 2 years ago

No, it was not my first experiment - again with this unbelievable condescension! I have been playing around with this stuff since GPT-3. That was my first practical use case where GPT wasn't a totally useless waste of money. It works very well with AI-related papers, maybe a 1% hallucination rate. I got the idea from an AI researcher and I was intrigued that ChatGPT might actually be useful for me. But it was not. The problem, as always with ANNs, is that I went slightly off the happy path. Even a skeptic like me assumed the AI papers successes was evidence that GPT-4 was better at remembering its pretraining data; instead I think it's evidence that a data contractor RLHFed the answers and GPT had it memorized.

LLMs are terrible search engines… except for the times when they are great search engines!

But note that what you said about finding papers was wrong, it works extremely well for AI research. The reason LLMs are useless to me across the board is that these unpredictable and arbitrary limitations apply to everything, not just search. "Learning when and what to use them for" is pure trial-and-error because it seems to amount to guessing what tasks the 3rd-party data contractors trained the LLM to solve.

I am not a Python or JavaScript developer, nor do I write code for extremely well-known libraries. I use F# for oddball projects (often analytics), and GPT-4 was utterly useless for F# codegen. My first experiments with GPT-3.5 showed that it would plagiarize hundreds of lines of public F# projects, including from my own GitHub, without any prompt engineering or trial-and-error - it was just blind plagiarism. GPT-4 isn't quite that bad, but it's still not even close to being good enough to help me - in particular it has no understanding of high-performance F#. I would be spending far more time auditing and optimizing its crappy code. And time spent writing code has never been the limiting factor in my F# development.

I also do some recreational mathematics on finite geometry and combinatorial group theory; GPT-4 was utterly useless here, even with CoT prompting, and even though it solved more complex graduate-level algebra problems without any difficulty. Of course, those problems were repeated and solved in dozens of graduate textbooks. My cute little groups, not so much. I believe CoT prompting is theoretically incapable of helping GPT here since the computational complexity is too high. What CoT prompting gives you is a bunch of insidious errors that take time and effort to unravel.

Otherwise there's nothing I do that would even conceivably benefit from an LLM: it can't play guitar, it can't play with my cats, and I would never use it to communicate with friends or family. I guess I could fill my brain with shallow subject knowledge about something, a few choice sentences. But I'd much rather understand something in depth by reading a book. I'm not too busy to read a book. Otherwise... maybe I could use LLMs to write polite no-thank-yous to unsolicited recruiters.

This tech truly has nothing to offer me. I think you are failing to understand that, as a Python developer who maintains one of the biggest Python web frameworks and writes a popular blog for general tech audiences, LLMs are unusually well-suited towards your use cases, due to reasons that will not extend to people working in more isolated corners of the world.

lolinder2• 2 years ago

There are a lot of white-collar jobs where LLMs do more harm than good because a 1/4 hallucination rate means you waste too much time on wild goose chases. I briefly thought GPT-4 was useful for finding papers given a description of the results - I “kicked the tires” with some AI research and was very impressed. But when I tried to find papers on animal cognition...

This is less a question of which jobs benefit from AI in general and which don't than it is a question of tasks and specific tools.

ChatGPT is not a search engine, so if you're looking for existing documents it's a very bad choice. But I've found myself using Perplexity—an LLM-powered search engine—more and more often because it reliably turns up results that Google fails to turn up.

I suspect Perplexity is still also the wrong tool for scholarly articles, but that's not a fundamental limitation of the tech, it's just a question of the focus of the tools so far.

dyauspitr• 2 years ago

What are you talking about? I haven’t had a hallucinated link since 3.5 to paper sources. They must have some sort of post processing in place to remove made up links.

rsynnott• 2 years ago

I just don’t understand the childish motivated reasoning behind assuming the skeptics are scared. Maybe if I spent “three sleepless nights” talking to ChatGPT I would be more enlightened.

A lot of people are very invested, whether emotionally or financially or both, in this stuff not being a flop. There’s a lot of motivated reasoning going on. It is necessary to believe that the heretics simply haven’t seen the light yet - to question that gets too close to questioning whether there’s any light to see.

karaterobot• 2 years ago

Q. Isn’t it inevitable that AI will make us lazier?
A. Calculators also made us lazier. Why aren’t we doing math by hand anymore? You should be taking notes by hand now instead of recording me. We use technology to take shortcuts, but we have to be strategic in how we take those shortcuts.

An unpopular opinion I have is that most of the doomsaying about technology making us dumber is true. Yes, even back to Socrates. I won't say all, but I'd safely say a lot. What happened was that we developed tools, lost certain capacities without necessarily losing the capabilities that came with them, and redefined the level a normal human should function at. My only point is that people don't like to think that maybe they themselves are less intelligent—in many ways—than people who urinated outside and didn't know what the sky was. But I don't see how it could be any other way. When we say things like "I don't need to remember, I can write it down", and "I don't need to do arithmetic in my head, I'll let a calculator do it", or "I don't need to read the article, someone will explain it in the comments" we are accepting the consequences of that, good and bad.

master_crab• 2 years ago

People aren’t dumber, or smarter. They just focus on what’s the next important thing to tackle.

For example, I doubt any website programmer knows the circuitry, assembly code, OS level calls, networking, etc, that make any webpage element do anything. Let alone can sit there and calculate any of the mathematical requirements needed to do any of that. But they know how to use an IDE and a framework like React.

All this is a long way of saying:

…on the shoulders of giants. AI is just the new tool needed for the next step up.

aliasxneo• 2 years ago

Yeah, I'm not sure I understand how moving up an abstraction layer necessarily makes you less intelligent. I also don't think it makes you smarter. For the average individual, I feel like it simply moves your "cursor" up the stack, but you're not necessarily increasing your context window.

Perhaps the confusion comes from the fact that we often produce more complex things as we move up layers. It's then assumed that the people who made them must be more intelligent, but as I said, I don't think that's a fair assessment.

I would say the real measurement for intelligence here is how much of the abstraction layers you actually understand. In other words, can you move your cursor back down the stack and operate just as well as in the higher layers? Can you do this while unifying the complex interactions between each layer into a cohesive model? I've noticed that even AI tends to be pretty bad at this last step. It often takes prodding to get it to see the subtle errors often introduced when working with complex systems.

mckravchyk• 2 years ago

It's a completely different thing. You are talking about civilisation constructs, parent is talking about things like mental fitness, abilities to perform tasks in real-time by yourself.

For example, card payments are a crutch. If you pay by card / phone everywhere and then out of the sudden you are to pay in cash, it becomes mildly challenging vs. if you are used to pay in cash you don't think about it. The brain is capable of a great deal of automation, performing learned actions is effortless. Unlike a calculator or a spreadsheet, the buyer is not doing anything, just buying. It's not a bicycle, it's a crutch. It simply atrophies the mental bandwidth. The mind becomes more lax, less sharp, when it does not engage.

Now imagine what it will do to people's brains when instead of thinking about solutions themselves, they will ask the AI for everything. Those neurons will atrophy and the person will be even less skilled to ask the AI the right questions than if they did not use the AI in the first place. I think the key will be a balance between doing the work yourself and delegating the stuff to AI, but it will be difficult to find that balance. Just like smartphones can be very useful but in the end are a net negative to society.

aniviacat• 2 years ago

redefined the level a normal human should function at

To a level much higher.

We stopped doing many repetitive, tedious things, but in return moved to things that are way more abstract and complex.

And that's happening everywhere. Even farmers are getting ever closer to being full on system architects.

Oh, you didn't learn to do quickly calculate square roots in your head? Instead you spent that time on learning about relativity in high school physics class.

By calling the people of the past smarter, you are really underselling the amount and depth of abstract though happening everywhere today.

Izkata3• 2 years ago

Oh, you didn't learn to do quickly calculate square roots in your head? Instead you spent that time on learning about relativity in high school physics class.

The math examples always get me, because there's a level of understanding missing here that, yes, you should still have:

You may not be able to do the square-root yourself, but you should have a good-enough understanding of what a square-root is that if you enter into a calculator the square-root of a 6-digit number and get back another 6-digit number, you should know immediately that's wrong, and double-check you entered it correctly.

Part of the concern a lot of us have is people who not only do not have that understanding, but that think it's unnecessary to get that understanding, and skip over it entirely.

mamcx• 2 years ago

I agree, and maybe a better frame is 'capable'.

I can do math with a calculator, but if it is taken away?

I can feed myself with doordash, but if it is taken away?

I can program a complex web-scale app, but if all those tools are taken away?

What is left?

Somebody who will die fast.

Reliance on all of this is removing agency and resiliency. By the law of numbers, the planet still has people who know some of the fundamentals that make the existence of the rest viable.

But if it is taken away?

valval4• 2 years ago

I think technology makes the people who create it smarter, and doesn’t affect average people.

disambiguation• 2 years ago

Oh but can't you see, laziness is one of the greatest motivators of all time. The desire to unburden ourselves from the tedious and faulty process of mental math lead to the invention of the calculator. Walking is hard so we domesticated horses and invented automobiles. Sending messages takes forever so we'll lay a telegram cable across the ocean floor.

I no longer need to calculate in my head, walk 10 miles to get food, or wait for weeks to hear from my friends who moved across the country. Maybe it's making me dumber, lazier, weaker compared to my forerunners, but dumbest of all would be to ignore these advantages just because I'm no longer appealing to some guys idea of Ideal Intelligence.

Smithalicious• 2 years ago

Damn kids, back in my day we had to copy-paste our homework from stackoverflow uphill both ways

Asraelite• 2 years ago

"Free browsing by accepting cookies" or "subscribe and decline".

Is this legal? It might be, but I've never seen other sites do it so it seems dubious.

bachmeier• 2 years ago

This type of article is so frustrating. "You need to use AI to make yourself more productive." Followed by zero explanations of how I can do that. In addition, no mention of the implications of sending all your personal information to an entity that is waiting for an opportunity to use it against you.

brookst• 2 years ago

I’m old enough to remember when the same claims were made of graphic calculators and high lever languages like Basic.

A better formulation is perhaps “students who use AI to reduce work learn different things”. It’s easy for purists to say there’s no value whatsoever in learning to use a tool rather than learning to do the work.

But that’s a judgment about the value of what is learned, and it’s kind of dishonest slight of hand to substitute that opinion.

EVa5I7bHFq9mnYK• 2 years ago

Students who use assembly language as a crutch, don't learn the machine codes properly.

ethn• 2 years ago

‪ZIZEK: that AI will be the death of learning & so on; to this, I say NO! My student brings me their essay, which has been written by AI, & I plug it into my grading AI, & we are free! While the 'learning' happens, our superego satisfied, we are free now to learn whatever we want‬

nunez3• 2 years ago

Yes, but therein lies the rub. Those that know how to learn will benefit. Those that don't will regress, possibly for life depending on when AI is introduced.

This is especially demonstrated in essay writing.

Many students associate essays with busy work because the topics they're asked to write about are boring. When the typical assignment that's given is "read this boring ass book from the 40s that's been in the curriculum for decades without revisiting its application in today's world, then write a 1000-word essay on a topic that's been discussed to death that you couldn't give less of a shit about; points will be deducted for views that stray too far from the norm," then it's absolutely unsurprising that most students will shove this into ChatGPT and call it a day.

On the flip side, when English or composition teachers are forced to assign thess assignments knowing full well that it's a crock of shit, then it is equally unsurprising that they will feed GPT into GPT and call it a day.

Students that know how to learn and are actually interested in becoming better writers will find ways around this. Teachers who have the freedom to design their own curriculums will be more creative about the types of prompts they assign and the books they have their students read.

The common link between the two? Money, of course!

xigoi2• 2 years ago

Why don’t you teach the student what they need in the first place?

lolinder2• 2 years ago

The headline implies something other than what the interviewee is saying:

Q. You don’t like to call AI a crutch.
A. The crutch is a dangerous approach because if we use a crutch, we stop thinking. Students who use AI as a crutch don’t learn anything. It prevents them from thinking. Instead, using AI as co-intelligence is important because it increases your capabilities and also keeps you in the loop.
Q. Isn’t it inevitable that AI will make us lazier?
A. Calculators also made us lazier. Why aren’t we doing math by hand anymore? You should be taking notes by hand now instead of recording me. We use technology to take shortcuts, but we have to be strategic in how we take those shortcuts.

vunderba2• 2 years ago

The danger in the eventual ubiquitous availability of large language models (LLMs) isn't necessarily that they can seemingly answer any question.

The real issue arises when it becomes far too tempting to immediately turn to an LLM for an answer, rather than taking a few moments to quietly ponder the problem on your own, engaging and manipulating, exploring different angles, etc. This kind of abstract thinking is a craft that only improves with consistent practice and deliberate effort.

renewiltord• 2 years ago

Highly useful tools are always described like this. Amusingly, search engines and then, to a lesser extent, Stack Overflow were both described like this. I can’t say that it’s very interesting a statement in its nth incarnation.

htk• 2 years ago

Terrible agenda-driven title, implying that the interviewee's main message was about the dangers of relying on AI, where in fact it was the opposite, how AI can be a great tool to elevate human capacity if used well.

StarterPro• 2 years ago

No shit. Just like people who "create AI art" aren't actually creating anything.

We've let these tech companies distill learning and creating down to a mouse click.

sirsuki• 2 years ago

In struggled with this trend even before the AI hype. I love learning and focused my work towards always learning. But it takes energy and lots of work; some days are better than others. My love of learning keeps me motivated over time. However, I’ve noticed a downward trend of juniors who are focused on being spoon fed the answers and avoiding any learning. It drives me nuts and I don’t know how to reconcile this kind of mental model.

adamnemecek• 2 years ago

Do students who don’t use AI learn anything though?

6SixTy• 2 years ago

I've been working with a CS student that often does not understand why her code doesn't work, does not want to understand how to improve herself, nor want break down a situation to it's bare components and apply them to new situations.

I could give a bunch of long (and probably butchered) stories of my sessions with her, but in short, she's expects to be spoon fed the answers without properly digesting why they are the answers. She turns to ChatGPT because it will just automatically give her the answers without forcing her to do any thinking, just copy + paste in order to tick off another assignment.

There's probably a middle ground where you could use ChatGPT properly, but I've had her reach towards ChatGPT once for what essentially amounts to spelling errors that she never notices herself or pay attention to errors telling her that she spelled something wrong. It's kind of hard to be supportive of a tool when the tool itself can essentially act like a replacement of your thinking cap and can't say no.

kazinator• 2 years ago

AI does so many things that we need to set guardrails on what we don’t want to give up.

Currently, we can count on one hand with AI does, and have fingers left over.

It manipulates language (including formal language like computer code) with a modicum of logical reasoning, provides conversational access into the knowledge stored in a corpus of text, and generates/transforms images.

AI will currently not fix your leaky faucet, cook you dinner or take your dog for a walk.

'Students who use AI as a crutch don't learn anything'

Related Stories