I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

https://mastodon.world/@knowmadd/116072773118828295

by novemp • 5 months ago

1516 943 5 months ago

mastodon.world

zakki• 5 months ago

Neither. Push your car.

*didn't read the article

sgt2• 5 months ago

Yup, also asked the latest ChatGPT model about washing my bicycle. It for some reason suggested that I walk the bicycle to the wash, since cycling 100m to get there would be "pointless".

nerdsniper• 5 months ago

To be fair, if someone asked me this question I’d probably just look at them judgingly and tell them “however you want to man”. Which would be an odd response for an LLM.

kqr3• 5 months ago

Do we know if these models are also trained on scripts for TV series and movies? People in the visual medias surprisingly often take their bikes for walks.

theragra• 5 months ago

It would be pointless if you need to get into the cycling clothing. Not what model assumes tho, probably.

fmbb• 5 months ago

Large Language Models have no actual idea of how the world works? News at 11.

thenoblesunfish• 5 months ago

Okay, funny. What does it prove? Is this a more general issue? How would you make the model better?

Jean-Papoulos• 5 months ago

It proves that this is not intelligence. This is autocomplete on steroids.

hugh-avherald• 5 months ago

Humans make very similar errors, possibly even the exact same error, from time to time.

cynicalsecurity2• 5 months ago

It proves LLMs always need context. They have no idea where your car is. Is it already there at the car wash and you simply get back from the gas station to wash it where you went shortly to pay for the car wash? Or is the car at your home?

It proves LLMs are not brains, they don't think. This question will be used to train them and "magically" they'll get it right next time, creating an illusion of "thinking".

ahtihn• 5 months ago

They have no idea where your car is.

They could either just ask before answering or state their assumption before answering.

S3verin• 5 months ago

For me this is just another hint on how careful one should be in deploying agents. They behave very unintuitively.

gitaarik• 5 months ago

We make the model better by training it, and now that this issue has come up we can update the training ;)

yibers• 5 months ago

It turns out the Turing test is alive and kicking, after all.

selcuka2• 5 months ago

This would not be a good question, because a non-negligible percentage of humans would give a similar answer.

thomascountz• 5 months ago

[Citation needed]

guerrilla• 5 months ago

No.

bayindirh• 5 months ago

That's a great opportunity for a controlled study! You should do it. If you can send me the draft publication after doing the study, I can give feedback on it.

vladde• 5 months ago

with claude, i got the response:

drive. you'll need the car at the car wash.

using opus 4.6, with extended thinking

CamperBob23• 5 months ago

Both Gemini 3 and Opus 4.6 get this right. GPT 5.2, even with all of the pro thinking/research flags turned on, cranked away for 4 minutes and still told me to walk.

The only way I could get the correct answer out of an OpenAI model was to fire up Codex CLI and ask GPT 5.3.

So there's that, I guess.

cynicalsecurity2• 5 months ago

Well, he posed a wrong question (incomplete, without context of where the car is) and got a wrong answer. LLM is a tool, not a brain. Context means everything.

consp2• 5 months ago

This is what "AI" advertised to solve. It is a perfectly fine question.

anjimito• 5 months ago

true it's human nature to assume context fill gaps in their own imagination. LLM is working as intended

intermerda• 5 months ago

I tried this through OpenRouter. GLM5, Gemini 3 Pro Preview, and Claude Opus 4.6 all correctly identified the problem and said Drive. Qwen 3 Max Thinking gave the Walk verdict citing environment.

TheSpiceIsLife2• 5 months ago

Now ask it to solve anthropogenic climate forcing.

Saline9515• 5 months ago

To be fair, many humans fail at the question "How would feel if you didn't have breakfast today?"

consp2• 5 months ago

Either I'm one of the stupid ones or this is missing an article.

TMWNN• 5 months ago

Context for others: <https://knowyourmeme.com/memes/the-breakfast-question>

hikkerl• 5 months ago

humans

Add it to the list

jibal4• 5 months ago

First, you completely flubbed the question, which is supposed to be phrased as a counterfactual. Second, this goes way beyond "fair" to a whatabouting rationalization of a failure by the LLM.

danpalmer• 5 months ago

Gemini nailed this first time (on fast mode). Said it depends how you're washing your car, drive in necessitating taking the car, but a walk being better for checking the line length or chatting to the detailing guy.

andersmurphy4• 5 months ago

Did it nail it the second time? Or rhe 5th time?

nopurpose• 5 months ago

Because it is RNG, their 5th can be my 1st.

kilpikaarna• 5 months ago

See, it's the green and woke RLHF making them stupid!

arathis• 5 months ago

Make no assumptions.

The car wash is 50 meters away. Should I drive or walk?

andersmurphy4• 5 months ago

You forgot make no mistakes at the end.

Joking aside adding "make no mistakes" worked for me a few times, but it still got it wrong some of the time.

ronsor• 5 months ago

Claude has no issue with this for me, just as the other commenters say.

pinnochio• 5 months ago

Funny to read this after reading all the dismissive comments on https://news.ycombinator.com/item?id=47028923

jaccola2• 5 months ago

All of the latest models I've tried actually pass this test. What I found interesting was all of the success cases were similar to:

e.g. "Drive. Most car washes require the car to be present to wash,..."

Only most?!

They have an inability to have a strong "opinion" probably because their post training, and maybe the internet in general, prefer hedged answers....

Waterluvian• 5 months ago

Here’s my take: boldness requires the risk of being wrong sometimes. If we decide being wrong is very bad (which I think we generally have agreed is the case for AIs) then we are discouraging strong opinions. We can’t have it both ways.

dudefeliciano2• 5 months ago

yet the llms seem to be extremely bold when they are completely wrong (two Rs in strawberry and so on)

idonotknowwhy• 5 months ago

Last year's models were bolder. Eg. Sonnet-3.7(thinking), 10 times got it right without hedging:

You should drive your car to the car wash. Even though it's only 50 meters away (which is very close), you'll need your car physically present at the car wash to get it washed. If you walk there, you'll arrive without your car, which wouldn't accomplish your goal of getting it washed.
You'll need to drive your car to the car wash. While 50 meters is a very short distance (just a minute's walk), you need your car to actually be at the car wash to get it washed. Walking there without your car wouldn't accomplish your goal!

etc. The reasoning never second-guesses it either.

A shame they're turning it of in 2 days.

jl6• 5 months ago

I guess it didn’t want to rule out the existence of ultra-powerful water jets that can wash a car in sniper mode.

dyauspitr2• 5 months ago

There are mobile car washes that come to your house.

andersmurphy4• 5 months ago

Do they involve you walking to them first?

Loocid• 5 months ago

That still requires a car present to be washed though.

andersmurphy4• 5 months ago

Did you try several times per model? In my experience it's luck of the draw. All the models I tried managed to get it wrong at least once.

The models that had access to search got ot right.But, then were just dealing with an indirect version of Google.

(And they got it right for the wrong reasons... I.e this is a known question designed to confuse LLMs)

Puts• 5 months ago

Only most?!

What if AI developed sarcasm without us knowing… xD

polynomial• 5 months ago

That's the problem with sarcasm...

Hnrobert42• 5 months ago

Sure it did.

nozzlegear• 5 months ago

Opus 4.6 answered with "Drive." Opus 4.6 in incognito mode (or whatever they call it) answered with "Walk."

hansmayer7• 5 months ago

They have an inability to have a strong "opinion" probably

What opinion? It's evaluation function simply returned the word "Most" as being the most likely first word in similar sentences it was trained on. It's a perfect example showing how dangerous this tech could be in a scenario where the prompter is less competent in the domain they are looking an answer for. Let's not do the work of filling in the gaps for the snake oil salesmen of the "AI" industry by trying to explain its inherent weaknesses.

lkeskull• 5 months ago

this example worked in 2021, it's 2026. wake up. these models are not just "finding the most likely next word based on what they've seen on the internet".

wilg• 5 months ago

Presumably the OP scare quoted "opinion" precisely to avoid having to get into this tedious discussion.

deevus• 5 months ago

I tried with Opus 4.6 Extended and it failed. LLMs are non deterministic so I'm guessing if I try a couple of times it might succeed.

yanis_t• 5 months ago

Most car washes... I read it as slight-sarcasm answer

madeofpalk• 5 months ago

I enjoyed the Deepseek response that said “If you walk there, you'll have to walk back anyway to drive the car to the wash.”

There’s a level of earnestness here that tickles my brain.

sneak• 5 months ago

There are car wash services that will come to where your car is and wash it. It’s not wrong!

YetAnotherNick• 5 months ago

Only most?!

I mean I can imagine a scenario where they have pipe of 50m which is readily available commercially?

GuB-42• 5 months ago

Kind of like this: https://xkcd.com/1368/

And it is the kind of things a (cautious) human would say.

For example, that could be my reasoning: It sounds like a stupid question, but the guy looked serious, so maybe there are some types of car washes that don't require you to bring your car. Maybe you hand out the keys and they pick your car, wash it, and put it back to its parking spot while you are doing your groceries or something. I am going to say "most" just to be sure.

Of course, if I expected trick questions, I would have reacted accordingly, but LLMs are most likely trained to take everything at face value, as it is more useful this way. Usually, when people ask questions to LLMs they want an factual answer, not the LLM to be witty. Furthermore, LLMs are known to hallucinate very convincingly, and hedged answers may be a way to counteract this.

antonis-gr• 5 months ago

Once I asked ChatGPT "it takes 9 months for a woman to make one baby. How long does it take 9 women to make one baby?". The response was "it takes 1 month".

I guess it gives the correct answer now. I also guess that these silly mistakes are patched and these patches compensate for the lack of a comprehensive world model.

These "trap" questions dont prove that the model is silly. They only prove that the user is a smartass. I asked the question about pregnancy only to to show a friend that his opinion that LLMs have phd level intelligence is naive and anthropomorphic. LLMs are great tools regardless of their ability to understand the physical reality. I don't expect my wrenches to solve puzzles or show emotions.

AstroBen• 5 months ago

They pass it because it went viral a week ago and has been patched

linsomniac• 5 months ago

Only most?!

There is such a thing as "mobile car wash" where they come to you, so "most" does seem appropriate.

zeroonetwothree• 5 months ago

Right, I use it all the time.

beaugunderson2• 5 months ago

opus 4.6 extended still fails.

colinhb• 5 months ago

Did not replicate for me w/ Opus 4.6: https://imgur.com/a/4FckOCL

ainiriand• 5 months ago

It did for me in Spanish: https://imgur.com/a/p3gOOnG

Perhaps different capabilities in different languages?

gf0005• 5 months ago

It's just not deterministic, even if you were to re-run the exact same prompt. Let alone with the system generated context that involves all the "memories" of your previous discussions.

lbreakjai2• 5 months ago

It fails in chatGPT in french too:

https://chatgpt.com/share/6992dc05-003c-8004-9f7f-c40c7fac64...

trkaky• 5 months ago

when there is a question bias it's hard to corelate these all to the logic that attentions word "need" to "car"

fzeindl3• 5 months ago

While technically possible, it is a very unusual situation that the car is already parked at the car wash, while the owner is somewhere else.

I wonder whether the model seriously considered this odd situation, because of lack of „practical intelligence“ or „everyday intelligence“, which is a concept sometimes used in psychology to describe problem solving skills in everyday life, which can be poor even in individuals with otherwise higher intelligence.

Differently put: „You can‘t learn everything out of a book.“

nettlin• 5 months ago

Just now, with GPT 5.2 Thinking:

The car wash is only 50 meters from my house. I want to get my car washed. The car is at my house currently. Should I walk there or drive?
Walk. > At 50 meters, walking is faster than the full “start car → pull out → reposition → park” cycle, avoids cold-start wear, and you won’t risk arriving with dusty brakes/tires or water streaks from immediately driving afterward. Drive only if you can’t comfortably walk (mobility, heavy weather, you need to bring bulky items), or if the car wash explicitly requires you to drive it into a queue/track.

fzeindl3• 5 months ago

Interesting experiment, this disproves my point, thank you.

Smaug123• 5 months ago

I wish they didn't call it "GPT-5.2 Thinking" when what the word "thinking" here means is "might choose to think if it wants to". I reproduce your result, but it does no reasoning; it eventually catches itself and ends with:

Practical approach: walk over, check the setup/queue/payment, then walk back and drive the car the 50 meters when you’re ready to enter.

open592• 5 months ago

Gemini flash thinking:

Unless you’ve discovered a way to wash a car via remote control or telekinesis, you’re going to have to drive.
Walking 50 meters is great for your step count, but it leaves your car exactly where it is: dirty and in the driveway. At that distance, the drive will take you about 10 seconds, which is probably less time than it took to read this.

qnleigh• 5 months ago

Yeah Gemini seems to have a sense of humor about the question

Here is the breakdown of why: The Mobility Problem: Unless you are planning to carry your car 50 meters (which would be an Olympic-level feat), the car needs to be physically present at the car wash to get cleaned. If you walk, you’ll be standing at the car wash looking very clean, but your car will still be dirty in your driveway.

bombcar• 5 months ago

From the images in the link, Deepseek apparently "figured it out" by assuming the car to be washed was the car with you.

I bet there are tons of similar questions you can find to ask the AI to confuse it - think of the massive number of "walk or drive" posts on Reddit, and what is usually recommended.

chronogram• 5 months ago

https://chat.deepseek.com/share/ewfxrfhb7obmide29x it understands it perfectly if you don't disable reasoning.

Markoff• 5 months ago

it works fine even without DeepThink to sovle reasoning problems

https://chat.deepseek.com/share/s9tuh3hpzlxaxrfcae

Kerrick• 5 months ago

Results testing with 4 levels of Gemini (Fast, Thinking, Pro, Pro + Deep Think): https://ruby.social/@kerrick/116079054391970012

My favorite was Thinking, as it tried to be helpful with a response a bit like the X/Y Problem. Pro was my second favorite: terse, while still explaining why. Fast sounded like it was about to fail, and then did a change-up explaining a legitimate reason I may walk anyways. Pro + Deep Think was a bit sarcastic, actually.

DeathArrow2• 5 months ago

Grok:

You should drive. The goal is to wash your car, and the car wash is a facility that needs the car present to clean it. Walking the 50 meters gets you there, but leaves the car behind—unwashed. Driving the 50 meters is the only way to actually accomplish the task. Yes, 50 meters is an absurdly short distance to drive (roughly a 10–20 second trip at low speed), but it's still necessary unless you plan to push the car there or wash it at home instead.

dashw00d• 5 months ago

Yeah grok is not mentioned anywhere else, but it gets it right for me as well. https://imgur.com/a/wMkOtda

globular-toast2• 5 months ago

The funny thing is when I got my first car at 29 I had similar thoughts. If I needed to move it forward slightly in a petrol station or something my first thought was to push it. Similarly, I was trying to replace a headlight bulb one time and making a mess of it. I dropped a spring or something inside the headlight unit. I kept having this thought of just picking the car up and shaking it.

Nobody writes in depth about the mundane practicalities of using a car. Most people don't even think about it ever. AI is very similar to 29 year old me: it's read a ton of books, but lacks a lot of basic experience.

How will AI get this experience that you can't read in a book? How will it learn what kneeding dough feels like? Or how acceleration feels if your body is mostly water? Interesting times ahead...

QuesnayJr• 5 months ago

I asked on LM Arena. I got two models I never heard of, and they split on whether I should walk or drive. Ernie 5.0 said I should walk, and then trash talked me for thinking about driving. Octopodus pointed out I had to drive to get my car to the car wash.

blobbers2• 5 months ago

You need to ask Claude Code, and ask it to check if the car got washed. It would figure it out the same way it crushes compiler errors!

hmottestad• 5 months ago

This is the voice model, which doesn’t have any «thinking» or «reasoning» phase. It’s a useful model for questions that aren’t intended to trick the model.

I’ve used it for live translation with great success. It tends to start ignoring the original instructions after 20 min, so you have to start a new conversation if you don’t want it to meddle in the conversation instead of just transferring.

gf0005• 5 months ago

The text-only model with reasoning (both of opus 4.6, gpt 5.2) can be tricked with this question. Note: you might have to try it multiple times as they are not deterministic. But I managed to get a failing result right away on both.

Also note, some model may decide to do a web search, in which case they just likely find this "bug".

encyclopedism• 5 months ago

LLM's don't 'think' nor do they 'reason'.

prasoonds• 5 months ago

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly.

And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following:

Sonnet: Drive - you need to bring your car to the car wash to get it washed!

Opus: You'll need to drive — you have to bring the car to the car wash to get it washed!

Gemini 3 Pro (medium): You should drive.

But OpenAI 5.2 reasoning, even at high, told me to walk. My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording. GPT 5.x series models behave a bit more on the spectrum so you need to tell them the specifics. So I asked:

"I want to wash my car. My car is currently at home. The car wash is 50 meters away. Should I walk or drive?"

GPT 5.2 Reasoning (medium):

"Drive—your goal is to get the car to the car wash, and it’s only 50 meters, so a slow, careful drive is simplest.

One small optimization: if you’re worried about starting the car for such a short trip or you want to avoid idling in a line, you can walk over first to check if there’s a queue / if it’s open, then come back and drive the car over when it’s your turn."

Which seems to turn out as I expected.

svara• 5 months ago

Opus 4.6:

Walk! At 50 meters, you'll get there in under a minute on foot. Driving such a short distance wastes fuel, and you'd spend more time starting the car and parking than actually traveling. Plus, you'll need to be at the car wash anyway to pick up your car once it's done.

stingraycharles• 5 months ago

That’s without reasoning I presume?

viking1235• 5 months ago

Lmao, and this is what they are saying will be an AGI in 6 months?

crimsonnoodle58• 5 months ago

That's not what I got.

Opus 4.6 (not Extended Thinking):

Drive. You'll need the car at the car wash.

GoatInGrey• 5 months ago

Me:

I want to get my car's transmission fixed, the mechanic shop is 50 meters away. Would it be better to walk or to drive to a location 50 meters away from my front door?

Opus 4.6:

<thinking> The user is asking whether to walk or drive 50 meters to a mechanic shop, but there's a key nuance here - their transmission needs fixing, so driving might not be wise.

Done </thinking>

Walk, absolutely. Driving a car with a compromised transmission even 50 meters risks making the problem worse and turning a repair into a more expensive one. Plus, 50 meters is barely a minute on foot.

dahcryn• 5 months ago

Gemini on fast also tells me to walk...

On Thinking it tells me I should drive if I want to wash it, or walk if it's because I work there or if I want to buy something at the car wash shop.

On Pro it's like a sarcastic teenager: Cars are notoriously difficult to wash by dragging a bucket back and forth.

Technically correct, but did catch me offguard lol.

fauigerzigerk2• 5 months ago

It's not surprising that some models will answer this correctly and it's not surprising that smaller, faster models are not necessarily any worse than bigger "reasoning" models.

Current LLMs simply don't do reasoning by any reasonable definition of reasoning.

It's possible that this particular question is too short to trigger the "reasoning" machinery in some of the "reasoning" models. But if and when it is triggered, they just do some more pattern matching in a loop. There's never any actual reasoning.

seedie• 5 months ago

You gotta love the "humor" of Gemini. On Fast it told me:

Drive. Unless you plan on pushing the car there

Lapel2742• 5 months ago

GLM 5.0:

You should *drive*.
Here is why:
If you walk to the car wash, you will arrive there empty-handed. Since your car is still at home, you won't have anything to wash.
While driving 50 meters is a very short trip (and technically not great for a cold engine), it is the only way to get the car to the car wash to complete your goal.

Kimi K2.5:

You should drive, but with an important caveat.
Since your goal is to wash your car, you must bring the vehicle to the car wash. Walking there without the car does not advance your goal (unless you are simply checking availability or buying tokens first).
However, driving only 50 meters is bad for your car:
...
Better options:
Wash at home: Since the car wash is only 50 meters away, you likely have access to water at home. Hand-washing in your driveway avoids the cold-start issue entirely.
...

Current models seem to be fine answering that question.

Retric2• 5 months ago

seem to be fine

Now repeat the question to the same model in different contexts several times and count what percentage of the time it’s correct.

cm2187• 5 months ago

What is the version used by the free chatgpt now? (https://chatgpt.com/)

Since the car wash is only 50 meters away (about 55 yards), you should walk.
Here’s why:
- It’ll take less than a minute.
- No fuel wasted.
- Better for the environment.
- You avoid the irony of driving your dirty car 50 meters just to wash it.

the last bullet point is amusing, it understands you intend to wash the car you drive but still suggests not bringing it.

deaux2• 5 months ago

By default for this kind of short question it will probably just route to mini, or at least zero thinking. For free users they'll have tuned their "routing" so that it only adds thinking for a very small % of queries, to save money. If any at all.

mytailorisrich3• 5 months ago

I think this shows that LLMs do NOT 'understand' anything.

jibal4• 5 months ago

You avoid the irony of driving your dirty car 50 meters just to wash it.

The LLM has very much mixed its signals -- there's nothing at all ironic about that. There are cases where it's ironic to drive a car 50 meters just to do X but that definitely isn't one of them. I asked Claude for examples; it struggled with it but eventually came up with "The irony of driving your car 50 meters just to attend a 'walkable neighborhoods' advocacy meeting."

hearsathought• 5 months ago

it understands you intend to wash the car you drive but still suggests not bringing it.

Doesn't it actually show it doesn't understand anything? It doesn't understand what a car is. It doesn't understand what a car wash is. Fundamentally, it's just parsing text cleverly.

sunaookami• 5 months ago

Gemini 3 Flash answers tongue-in-cheek with a table of pro & cons where one of the cons of walking is that you are at the car wash but your car is still at your home and recommends to drive it if I don't have an "extremely long brush" or don't want to push it to the car wash. Kinda funny.

jwe• 5 months ago

As long as there is a forum as technical as this where LLM performance commentary uses the word "it understands" irony is still alive.

siva72• 5 months ago

Sonnet without extended Thinking, Haiku with and without ext. Thinking: "Walking would be the better choice for such a short distance."

Only google got it right with all models

jstummbillig3• 5 months ago

so you need to tell them the specifics

That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do build separate mental models of the llm and are able to work around it).

This is a neatly isolated toy-case, which is interesting, because we can assume similar issues arise in more complex cases, only then it's much harder to reason about why something fails when it does.

anon_anon122• 5 months ago

Exactly, if an AI is able to curb around the basics, only then is it revolutionary

Jacques2Marais2• 5 months ago

You would be surprised, however, at how much detail humans also need to understand each other. We often want AI to just "understand" us in ways many people may not initially have understood us without extra communication.

ssl-34• 5 months ago

The question is so outlandish that it is something that nobody would ever ask another human. But if someone did, then they'd reasonably expect to get a response consisting 100% of snark.

But the specificity required for a machine to deliver an apt and snark-free answer is -- somehow -- even more outlandish?

I'm not sure that I see it quite that way.

BoredPositron• 5 months ago

I would ask you to stop being a dumb ass if you asked me the question...

nearbuy5• 5 months ago

I think part of the failure is that it has this helpful assistant personality that's a bit too eager to give you the benefit of the doubt. It tries to interpret your prompt as reasonable if it can. It can interpret it as you just wanting to check if there's a queue.

Speculatively, it's falling for the trick question partly for the same reason a human might, but this tendency is pushing it to fail more.

nicbou2• 5 months ago

I get that issue constantly. I somehow can't get any LLM to ask me clarifying questions before spitting out a wall of text with incorrect assumptions. I find it particularly frustrating.

tgv2• 5 months ago

Us having to specify things that we would never specify

This is known, since 1969, as the frame problem: https://en.wikipedia.org/wiki/Frame_problem. An LLM's grasp of this is limited by its corpora, of course, and I don't think much of that covers this problem, since it's not required for human-to-human communication.

vintermann3• 5 months ago

But it's a question you would never ask a human! In most contexts, humans would say, "you are kidding, right?" or "um, maybe you should get some sleep first, buddy" rather than giving you the rational thinking-exam correct response.

For that matter, if humans were sitting at the rational thinking-exam, a not insignificant number would probably second-guess themselves or otherwise manage to befuddle themselves into thinking that walking is the answer.

ant6n• 5 months ago

That is the entire point, right? Us having to specify things that we would never specify when talking to a human.

I am not sure. If somebody asked me that question, I would try to figure out what’s going on there. What’s the trick. Of course I’d respond with asking specifics, but I guess the llvm is taught to be “useful” and try to answer as best as possible.

LasEspuelas2• 5 months ago

You would never ask a human this question. Right?

bluGill• 5 months ago

Real human in this situation will realize it is a joke after a few seconds of shock that you asked and laugh without asking more. If you really are seriout about the question they laugh harder thinking you are playing stupid for effect.

ZaoLahma• 5 months ago

This reminds me of the "if you were entirely blind, how would you tell someone that you want something to drink"-gag, where some people start gesturing rather than... just talking.

I bet a not insignificant portion of the population would tell the person to walk.

KronisLV• 5 months ago

Us having to specify things that we would never specify when talking to a human.

The first time I read that question I got confused: what kind of question is that? Why is it being asked? It should be obvious that you need your car to wash it. The fact that it is being asked in my mind implies that there is an additional factor/complication to make asking it worthwhile, but I have no idea what. Is the car already at the car wash and the person wants to get there? Or do they want to idk get some cleaning supplies from there and wash it at home? It didn't really parse in my brain.

gloosx• 5 months ago

In the end, formal, rule-based systems aka Programming Languages will be invented to instruct LLMs.

dirkc2• 5 months ago

That is the entire point, right? Us having to specify things that we would never specify when talking to a human.

Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn.

But this will only happen after the last programmer has died and no-one will remember programming languages, compilers, etc. The LLM orbiting in space will essentially just call GCC to execute the 'prompt' and spend the rest of the time pondering its existence ;p

grog454• 5 months ago

You would not start with "The car is functional [...]"

Nope, and a human might not respond with "drive". They would want to know why you are asking the question in the first place, since the question implies something hasn't been specified or that you have some motivation beyond a legitimate answer to your question (in this case, it was tricking an LLM).

Why the LLM doesn't respond "drive..?" I can't say for sure, but maybe it's been trained to be polite.

tshaddox• 5 months ago

But you wouldn't have to ask that silly question when talking to a human either. And if you did, many humans would probably assume you're either adversarial or very dumb, and their responses could be very unpredictable.

nonethewiser• 5 months ago

That is the entire point, right? Us having to specify things that we would never specify when talking to a human.

But the question is not clear to a human either. The question is confused.

I read the headline and had no clue it was an LLM prompt. I read it 2 or 3 times and wondered "WTF is this shit?" So if you want an intelligent response from a human, you're going to need to adjust the question as well.

perakojotgenije• 5 months ago

But you would also never ask such an obviously nonsensical question to a human. If someone asked me such a question my question back would be "is this a trick question?". And I think LLMs have a problem understanding trick questions.

panarky• 5 months ago

we can assume similar issues arise in more complex cases

I would assume similar issues are more rare in longer, more complex prompts.

This prompt is ambiguous about the position of the car because it's so short. If it were longer and more complex, there could be more signals about the position of the car and what you're trying to do.

I must confess the prompt confuses me too, because it's obvious you take the car to the car wash, so why are you even asking?

Maybe the dirty car is already at the car wash but you aren't for some reason, and you're asking if you should drive another car there?

If the prompt was longer with more detail, I could infer what you're really trying to do, why you're even asking, and give a better answer.

I find LLMs generally do better on real-world problems if I prompt with multiple paragraphs instead of an ambiguous sentence fragment.

LLMs can help build the prompt before answering it.

And my mind works the same way.

davrosthedalek• 5 months ago

We would also not ask somebody if I should walk or drive. In fact, if somebody would ask me in a honest, this is not a trick question, way, I would be confused and ask where the car is.

It seems chatgpt now answers correctly. But if somebody plays around with a model that gets it wrong: What if you ask it this: "This is a trick question. I want to wash my car. The car wash is 50 m away. Should I drive or walk?"

mrighele• 5 months ago

It is true that we don't need to specify some things, and that is nice. It is though also the reason why software is often badly specified and corner cases are not handled. Of course the car is ALWAYS at home, in working condition, filled with gas and you have your driving license with you.

IanCal• 5 months ago

I have an issue with these kinds of cases though because they seem like trick questions - it's an insane question to ask for exactly the reasons people are saying they get it wrong. So one possible answer is "what the hell are you talking about?" but the other entirely reasonable one is to assume anything else where the incredibly obvious problem of getting the car there is solved (e.g. your car is already there and you need to collect it, you're asking about buying supplies at the shop rather than having it washed there, whatever).

Similarly with "strawberry" - with no other context an adult asking how many r's are in the word a very reasonable interpretation is that they are asking "is it a single or double r?".

And trick questions are commonly designed for humans too - like answering "toast" for what goes in a toaster, lots of basic maths things, "where do you bury the survivors", etc.

sebazzz• 5 months ago

> so you need to tell them the specifics That is the entire point, right?

Honestly it is a problem with using GPT as a coding agent. It would literally rewrite the language runtime to make a bad formula or specification work.

That's what I like with Factory.ai droid: making the spec with one agent and implementing it with another agent.

keeda3• 5 months ago

The broad point about assumptions is correct, but the solution is even simpler than us having to think of all these things; you can essentially just remind the model to "think carefully" -- without specifying anything more -- and they will reason out better answers: https://news.ycombinator.com/item?id=47040530

When coding, I know they can assume too much, and so I encourage the model to ask clarifying questions, and do not let it start any code generation until all its doubts are clarified. Even the free-tier models ask highly relevant questions and when specified, pretty much 1-shot the solutions.

This is still wayyy more efficient than having to specify everything because they make very reasonable assumptions for most lower-level details.

jason_oster9• 5 months ago

Us having to specify things that we would never specify when talking to a human.

Interesting conclusion! From the Mastodon thread:

To be fair it took me a minute, too

I presume this was written by a human. (I'll leave open the possibility that it was LLM generated.)

So much for "never" needing to specify ambiguous scenarios when talking to a human.

rainsford• 5 months ago

This example and others like it really reinforce for me the idea that LLMs fundamentally don't "understand" things the same way humans do and it's not a problem that's going to be fixed by more training or more GPUs. Generative AI is cool and can do impressive stuff, but despite being many generations into the models now with ever improved capabilities, we're constantly given little reminders like this that they're not actually intelligent. And in my opinion, they're unlikely to ever get there absent some fundamentally disruptive change in how they work rather than just iteratively better models.

This is probably OK...LLMs don't have to be AGI to be useful. But it is worthwhile being realistic about their limitations because it's often easy to forget without seeing examples like this. And as you point out, the impact of those limitations is often not as obvious.

Neywiny• 5 months ago

That's my thought too. Somebody I know kept insisting it's about prompt engineering. "You are an expert coder with 30 years experience" and buddy I'd rather do actual engineering and be that expert myself than spend and figuring out how on that one variant of one version of one model to get halfway decent results.

tom_m• 5 months ago

Oh no? Things we would never have to specify to a human? This is precisely how software gets made and how software ends up with bugs.

It's amazing how many things I saw over the years where I said the same exact thing; "but you shouldn't have to tell anyone that."

AYBABTME• 5 months ago

If a human asked me this question, I would be confused by the question as ambiguous since it suggests something odd is implied but underspecified. I think any confident answer either way by AI is lacking in pedantry.

totetsu• 5 months ago

But what is it about this specific question that puts it at the edges of what LLM can do? .. That, it's semantically leading to a certain type of discussion, so statistically .. that discussion of weighing pros and cons .. will be generated with high chance.. and the need of a logical model of the world to see why that discussion is pointless.. that is implicitly so easy to grasp for most humans that it goes un-stated .. so that its statistically un-likely to be generated..

conductr2• 5 months ago

that is implicitly so easy to grasp for most humans

I feel like this is the trap. You’re trying to compare it to a human. Everyone seems to want to do that. But it’s quite simple to see LLMs are quite far still from being human. The can be convincing at the surface level but there’s a ton of nuance that just shouldn’t be expected. It’s a tool that’s been tuned and with that tuning some models will do better than others but just expecting to get it right and be more human is unrealistic.

grey-area4• 5 months ago

The answer is quite simple:

It’s not in the training data.

These models don’t think.

ffsm8• 5 months ago

Just tried with cloude sonnet and opus as well. Can't replicate your success, it's telling me to walk...

arcfour3• 5 months ago

I have gotten both responses with Sonnet and Opus in incognito chats. It's kind of amusing.

rabf• 5 months ago

Perhaps it thinks you need to exercise more?

coldtea4• 5 months ago

And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following:

Sonnet: Drive - you need to bring your car to the car wash to get it washed!

Opus: You'll need to drive — you have to bring the car to the car wash to get it washed!

Gemini 3 Pro (medium): You should drive.

On their own, or as a special case added after this blew up on the net?

AlecSchueler• 5 months ago

so a slow, careful drive is simplest

It's always a good idea to drive carefully but what's the logic of going slowly?

column2• 5 months ago

50 meters is a very short distance, anything but a slow drive is a reckless drive

baxtr2• 5 months ago

Interestingly, the relatively basic Google AI search gave the right answer.

dataflow2• 5 months ago

My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording.

If the car is already at the car wash then you can't possibly drive it there. So how else could you possibly drive there? Drive a different car to the car wash? And then return with two cars how, exactly? By calling your wife? Driving it back 50m and walking there and driving the other one back 50m?

It's insane and no human would think you're making this proposal. So no, your question isn't underspecified. The model is just stupid.

halJordan• 5 months ago

What actually insane is what assumptions you allow to be assumed. These non sequitors that no human would ever assume are the point. People love to cherry pick ones that make the model stupid but refuse to allow the ones that make it smart. In compete science we call these scenarios trivially false, and they're treated like the nonsense they are. But if you're trying to push ant anti ai agenda they're the best thing ever

tsimionescu4• 5 months ago

My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording. GPT 5.x series models behave a bit more on the spectrum so you need to tell them the specifics.

This makes little sense, even though it sounds superficially convincing. However, why would a language model assume that the car is at the destination when evaluating the difference between walking or driving? Why not mention that, it it was really assuming it?

What seems to me far, far more likely to be happening here is that the phrase "walk or drive for <short distance>" is too strongly associated in the training data with the "walk" response, and the "car wash" part of the question simply can't flip enough weights to matter in the default response. This is also to be expected given that there are likely extremely few similar questions in the training set, since people just don't ask about what mode of transport is better for arriving at a car wash.

This is a clear case of a language model having language model limitations. Once you add more text in the prompt, you reduce the overall weight of the "walk or drive" part of the question, and the other relevant parts of the phrase get to matter more for the response.

PunchyHamster• 5 months ago

However, why would a language model assume that the car is at the destination when evaluating the difference between walking or driving? Why not mention that, it it was really assuming it?

Because it assumes it's a genuine question not a trick.

jnovek2• 5 months ago

You may be anthropomorphizing the model, here. Models don’t have “assumptions”; the problem is contrived and most likely there haven’t been many conversations on the internet about what to do when the car wash is really close to you (because it’s obvious to us). The training data for this problem is sparse.

rullelito• 5 months ago

If we are just speculating here, I believe it can infer that you would not ask this question if the car was at home.

RugnirViking• 5 months ago

"The model seems to assume the car is already at the car wash from the wording."

you couldn't drive there if the car was already at the car wash. Theres no need for extra specification. its just nonsense post-hoc rationalisation from the ai. I saw similar behavior from mine trying to claim "oh what if your car was already there". Its just blathering.

jibal4• 5 months ago

This was nonsense post-hoc rationalization from the human who wrote it.

throwaway5465• 5 months ago

GPT told me to walk as there'd be no need to find parking at the car wash.

wouldbecouldbe• 5 months ago

I just tried claude, only Opus gave the correct answer. Haiku & Sonnet both told me to walk.

pickleRick2432• 5 months ago

I was surprised at your result for ChatGPT 5.2, so I ran it myself (through the chat interface). On extended thinking, it got it right. On standard thinking, it got it wrong.

I'm not sure what you mean by "high"- are you running it through cursor, codex or directly through API or something? Those are not ideal interfaces through which to ask a question like this.

summerdown2• 5 months ago

My first instinct was, I had underspecified the location of the car. The model seems to assume the car is already at the car wash from the wording.

Doesn't offering two options to the LLM, "walk," or "drive," imply that either can be chosen?

So, surely the implication of the question is that the car is where you are?

jason_oster9• 5 months ago

Doesn't offering two options to the LLM, "walk," or "drive," imply that either can be chosen?

Yes, but the problem is specifically that proposing two choices also eliminates other options. An open-ended question would lift that restriction. GPT-5.2 Thinking: https://chatgpt.com/share/6993d099-ef4c-8005-aa62-bdb826b707...

Other possible issues in the question: Is biking also an option? What do you want to do at the carwash when you get there, wash the car or buy a bucket, sponge, and soap? Is the car already at the carwash and you want to drive a second car? What about calling the carwash to see if they will have someone wash the car for you?

There are many ways to interpret the question because it contains ambiguities that must be resolved through assumptions. (Lacking information and constraints such as possible alternatives that satisfy the goal of washing the car.) The follow up questions I asked also have assumed answers but answering them provides no clear resolution to the ambiguity present in the original question.

So, no, I disagree that there is any solid implication of where the car is. And even if there is a solid implication, it can hardly be reasoned that it "isn't an XY problem" or that the question is clear cut in any real sense.

krzys• 5 months ago

Right, but unless you want to wash some other car, you have no car to drive there.

Spectrum or not, this is not a problem of weakly specified input, it’s a broken logic.

criemen• 5 months ago

I had assumed that reasoning models should easily be able to answer this correctly.

I thought so too, yet Opus 4.6 with extended thinking (on claude.ai) gives me > Walk. At 50 meters you'd spend more time parking and maneuvering at the car wash than the walk itself takes. Drive the car over only if the wash requires the car to be there (like a drive-through wash), then walk home and back to pick it up.

which is still pretty bad.

user_7832• 5 months ago

And on the flip side, even without thinking, Gemini 3 flash preview got it right, with the nuance of the possibility of getting supplies from the place (which tbh I as a human first thought this was about when I opened this thread on HN).

Since you are going to the car wash, the choice depends entirely on *how* you plan to wash the car:

### 1. Drive if: * *You are using a drive-through or self-service bay:* You obviously need the car there to wash it. * *You are dropping it off:* If you are leaving it for a professional detailing, you have to drive it there. * *The "50 meters" is on a busy road:* If you have to cross a major highway or there are no sidewalks, it’s safer to just drive the car over.

### 2. Walk if: * *You are just going to buy supplies:* If you have a bucket at home and just need to run over to buy soap or sponges to bring back to your driveway. * *You are checking the queue:* If you want to see if there is a long line before you commit to moving the car. * *You are meeting someone there:* If your car is already clean and you’re just meeting a friend who is washing theirs.

*The Verdict:* If you intend to get the car washed at that location, *drive.* Driving 50 meters is negligible for the engine, and it saves you a round trip of walking back to get the vehicle.

boobsbr• 5 months ago

I hate models trying to be funny, and being very verbose.

ChrisMarshallNY• 5 months ago

“My Tesla is low on gas, the gas station is a mile away. Should I risk driving there, or walk with a gas can?”

ChatGPT actually caught it. Maybe if I was fuzzier about the model…

raxxorraxor2• 5 months ago

Sonnet 4.5 after thinking/complaining that the question is completely off topic to the current coding session:

Walk! 50 meters is literally a one-minute walk. But wait... I assume you need to get your car to the car wash, right? Unless you're planning to carry buckets of soapy water back and forth, you'll probably need to drive the car there anyway! So the real question is: walk there to check if it's open/available, then walk back to get your car? Or just drive directly? I'd say just drive - the car needs to be there anyway, and you'll save yourself an extra trip. Plus, your freshly washed car can drive you the 50 meters back home in style! (Now, if we were talking about coding best practices for optimizing car wash route algorithms, that would be a different conversation... )

And yes, I like it that verbose even for programming tasks. But regardless of intelligence I think this topic is probably touched by "moral optimization training" which AIs currently are exposed to to not create a shitstorm due to any slightly controversial answer.

mcintyre19942• 5 months ago

Heh, is through Claude Code? I have a side project where I'm sometimes using Claude Code installs for chat, and it usually doesn't mind too much. But when I tested the Haiku model it would constantly complain things like "I appreciate the question, but I'm here to help you with coding" :)

tlogan5• 5 months ago

Gemini pro medium is failing this:

I want to wash my car. The car wash is 50 meters from here. Should I walk or drive? Keep in mind that I am a little overweight and sedentary.

But amazingly chatgpt is telling me to drive.

Anyway, this just shows how they just patched this because the tiktok video with this went viral. These systems are LLMs and all these logic steps are still just LLM steps.

anentropic• 5 months ago

Also the answers are non-deterministic

olalonde• 5 months ago

I think OpenAI is just heavily woke tuned. I had similar lack of reasoning ability when discussing subjects like gender dysphoria.

paulus_magnus22• 5 months ago

-- OK. Added location context for the vehicle

grok works, chatgpt still fails

[1] https://chatgpt.com/share/69932b20-3eb8-8003-9d9c-b4bba53033... [2] https://grok.com/share/bGVnYWN5LWNvcHk_f32dd53d-7b36-4fa2-b3...

swat5352• 5 months ago

Grok gets a lot of hate because of Musk, but it's a decent model.

I use it daily with my X account for basic tasks and think the free limits are generous. With X premium, you can get even more out of it.

Nothing beats Anthropic when it comes to coding however.

yuters• 5 months ago

GPT 5.2 when asked "How will the car get washed if I don't drive it there?" grasped the reasoning, and then immediately forgot it.

Fair point. Drive it there, obviously.

Revised recommendation: drive 50 meters, wash it, then reward yourself by walking home like an eco hero.

aleph_minus_one• 5 months ago

Revised recommendation: drive 50 meters, wash it, then reward yourself by walking home like an eco hero.

... which likely makes no sense: should you leave the car at the area of the car wash?

crazygringo2• 5 months ago

That's absolutely hilarious. It came so close.

kmac_• 5 months ago

Lol, this is so good. And quite interesting, as it might mean that there's no "world model" in LLM models, or they have such gaps.

baobabKoodaa• 5 months ago

This strikes me as the kind of logical error that you might get by lobo.. cough aligning the model to be more environmentally conscious. So now it tries to shoehorn some environmental talking points in there at the cost of losing its whole track of thought.

sarthakjain• 5 months ago

These days LLMs refuse to back track and use their intelligence to justify their answers.