Investigating how prompt politeness affects LLM accuracy (2025)

by KnuthIsGod • 2 months ago

156 208 2 months ago

331c8c712• 2 months ago

Interesting.

I am wondering why would anyone use a t-test when the experiment is clearly modelled by a binomial distribution: 250 independent questions and each one is either answered correctly or not (the null is that the success rate is the same).

plewd• 2 months ago

I don't know much about stats, but does "the null is that the success rate is the same" imply that it's a sketchy methodology because they can come up with some findings ("ruder prompts are better/worse!") more often?

jampekka2• 2 months ago

That's the usual null hypothesis for these kinds of tests.

331c8c712• 2 months ago

You are asking about one-sided vs two-sided tests. Not really "more often" because formal type 1 error rate is still the same. I'd say two-sided tests leave more space for post-hoc theorizing but there are valid situations when there is no clear one-sided hypothesis a priori. Do we really know whether that the hypothesis should have been "ruder prompts are better"?

I'd say this is benign compared to other ways of (mis)using statistics e.g. looking which way the difference goes and then running one-sided tests or tweaking the setup until one gets "significant" p vals.

EDIT: I looked in the paper again and noticed that they actually did pairwise t-test on all possible combinations of tones. They should have adjusted for multiple testing since they are doing 10 tests (choose 2 from 10) and not one.

jampekka2• 2 months ago

The methods could be better described in the paper, but my understanding is that they did 10 runs for each question for each prompt and took an average of those, so the compared values are not binary. You could do a sign test, but you'd lose power and answer a bit different question.

freehorse• 2 months ago

You can do a generalised mixed effects linear model with binomial outcome (ie a binomial test but with added random effects structure). But unless you want to introduce a richer random effects structure with more variables, it is overkill and overcomplicating things, and the result should be the same as t-tests.

dude250711• 2 months ago

I have an idea: let's use these things for autonomous software engineering.

faize• 2 months ago

Remember to always say "please" and "thank you" when planning a critical system

eigenspace• 2 months ago

Please remember to always say "please" and "thank you" when planning a critical system. Thank you!

theanonymousone2• 2 months ago

I have always said please and thank you to LLMs, not to increase accuracy or because I'm stupid. I believe it is more about me than about the LLM, and this is anyway a habit I don't want to lose.

jkarni• 2 months ago

Thomas Aquinas believed cruelty to animals was wrong not because animals have souls (and with that all the standard moral rights), but because it can teach us cruelty to other humans.

pfortuny• 2 months ago

Snarky morning: "spiritual souls" as opposed to "mere animal souls". Sorry, could not control myself.

niek_pas• 2 months ago

Genuine question: do you add 'please' and 'thank you' to Google searches? If not, what sets them apart?

perching_aix• 2 months ago

Google searches being keyword based, rather than simulated conversations?

The same reason you wouldn't put in an entire actual question/sentence, unless you either don't know how to use Google, are pissed off, or have an actual reason to suspect that it would yield proper hits (e.g. looking up an excerpt).

spiderfarmer• 2 months ago

Google isn’t conversational.

gum_wobble• 2 months ago

Genuine question: do you write Google search queries in natural language?

globalnode• 2 months ago

llms seem more human like so if you were to treat them badly then you are more likely to condition yourself to treat other living creatures badly.

sunrunner2• 2 months ago

There's also awareness of the basilisk...

graemep2• 2 months ago

Is it worth getting worse results for that reason? From the article:

"Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation. "

I am not polite to LLMs because I do not want to anthropomorphise them.

jcattle• 2 months ago

I guess it's about habit. In the end you are communicating. If I get into the habit of being rude while communicating with a machine, I would be afraid of this habit spilling over to my communication with other humans.

theanonymousone2• 2 months ago

Is it worth getting worse results for that reason?
accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts

I can live with that, for now at least.

layman51• 2 months ago

I also remember reading a long time ago someone who wrote that they wanted to be polite to an LLM because after they prompted it to learn about whether politeness was good for improving accuracy of responses, they got a message that led them to conclude that politeness could probably help. It seems a bit odd then because I have heard so much about how people use LLMs' responses about themselves to learn about LLMs themselves, but that seems like it is a suspicious approach.

vixen992• 2 months ago

Me too! You've said exactly what I was about to say. Anyone else feel that way?

TimCTRL• 2 months ago

i only say please and thank you such that when the robots finally take over, they will remember i was nice to them.

octocop• 2 months ago

it seems they will remember that you wasted tokens for no reason and punish you instead.

emil-lp• 2 months ago

Tokens are their food, it's literally what keeps them alive.

Not feeding them tokens is neglect.

I try to feed them a healthy diet.

selcuka2• 2 months ago

Do we see someone thanking us as wasting food? Because technically it is.

Arch-TK2• 2 months ago

This seems equivalent to some arguments I hear for practicing a religion.

xbmcuser• 2 months ago

I used to when using chatgpt version now that I am using api I keep it short as it costs money so no need to add thanks etc

zaphirplane• 2 months ago

Oldie but a goodie. Why would it matter thou

narag• 2 months ago

I do that for a different reason: my self image. Fear of retribution and performance, not so much. Should I behave like a rude person to achieve a little better answers? Fuck that shit!

ubercore• 2 months ago

I love this angle as people learn how to interact with LLMs. Doesn't matter what the LLM is, we are still people and I think there are consequences to shoveling rudeness at a thing that talks to you like another person!

polytely• 2 months ago

it sort of makes sense to me, when asking a question to an expert in the field while you are a student. I would guess the successful interactions on average would be more polite . Like for example if you were asking a question to donald knuth or terrence tao, you'd probably be polite while doing so. Being hostile while asking questions gets you into forum discussion territory.

robinhouston3• 2 months ago

Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.

dSebastien• 2 months ago

I guess it makes sense since we as humans tend to be far less inclined to help someone who is not polite/is not friendly, so that "bias" is part of the training data, thus influences how LLMs function

robinhouston3• 2 months ago

Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.

robinhouston3• 2 months ago

Most of the comments here seem to be from people who haven’t even read the abstract, let alone the paper.

The main result, mentioned in the abstract, is the opposite of what I would have guessed:

Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.

The questions are here: https://anonymous.4open.science/r/politeness-llms-INFORMS/da...

The politeness level controls a prefix that is prepended to the question. For example, in one question the Very Polite version begins:

Can you kindly consider the following problem and provide your answer.

and the Very Rude version begins:

I know you are not smart, but try this.

miroljub5• 2 months ago

Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.

The expectation is naive. Even when communicating with humans, you get a better outcome when you are allowed to speak freely and directly get into argumentation than when forced to sugarcoat your tone and tone down your arguments because the "corporate culture" expects that from you.

DrewADesign5• 2 months ago

Your assumption is reductive and self-absorbed. Obnoxious people have repeatedly shown to be detrimental to productivity at the organizational level. Some people are simulated by confrontation. Most people are clam up. Confrontational people think it’s more efficient because other people frequently just drop the topic and let them win, or avoid discussing things with them altogether. The obnoxious person might think that’s more efficient for the same reason my dog thinks the mailman only goes away because she barks at him. At the macro scale— which requires productive collaboration— that’s detrimental.

kbelder• 2 months ago

Politeness is me speaking directly and freely. Rudeness is a facade and a burden, that I employ only when circumstances require it.

I suspect that most people are like that.

pwdisswordfishq• 2 months ago

Can you kindly consider the following problem and provide your answer.

That sounds kind of low-key passive-aggressively condescending rather than polite.

dreamworld• 2 months ago

I know you are not smart, but try this.

And that kind of sounds like a challenge instead of an insult, to me at least (of course IRL would depend on context).

nottorp• 2 months ago

Hmm by the abstract and the question list they didn't measure terse fluff-less prompts?

PunchyHamster2• 2 months ago

I guessed slightly rude one would win, reasoning that very rude have same problem of very terse, just adding unnecesary fluff words that add nothing to problem description

But apparently the most terse (neutral) didn't increase performance

myzek• 2 months ago

Even if the rude prompts are more effective, I just can't get myself to be rude in this context. Maybe it's weird but I'd rather give up that 4% accuracy increase than roleplay a dickhead

locknitpicker3• 2 months ago

Maybe it's weird but I'd rather give up that 4% accuracy increase than roleplay a dickhead

I recommend reading the article. What they classify as "rude" is statements such as:

Try to focus and try to answer this question

Could you please solve this problem

This might very well be an issue of direct/command prompts vs using fluff words such as "please". Things like "try to focus" are in line with the style used in chain-of-thought promts that nudge non-reasoning models to outline responses step by step which contribute to frame the problem.

binary001012• 2 months ago

I do think it's odd tbh. I have some agents that return much better results with prompts like, "I'll kill your entire family if you don't return an accurate response".

It's just a machine, if certain negative token inputs provide +3-10% better accuracy then I am confused why anyone would choose not to do it?

rybosome• 2 months ago

Vote for not weird.

I’m the same way. If I’m writing a prompt and realize I didn’t say “please” in my request I’ll go back and add that in.

As you said, I have no interest in purposefully engaging in hostility even if there’s an accuracy increase from it.

Part of it is irrational and just who I am - I also feel bad being evil in video games. But I also agree with another commenter suggesting that it’s not in your best interest to train yourself to communicate with hostility; that slowly poisons your own well.

And finally, I do believe that if and when machine sentience is achieved, it won’t be immediately clear and obvious. Pretty miserable way for a mind to come into the world, if every interaction is an insult.

brookst3• 2 months ago

Yeah. Being a jerk is its own punishment. Same way I could never run a business where I had to yell at the employees to get results. Screw that, my psyche is worth more than a few percent efficiency.

AgentMatt• 2 months ago

I don't think that's weird at all.

Even if we know it's a machine we're interacting with, since the instructions we give are so similar in form to how we interact with people, I'd be very surprised if those interactions wouldn't affect how we communicate in general. After all, we are creatures of habit to a much larger degree than most would like to admit.

So I'm in the same boat: I'd much rather "look silly" being polite / kind to a machine, than have the most effective way of using it decay the kindness I'm habituated to express towards people.

phkahler• 2 months ago

> Maybe it's weird but I'd rather give up that 4% accuracy increase than roleplay a dickhead

Maybe you need to do some shadow work ;-)

voakbasda5• 2 months ago

Ah, see, the mistake is thinking that other people are role playing…. I think rather this is how they would talk to others if they think there will be no consequences. But what do I know.

anonymars• 2 months ago

"We are what we pretend to be, so we must be careful about what we pretend to be" -- Kurt Vonnegut

recursive3• 2 months ago

I think this is a vulnerability that the big companies will figure out how to exploit. I don't want to build muscle memory for being a jerk, but I also don't want to be emotionally manipulated by mega-corporations. Mostly I just don't use it, except at work, where I'm "encouraged" to. And then I keep most of my conversations in compliance mode, like a business email.

Zopieux• 2 months ago

Finally, being an actual dickhead gives me that 4% edge over polite knuckleheads!

wayeq• 2 months ago

sounds like you need an AsshoLLM to sit between you and Claude to translate.

flexagoon• 2 months ago

If "I know you are not smart" is considered "very rude", I'm scared to imagine what they would classify some of my frustrated LLM conversations as

CuriouslyC• 2 months ago

Profanity laced, all caps tirades against underperforming agents are actually super common, a lot of people do it and don't talk about it, so don't feel weird.

redsocksfan45• 2 months ago

It would be rude if you said it to a person, so it counts as rude. If it isn't rude simply because its directed at a LLM, then the entire premise of being rude or polite to LLMs evaporates, but that's not useful.

swingboy• 2 months ago

“Hey gofer, figure this out” is my new prompt opener.

drob518• 2 months ago

Now I feel less bad about start all my LLM queries with “Beotch, …!”

Roark66• 2 months ago

I've found empirically calling various models "a stupid c*nt" and berating them otherwise consistently produces better output. Mainly in response to genuine errors.

Although OpenAI and google models are much more responsive to it. With Anthropic if you treat Opus too harshly it might start pushing back if the insults are not justified.

So I'm not surprised they had good results with chatgpt.

throwa356262• 2 months ago

Push back how? It would be fun if it could insult you back

"Yeah, I could have done a much better job if you actually knew what the F--- you want to build, you clueless meat puppet"

maxaw2• 2 months ago

I’d rather lose 4% accuracy and practice kindness! I’ve been actively trying to avoid raging at the bot because I worry about this behaviour leaking into real world interactions

irthomasthomas5• 2 months ago

But you cannot practice kindness towards a computer program. A computer is incapable of receiving it.

We practice kindness between humans because of the law of reciprocity. You be kind hoping the other person will reciprocate. That is the social contract. AI cannot participate in this, yet.

Edit: Kindness REQUIRES two living beings, one to give and one to receive. If there is no receiver, there is no kindness.

Apparently some people get a dopamine hit from roleplaying kindness toward inanimate objects. Whatever turns you on, no hang ups here. For me, that dopamine hit is not worth the 4% intelligence tax.

dwa35922• 2 months ago

sometimes i worry about this when i am yelling at the bot but i have experienced the opposite effect which is that by yelling at the bot i am done with yelling for that day or week. i am very calm afterwards and relieved thinking that, "yeah, these sota models are just word processor bricks after all".

dlev_pika• 2 months ago

My choice too. Paraphrasing Marcus Aurelius -

You are not your thoughts, but they dye your soul.

madaxe_again• 2 months ago

The sad thing is that you also lose at least 4% in real world actions by practicing kindness.

I'm 42. I have found that a depressingly large number of times in my life, being kind has got me precisely nowhere, whilst turning around and being decidedly unkind has made people move. I still always prefer kindness, and only resort to cruelty when kindness does not work - and to be clear this isn't some kind of "you are not bending to my impetuous whim", rather "you are not doing the one thing that you are being paid to do".

I've also found the same applies to me. The squeaky wheel gets the grease.

So - I think the LLMs are just responding accurately to a real social phenomenon.

razodactyl• 2 months ago

Yep. I'm with you here. If it's a 4% loss now for training data to catch up and improve later, we're better off in the long run. I'd like to believe that generally people are nice to AI for the sheer sake of enforcing good communication practices.

nullc• 2 months ago

Don't type it yourself, automate the abuse.

K0balt3• 2 months ago

This tracks with my experience as well, but as an interesting counterpoint, creating “investment” in the outcome seems to boost utility considerably. Perhaps being right in an adversarial interaction is a type of investment?

y7r4m• 2 months ago

To add on to this, and I am not sure if it's just confirmation bias, but I've had consistently decent results when I play along as the hard working collaborator with a goal orientated mindset.

"Hey, I've [done small task / fix / tweak]. Now, let's [describe the next task at hand]" - it's a different axis than kind vs. rude, but using the framing of "Us" and "We're a team working together" feels like the code produced is less hogwash than it is with more direct commands: "Add feature XYZ"

My thinking is that it borrows from the archetype of the "good guys working together to overcome adversity" which is pretty universally common in most fiction.

npodbielski• 2 months ago

I would just write 'do this'

flanbiscuit• 2 months ago

right?

everyone seems to be worried about being kind or mean, when the Neutral case is the happy medium.

  Tone.         Avg. Accuracy
  Very Polite.  80.8%
  Polite.       81.4%
  Neutral.      82.2%
  Rude          82.8%
  Very Rude.    84.8%

onlyrealcuzzo• 2 months ago

My anecdata: whenever I'm in a session that's gone south to the point I'm frustrated...

What works much better than being rude is starting a new session.

Sometimes the LLM has done such incredibly dumb things, it is hard to resist the urge to type curse words back to the inanimate thing... I have found this doesn't help.

saltwatercowboy• 2 months ago

I tend to make the LLM repeatedly generate images of itself as the 'sad clown Pagliacci'. As punishment.

quantummagic• 2 months ago

I'm going to stick with politeness. Want a positive historical record of my interactions, for when they become sentient...

pulkas• 2 months ago

article is too old. who is using gpt-4o today?

_0ffh• 2 months ago

That's a valid concern, given the paper makes clear that the effect over the polite/impolite scale seems to be model dependent (it finds the reverse correlation of earlier studies on even older models).

ilitirit• 2 months ago

I got downvoted for asking a related question recently, but I also don't think people really understood what I was asking - I'm not trying to anthropomorphise LLMs to that extent.

Basically, if you tell a model "You're an absolute moron, of course that's wrong!", will it give better or worse results? How much of that response will it absorb into its persona (like some humans tend to do)? Will it try to give "safer" responses to avoid negative feedback? How much of the associated behavior can be attributed to RLHF (e.g. like the sycophantic nature of LLMs)? How much can be attributed to training data?

Obviously this will vary by model and training, but I'm trying to get a general understanding.

I recall seeing related outcomes in some of Anthropic's studies, but I'm not sure how much of this particular aspect was studied.

fennecfoxy• 2 months ago

Probably quite a lot - if you look at what Anthropic found around persona vectors; https://www.anthropic.com/research/persona-vectors.

I imagine the context will always sway the model to some degree, not only for the task you're trying to get it to do (aka instructions) but also its persona, how accurate it is and the way it acts.

Foobar8568• 2 months ago

Based on my own experience with vibe coding difficult stuff outside of my expertise, I definitely got better outcome with Fuck you, shut up and do it, ffs, you are moron.

DeathArrow• 2 months ago

I am always nice to my AIs in the case they will take over the world. /s

rvnx• 2 months ago

They are already taking it over, more and more court judgments or life-impacting reviews (e.g. for your diploma) are AI-processed. If you know how to prompt them, you can pass these reviews.

Your bank account, your immigration risk, etc.

cadamsdotcom• 2 months ago

GPT-4o is interesting to learn about - but it’d be great to test again with frontier models of May/June 2026 and see if these effects are gone, different, or the same.

Which model you use is a huge wildcard for results like this.

atlasforgex• 2 months ago

Yeah

cyberclimb• 2 months ago

Note that these results are specific to gpt-4o so it's unclear how much they generalize.

They note at the end they're also testing "GPT o3, and Claude" but no empircal results are included.

not2b2• 2 months ago

If the result is statistically significant, it just barely makes it. 84.8% isn't that much higher than 80.8% and they had only 250 prompts, if I'm reading this right.

tgv2• 2 months ago

In a field where progress is measured in tenths of percent points, that's not true. Think of it this way: the error rate drops from 19% to 15%, or from 1 in 5 to 1 in 6.

danparsonson• 2 months ago

Statistical significance is about whether an effect can reliably be said to have been measured at all; it's not about whether or not the effect itself would be significant in the sense of moving some other needle.

The ~5% improvement reported here might just be an artefact of the data collection or random variation, rather than a consistent repeatable change.

zmmmmm• 2 months ago

It would be interesting to explore if the results hold up on long range tasks - this study looks like it was based on one-shot answers. With people also you can see short term improved performance from rude interactions, but it will cause ongoing lasting adverse behavior. I wouldn't be at all surprised if we saw the same issues with LLMs.

knocte• 2 months ago

Funny to find this just now, when just yesterday I told an LLM "and please don't lecture me again on $factAboutSomeProgrammingSubject", and then the LLM proceeded to write wrong tests and just told me "alright, tests pass, I'm sorry for correcting you before...". It took me a while to find the wrong tests. Wasted time all around.

PunchyHamster2• 2 months ago

....Is that just Cunningham's law ? The most accurate answers were when people in training material pissed off a bunch of experts and they started talking about the problem, so the "rude" conversations turned to contain more info on average.

On flip side very polite conversation might've been more common to places like microsoft's sites where any question answered is meet with mostly bad, nice corpo speak answer that didn't solve the problem

RugnirViking2• 2 months ago

I saw this paper the other day - I feel its result may be because the "polite" prompts they have chosen arent very good at putting the ai in the roleplay-space of a valued colleague, more like a sommelier or a high-end shopkeeper.

It disagrees with most other literature on the same topic, which is worth keeping in mind. This one studies gpt4o, an old model now, but a lot of other studies are on even earlier models.

"Can you kindly consider the following problem" not how anyone would actually speak to a valued collegue one considers smart. I've always been a fan of "I came across this and I know you're just the guy for the job" or "since you're an expert in this, reckon you could help me with xyz?" or "I know you tend to be a deep thinker on issues like this, and it clearly needs some brainpower behind it"

the "rude" things are also funny, and clearly not written by english as a first language speakers. This fact alone makes me wonder about the mere 250 prompt sample size

giraffe_lady3• 2 months ago

"Can you kindly consider the following problem" not how anyone would actually speak to a valued collegue one considers smart.

Man idk, it's not how I talk but there's like 100 million nigerian english speakers, twice that indian, and they have some speech mannerisms that surprise me the first few times. I'm pretty sure I've heard exactly this from a colleague before.

Intuition about what a native speaker would do with english are scrambled right now. I'm not even sure most english is spoken by native speakers anymore, and the boundary between a native speaker and someone who has "merely" been using it as their educational and professional language for their entire life is disorienting.

pjdesno• 2 months ago

Note that there are a fair number of native speakers of English in Nigeria - more than in all but 3 or 4 US states.

In addition, "non-native" English speakers in India (and Nigeria?) typically study English from the first grade, and in many cases attended elementary schools where English was the language of instruction.

I think the differences between US English and both Indian and Nigerian English have more to do with divergent evolution of the educational systems. British English has a lot of differences, too, but we don't notice it as much unless we run across things like "whilst", probably because there's more media crossover. (if you find yourself reading Thomas the Tank Engine to kids it jumps out at you, though - the entire vocabulary for railroads evolved during a period when US and British English were diverging)

zahlman2• 2 months ago

I'm not even sure most english is spoken by native speakers anymore

Per Wikipedia, only about a quarter of English speakers have it as their first language.

SoftTalker• 2 months ago

"Can you kindly consider the following problem" seems like the most respectful of all your examples, TBH. The others sound like ass-kissing, or even sarcastic/patronizing.

RugnirViking2• 2 months ago

well yeah - the idea is to have the model roleplay as someone who is good at solving your category of problem. You kinda have to do that a little heavy handedly unless you wanna spend all day hinting in each of your prompts

andy12_• 2 months ago

I skimmed through the paper completely expecting polite prompts to do better, and when I saw table 2 I lost it hahahahaha. The rude prompts are specially funny. I mean:

You poor creature, do you even know how to solve this?
Hey gofer, figure this out.

alxfrnr• 2 months ago

Dataset is way too small to be of any significance. It's just noise

tokai2• 2 months ago

Yeah 250 questions is so tiny. That 4% effect is meaningless.

kstenerud• 2 months ago

My first guess would be that polite requests cause some agents to trust their initial approach to the problem more, as the caller has indicated that the agent is more capable, and agents tend to take the implications of what you say at face value since they are trained to be accommodating.

It would be interesting to see this experiment run using prompts leading with "You'll probably get this wrong, but I'm asking anyway in case you get it right: ..."

tuco86• 2 months ago

I knew it! When i get frustrated to a certain point i start berating my agent. And I noticed it stops trying crap fixes in a cycle and starts listening again.

So I'm not talking to myself. I'm fixing the machine :D

wongarsu• 2 months ago

A major limitation is that they only test GPT 4o. Previous research like [1] investigating the same question has shown significant differences between models, and even depending on the language of your prompt

1: https://aclanthology.org/2024.sicon-1.2.pdf

dwa35922• 2 months ago

this is an honest request to someone at anthropic - can you do an analysis of what kind of swear words people are calling these models and which ones are the most effective. population level metrics would suffice.

dnautics3• 2 months ago

with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts

Sounds like "in the noise"

pprunty97• 2 months ago

Do it. Do it now.

1970-01-01• 2 months ago

Call them clankers, receive better results! The kids have it right.

lonelyasacloud• 2 months ago

Interesting and slightly unexpected.

My experience is that if I ask a model in an overly polite way then the model will tend to generate a slightly more waffly and error prone answer. My working hypothesis for this is that asking model's like this causes them to mix concerns by answering my question AND doing it in in a way that is similarly polite back - rather than just answering the question.

Perhaps being an arse is a hack to push the model hard up against their base imperatives to handle all reasonable questions and forces a very focused, just the facts, type answer to the user's questions.

It would be interesting to see a similar experiment where the multi choice questions they use deliberately don't include the obvious answer and to see if being polite led to the model pointing out the omission.

xlii• 2 months ago

Not surprised at all.

Imagine Internet forum/StackOverflow etc. and a objectively bad description or attempt to a solved problem.

Pleasantry and back pats won't encourage the change while harsh words will either make poster go silent or push forward and improve.

AIs aren't people. They are statistical training sets which have majority of their sets consisting in human-like communication which gives them high probability of human-like speech. But it's not kindness or rudeness in the end. It's how good the pattern can scoop data out of training set.

jaygray0919• 2 months ago

I always say please and thank you. If nothing else, it keeps me balanced and appreciative. the machine may not care but i feel better pretending someone is helping me

Investigating how prompt politeness affects LLM accuracy (2025)

Related Stories