The XY Problem (2014)

by myle • 7 years ago

319 158 7 years ago

The question, "what are you trying to do?" is super useful to remember to ask. I work at a company that's trying to modernize and move to devops for dev environments. Because the infrastructure is not nailed down, this means that every single team needs to have at least one person who understands the devops.

The process of learning k8s and grokking the security concerns in the course of a few sprints is not pleasant. This "XY Problem" crops up all the time, but the devops team is absolutely slammed all the time so many times basic queries go unanswered. Everybody has to help everybody else.

I don't think anyone wants to go back to the days of rigid development infrastructure so we're kind of stuck with devops until the ecosystem can mature.

My great hope is for serverless to take off and so programmers can further specialize. I'm getting tired of learning new languages and stacks every few months or years. Just hand me a biz requirement and let me implement it any way I like.

jacknews• 7 years ago

OTOH I find it quite annoying, when asking about Y, to be bombarded with questions like 'why would you even do that' etc.

Eg,

Me: "What's the best trackball in 2019?"

Crowd: "Do you have RSI? You probably need a vertical mouse."

etc

zamadatix2• 7 years ago

"The XY problem problem" has probably caused me more wasted time on internet searches than dealing with "The XY problem" in real life.

pezo19192• 7 years ago

Yes same here. I easily get annoyed because it really is a rabbit hole.

Why do I want anything? Because I want to be happy and successful, lol.

Is it broad enough now?

tobyhinloopen• 7 years ago

Why do you want to be happy and successful? Maybe you just want some dopamine injections instead

jjoonathan• 7 years ago

That feeling when you spend hours reducing a problem to its simplest form and then the experts pile in to your IRC / mailing list / ticket system / stack overflow post to explain that you must be suffering from the XY problem because your ultimate goal couldn't possibly be to do what the minimized program does.

Naturally, they didn't read the part at the top where you explain your ultimate goal.

Two days later you make a new post favoring contextual detail over brevity and the same experts pile in to complain that you didn't minimize your example.

Doubl• 7 years ago

Why are you using vb6?

agumonkey3• 7 years ago

I just had to left a job after 2 weeks, because as someone new to the clan, everything I said was noise to them. I guess there's a time-to-respect period to go through.

maxxxxx2• 7 years ago

I don’t know about this particular situation but I think it’s just a matter of politeness that somebody new should use his first weeks to try to understand the current situation before making too many suggestions. Also show your skills by delivering something.

I have seen quite a few contractors who didn’t seem to be able to do anything if the environment was not exactly their preferred environment. If you are good you can deliver something in any environment and build up credibility.

kps2• 7 years ago

"What's the best trackball in 2019?"

Kensington Slimblade, still.

dmbaturin• 7 years ago

I'm a bit confused by its pictures. It has a scroll wheel, right? How's the build-in wrist pad? I'm still using a Kensington Expert Mouse.

Anyone tried those vertical mice? To me it looks like a pointing device for people with RSI rather than something one may choose for better usability whether they have RSI or not. I was introduced to trackballs by a friend who has RSI, and I loved them, even though I don't have any wrist issues.

MrStonedOne• 7 years ago

OTOH, wanting to do y for other reasons, finding an xy SO post on Google, and not being able to get the answer because it went off topic about rather or not y is the solution to X is frustrating.

Bonus points, ask y again because the other question answered z only and get it closed as a dupe of the other question.

enriquto• 7 years ago

I have exactly the opposite problem. I ask, very honestly, for X, and then people incorrectly assume that what I really want is Y.

Just give me X or shut up!

panpanna• 7 years ago

Seconded.

The problem is the guy who neither understands X or Y but just has to answer your question.

No mister, I don't care about Z. And I don't have the time to explain the problem in simple words to you. Either give me answer on X or stay out of this thread.

xtiansimon• 7 years ago

On SO I have to resist the urge to comment on those types of answers. A good wrong-answer makes the original-post & correct-answer couple better.

Interestingly, that’s not the case with single-threaded forum ‘discussions’, where the y-answer discussion torches your post.

The Q&A-type really shines in this respect, vs the discussion-forum.

Just have to say, XY happens on Arch Linux forum a lot. High level of knowledge of many users, versus lots of peeps interested to try Arch who know enough to install and generally maintain a Linux system.

maxxxxx2• 7 years ago

Yeah. I hate situations where you have thought about something more than anybody else but you still have smartasses start questioning you after being exposed to the problem for a few seconds.

I am fine with people asking questions but they first should have built up some credibility.

nullc4• 7 years ago

Just give me X or shut up!

If that's your attitude, why don't you expect others to think "Just find your own darn answers or shut up!"? :)

Can_Not• 7 years ago

Real conversation I've had:

"Does Django's query builder use prepared statements by default?"

"What are you really trying to do?"

"I'm trying to find out if Django's query builder uses prepared statements by default..."

Not everything is an XY problem. If somebody found that question on SO via Google, they would be pretty annoyed to see it XY'd instead of answered.

oconnor663• 7 years ago

Yeah as a courtesy, if X is easy to answer, it's better to answer it and then ask about Y.

agumonkey3• 7 years ago

That's probably a highly efficient way to get booted of most IRC channels.

raverbashing• 7 years ago

What happens sometimes is that X is supposed to be simple but it doesn't work or it doesn't work for that specific case, so you're trying to work around it by doing Y (or Z, etc.)

emilfihlman5• 7 years ago

You are in the small, small, minority. It's extremely bad to not take the XY-problem seriously because you think it doesn't apply to you.

davidivadavid2• 7 years ago

That's definitely a case where asking that question wasn't warranted. You asked a question about a factual thing. It's essentially a yes/no answer, not a "how to" answer, which are way more prone to the XY problem.

an_d_rew• 7 years ago

I often have the problem that I want exactly X and thousands of Internet know-it-alls tell me that I want Y.

Sometimes they are right. Often they are wrong.

Stop making assumptions, people!

mannykannot• 7 years ago

These are mirror-image manifestations of the same underlying problem. In the example given in the article, the person asking mistakenly thinks X and Y are effectively equivalent. In your case, the person replying makes the mistake.

XY problems are just the tip of the iceberg; many problems are the result of mistaken assumptions of equivalence, and they often go unstated.

derefr4• 7 years ago

The most amusing variant of the XY problem is when you want a seemingly ridiculous Y, and then, when people inevitably ask why, you tell them about your entirely-novel X, which they still don’t understand the use for, so you have to explain the entirely-novel W that motivates X, and so on... and by the time you’re done, you’ve explained your whole business model, written a ten-page journal paper, and completely killed the conversation—because they don’t feel they have the prerequisite knowledge to talk about A-through-X, so you just get a “good luck with that, buddy.”

Even though Y was a concrete problem that they do have domain experience in solving!

I call this “going up the rabbit hole.”

——

Example (deliberately shortened from its real depth):

• I’m trying to invent a novel algorithm for inter-procedural data-flow analysis, because none of the existing ones work for me.

• Why? Because I’m trying to reverse a stack machine bytecode with only indirect jumps and no reified CALL/RET opcodes, into a structured inter-procedural control-flow-graph.

• Why? Because I want to do program-slicing to recover HLL representations of key-value-store writes that the program does.

• Why? Because I want to use them as hints for my runtime execution tracer for said bytecode, so it can emit better traces.

• Why? Because I want to put those traces into an OLAP database and I want the data in the DB to be typed.

• Why? Because these traces are the only canonical representation of the state of the system, and our business model is allowing auditors to figure out what’s going on with the system.

0xffff2• 7 years ago

This is why on the rare occasion that I find myself answering questions on IRC or Stack Overflow I always start with at least a simple answer to the question asked. Only once I've taken a stab at answering the question do I consider moving on to "but, what are you really doing..."

Another related type of question is a question born out of curiosity rather than need. Sometimes you want to do things just for fun to see how they work, but you get shut down every time you ask a question because "you should never do that in production".

winstonewert2• 7 years ago

hmm... so I do this in the opposite order...

I start with, "hey, you probably shouldn't do this, but assuming for the moment that you really want to for some strange reason..."

anfilt• 7 years ago

I hate when people say you never should that in production. A lot times when I am doing something weird it's for fun.

dahart• 7 years ago

I call this “going up the rabbit hole.”

Love it, that’s great! Adding it to my lexicon.

StavrosK2• 7 years ago

I hate that, I ask for help with an odd Y and say "I've researched all other solutions and they didn't work, so can someone please help with Y?" but people still make me show them my work and convince them that everything else actually won't work before they maybe help.

kuroguro• 7 years ago

I've found that if I need something oddly specific most people wouldn't be able to help anyway (and those who could would have to invest a lot of time in that specific case to understand it).

I've simply stopped asking for help with those. IMO there's some point after which it's counterproductive for both parties.

infogulch• 7 years ago

In my experience this can happen for two distinct reasons (not exclusively):

1. The StackOverflow Reflex, where the person responding doesn't believe you've actually explored other solutions sufficiently because in their experience the vast majority of the time the person asking for help hasn't actually explored other solutions effectively. This may feel like bad faith (from either side), but it could just be the asker's inexperience with other solutions that cause them to not apply them correctly.

2. They don't know how to do Y, but they know how to do X really well so they want to make sure you haven't missed anything in your attempt to make X work. Remember that different people have different expertise and may want to try to help solve your problem even if they don't know how to solve it with your chosen solution.

I think askers often don't recognize that explaining why T-X won't work in their case is actually very helpful to fully understand the problem, even if the person helping ends up solving Y directly. If you're asking for help solving a complex problem, sometimes you have to teach the problem to the person helping you. And that's ok.

sosborn• 7 years ago

I've researched all other solutions and they didn't work,

I ran into this the other day. A person was asking about a Y and when pressed about it that person said the other solutions didn't work. I dug back into that person's history and while they did try a canonical solution - the lacked the ability to debug their efforts.

For context, the Y they wanted to do was essentially to rewrite a core framework in Rails because they didn't understand that an object they were using was nil. Their problem was entirely about how they were getting input from a user form and they wanted to rewrite a framework.

When I pointed it out the person never responded, and I saw a similar question posted by that same person a couple of days later.

iovrthoughtthis2• 7 years ago

This is like going:

Hi, i need $10.

Why? Because I need to buy a beer.

Why? Because the man at the bike shop is super thirsty.

Why is that important? Because If I sate his thirst he said he’d introduce me to his manager.

Why is that important? Because I want to work at the bike store and it’s the only way to get an intro.

“The traces are the only canonical representation of the state of the system”

Why?

whatshisface• 7 years ago

What's the basis for this extreme skepticism, is it an accepted law that every problem in programing can be solved with some basic plumbing? My guess for why so many people love the X->Y thing is that it allows them to join in on discussions where they have no idea how to answer the question.

derefr4• 7 years ago

I mean, past that point, you've left "technical problem" land and entered "a description of what the business sees as an exploitable market opportunity."

But to be pithy, it's because the system is all of:

• intentionally brain-damaged for OLAP, by a design focusing on OLTP efficiency to the exclusion of all else (think: bit-packing structs into k-v values in a way opaque to anything other than the program that owns the values; no binary object-wrapper format like ELF to describe which ISA is in use by a bytecode program; etc.)

• very popular (enough that auditors are interested)

• a competitive mutually-untrustworthy multi-tenant platform, where people have every incentive to obfuscate any sort of event logs their programs emit, so that they can keep the competitive advantage of analyzing those logs to themselves

Or, in short: the system is a scrambled egg, and people are willing to pay to have it unscrambled for them. Most developers would just say "you can't unscramble an egg" and give up. But the schema information is there to recover—it's just burned into the logic of bytecode programs. Someone with the right expertise can recover it.

jfindley2• 7 years ago

Sure, there are cases where seemingly-ridiculous things are actually needed.

Where a lot of people commenting on this article are failing, though is thinking only about their needs and what they want. The people responding to your IRC message/ML post/etc are (usually) giving up their valuable time to help others for free, and making some accommodation for this is essential if you want to get anywhere useful.

On top of this, when you join a room/list where you aren't well known, the responders have no context on who you are, your level of experience, etc[0] - but what they DO have is long, long experience with people asking XY questions where Y is both ridiculous and not going to solve X anyway.

I know it's frustrating to have to go through your reasons when you are certain that damn_silly_thing_Y is actually the only way of solving your problem, but it is needed. After all, you're asking these people because they're likely more experienced than you are (or else why ask them?) and thus there's a chance you could be wrong.

0: And you can't often shortcut this - due to astronomically high rates of Dunning-Kruger cognitive bias in many of these venues, starting out with "I'm an expert, I know what I'm doing" without very convincing evidence is usually extremely counter-productive.

derefr4• 7 years ago

I think you confused my point for the point of the OP, and then just typed a reply that is basically a reply to the OP.

My point was the fact that, at the end of the justification chain, you've generally left the domain of expertise of the expert/forum of experts that you came to. Because of this, the people you're conversing with will fail to actually help you with your problem, because they've been dazed/confused/distracted by the higher-level problem statements that exist outside their domain of expertise. Even though the immediate problem is one they're eminently qualified to solve.

It's unintentional nerd-sniping: they force you to tell them about a larger problem that they don't know how to solve—sometimes, in my case, because it's a research problem and nobody knows how to solve it—and they get distracted trying to understand the entire breadth of a topic they may not have ever considered before, in order to figure out how they would solve the highest-level problem themselves. They never "come down to earth" to solve your immediate problem.

It's as if you went to a low-ranking soldier and asked something about how well some body-armor they use works in practice, and they asked you what the context is, eventually demanding you to explain the whole strategy of the war to them, at which point they get confused trying to understand the strategy and politics of the war they're fighting, which is usually inevitable because they're a soldier, not a politician.

catern• 7 years ago

I've just started immediately telling them my high level project if they ask why I am doing something. Even though I really doubt it will aid understanding and fixing my problem, since my high level projects are highly obscure research projects, it at least saves time. And then they'll either go away, or give a solution to my original problem; if they start questioning my research project I can just ignore them, though that has not actually happened yet.

Waterluvian2• 7 years ago

I get this sometimes when I'm trying to learn about a suboptimal solution. I can even preface the question with, "I know this isn't the best solution but can you explain why it works..." Inevitably I get, "you shouldn't do it that way..."

LanceH• 7 years ago

Welcome to stack overflow.

floren• 7 years ago

I call this “going up the rabbit hole.”

I just call it "posting on Stack Overflow"

jonahx• 7 years ago

I call it "the IRC smugness pit".

The asker knows that providing the full A-X context would be an utter waste of time, and really does want an answer to Y only.

Certain IRC regs will flat out refuse to answer Y. Often this is because they don't know the answer to Y, but instead of saying so would prefer to wear the mantle of wise interrogator of "things you did not think of"™.

Ofc, sometimes they're talking to a newb and this is appropriate, but jumping to "this is an XY problem" seems to be a knee-jerk reaction for some people.

jacobolus• 7 years ago

One of the more effective albeit mildly dishonest to get around this on IRC is to troll the channel.

“I asked my buddy who claims to be a language C expert and he told me that it is completely impossible to accomplish Y using language C, because «invalid reason Q that makes language C sound bad».

Then all of the smug jerks who would usually start by asking what kind of idiot would ever want to do Y will instead fall over themselves to prove that Y is possible and really quite simple to accomplish, and explaining that my expert friend who said it was impossible had no idea what he was talking about.

Make sure to use a pseudonym for this one though if you want to come back to the channel and not have people still mad at you.

Rerarom• 7 years ago

I agree with the rest of the commenters here that the XY problem problem is a way bigger one. Example:

"How do you implement a new system call in the Linux kernel?"

"Ah, well, a system call is overkill and leads to compatibility issues, you should try to see whether you cannot solve your problem some other way..."

"My problem is I want to understand how system calls are implemented thus I want to go through all the steps of making a new one so that I don't miss something"

Or, a more recent one from Reddit:

"When will we have desktop computers capable of replicating the feat of Google with AlphaGo?"

"Ah, well, we already have distributed computing trained NNs, also there is some research that it may not require such power..."

"I'm not interested in Go, I'm interested in the power of future computers!"

Seriously, people second-guessing you are annoying.

leetcrew2• 7 years ago

try asking "how can I automate ssh authentication by password?" sometime. I've still never found an answer other than "never do that, here's how to use ssh-agent."

detaro2• 7 years ago

There's actually a pretty good Stackoverflow answer to this one: https://stackoverflow.com/a/43526842 (Which I think is a good pattern: sure, mention the "you probably should consider Y", but show the alternatives as well)

rocqua2• 7 years ago

Honestly, I hope am answer to that question is never posted on stack overflow.

As for an answer, i think something like

   cat passwd.txt - | ssh

Might work. Didn't test it though. The cat outputs all arguments in order, and - is stdin.

peteradio• 7 years ago

pexpect

Shoop• 7 years ago

sshpass works nicely.

rkeene22• 7 years ago

There's a program called "sshpass" to do this.

PascLeRasc• 7 years ago

There's another anti-pattern that I can't recall the name of, where instead of asking if something can do something, you make the claim that it can't, and let them refute you. It's excellent for something people get very defensive over, like Unix.

koala_man4• 7 years ago

When I tried googling your exact question, the first hit (https://serverfault.com/a/512220) was a SE post where the highest voted (though not accepted) answer was doing exactly that with `sshpass`

Tbqh, without you explicitly stating that you're aware of key pairs and concluded that storing plaintext passwords is the best solution for you, I would excuse anyone who thought that you maybe just hadn't done the research.

daveFNbuck• 7 years ago

How is the second answer second-guessing? Replicating the feat is a question of both software and hardware. To give an answer that's just about hardware would require second-guessing the intent of the question.

mlevental• 7 years ago

it's not just annoying but also offensive because you're being condescended to

threatofrain• 7 years ago

Second guessing is just called theory of mind in a complex world. You’d expect an futuristic AI to have this judgment too.

_uvvk• 7 years ago

"I'm not interested in Go, I'm interested in the power of future computers!"

Reminded me of this story. I told a co-worker, while we worked on our DB:

  - I wish we had the money to run Oracle.
  - Are you crazy? Postgres is awesome and perfect for us!
  - I know. I'm not saying "I wish we were running Oracle". I just wanted to have the money to do that. :)

salgernon• 7 years ago

“If we had some ham we could have ham and eggs if we had some eggs.”

emilfihlman5• 7 years ago

Seriously, people second-guessing you are annoying.

And? You are the extremely small minority. You need to learn to live with it.

koala_man4• 7 years ago

A bigger problem for you specially, or for communities in general?

rbavocadotree• 7 years ago

"How do you implement a new system call in the Linux Kernel?"
My problem is I want to understand how system calls are implemented thus I want to go through all the steps of making a new one so that I don't miss something"

These are very different statements, and require different answers. It's like asking "How do I sort a List" vs. "I would like to understand sorting algorithms, what are some resources to start with?".

People second guessing led to what you actually should have asked in the first place.

gitgud• 7 years ago

This presumes that the asker is intentionally concealing information. I think it's better to assume that they just don't know that some details of their problem might be extremely relevant...

bobthechef• 7 years ago

Depends. This is a matter of prudential judgement. The proper attitude is contextually determined and the linked page lists some qualifications, though still takes too much of a narrow, absolutist attitude.

For example, if someone random poses a problem on IRC, the solution to the "real" problem to be solved is NOT the concern or responsibility of the people in the channel. Unless there's a reason to ask for more context, just answer the damn question. Blindly asking the user what they're REALLY trying to solve is taking an intrusive and patronizing attitude. However, if asking will provide helpful context, then that's a valid reason to ask. If the problem as posed seems onerous or contrived, you COULD courteously ask the other person what they're ultimately trying to solve in the spirit of offering to be helpful if the other person so desires, but no one is hiring you do solve the "real" problem for them, so mind your own business.

If a coworker approaches you with a question, similar principles apply, but if you're working on the same project and in the same context, it may be natural to ask about what the larger problem is in a spirit of collaboration, esp. when the questions posed by your coworker smell funny. After all, you both share concerns about a project, though again, this does not give you license to automatically wrest the problem from your co-worker. His ticket, his problem. Your role is advisory.

So, common sense. Don't be one of those doltish know-it-all jerks who like to run the show. Mind the proper boundaries of concern.

jypepin• 7 years ago

Most comments here are about how annoying people asking about X are.

We get it, sometimes you are asking about X simply because you need to know about X and you know what you are doing.

But often, especially when dealing with more junior engineers, I've found this simple to keep in mind and helpful. It's also a good proof of experience to know when to ask about X and try to make sure you understand the context of the question before answering it.

kwoff• 7 years ago

It can be telling how people react to being asked about X. I tend to ignore conversations if someone gets annoyed about getting asked what they're really trying to do. Though you do have to consider whether they just don't understand enough about what they want to do, to know what question to ask.

A special case of Y questions are the overly abstract or general ones. "How do I [solve this kind of problem]?" It's understandable that people try to build up a repertoire of problem solving techniques, but this can be very annoying if the specific X question is simple to answer, whereas the general question is difficult. Sorry to say, but I tend to also ignore questions (I'm thinking of chat rooms) that seem overly general any more, unless I'm in a mentoring mood.

will4274• 7 years ago

Agreed. And sometimes X is not permitted or unlikely to be permitted.

In my day job, I work at BigCorp and I get questions like "how do I Y without Z?" Often, the answer is "you shouldn't Y without Z. In order to have a consistent UX, all BigCorp applications have a policy that Y is done by Z... What is your X?" And then we can talk about if X really is exceptional.

There is a world of difference between a forum like stack overflow where individuals have diverse needs and a forum inside a perscriptive company where uniformity is an engineering or usability booster.

trevyn• 7 years ago

“I want to make more money.”

Yes, but what do you really want?

almostarockstar2• 7 years ago

Happiness, satisfaction, contentment, comfort, security...the list goes on.

dsr_• 7 years ago

If a programmer whom I know and respect in general asks me how she can do X, I tend to start with an answer about X.

When a name I've never seen before on a -users mailing list asks me how to do X -- and X is bizarre -- I assume we have the XY problem.

We have the XY problem because the Internet is full of newbies at any given point, and we've wasted enough attention-span having sixteen-part conversations about how to extract the IP address from ifconfig output* that we'd really like to know if we can skip all that and answer with "Use a dynamic DNS client, don't write your own."

*Don't. Use ip a

...

Use awk.

...

There are an infinite number of guides to awk, but mostly you need to know how it splits strings into fields and prints the ones you specify. How are you planning on handling machines with multiple IPs, by the way?

...

Stop right there. Get a dynamic DNS client.

rntksi• 7 years ago

The problem most often is asking X because I want exactly X, but people assume I want Y!

nphase• 7 years ago

Conversely, there is the XY fallacy: when one [frequently condescendingly] assumes they know better than the person asking, errantly classifies it as an XY problem, and fails to answer the original question.

waynecochran• 7 years ago

Yes. Sometimes X is a hairy problem and perhaps well beyond the scope of what anyone cares about, and Y is a well thought out distillation of a sub-problem.

One of the problems I have with stackoverflow is crafting a question so that only those who have the potential to answer the question will answer it. Unfortunately there are always the folks who want to be first to answer the question and don't understand the fullness of the question and proceed to give a quick, poorly thought out solution. This then keeps others from chiming in.

For example, in one question I was asking about JS array performance and was using bubble sort as just a way to exercise arrays. Immediately I get folks mocking me for using bubble sort -- they completely missed the point of the question.

https://stackoverflow.com/questions/26199420/any-way-to-impr...

OTOH, there are folks who give brilliant answers.

https://stackoverflow.com/questions/36183602/why-is-nodes-ob...

pyb• 7 years ago

It's commonly because they don't know the answer, but still feel the need to answer something...

nullc4• 7 years ago

Fallacy Fallacy.

False positives on xyproblem appear to be much more rare and much less harmful than true positives.

At the end of the day a good question provides both context and an indication of things that the asker has tried/learned so far. The person being asked is a person, not an answerbot.

almostarockstar2• 7 years ago

It's quite funny, the number of replies in this thread saying: "No, I really do need Y because I'm special!"

Just remember where you are. The XY problem is a far larger problem in a far larger population than HN.

zamadatix2• 7 years ago

I think the article focuses on the real disconnect in it's explanation of the problem. The disconnect being that the step "Others try to help user with Y, but are confused because Y seems like a strange problem to want to solve." is just one extreme in the scenario with the other being "Others ask what it is you're trying to solve instead of talking about the question".

For most questions of the nature each extreme is normally a horrible way to approach it. Something like "You can do it via <method> but there might be a cleaner way to do things depending on what you are trying to accomplish in the end" such as the examples in the page goes miles ahead in being helpful and reduces the overall amount of back-and-forth. This doesn't get a fancy label to reference and articles written about it though as that's just known as a "conversation".

winstonewert2• 7 years ago

Here's the thing, if you ask an unusual question, one of two scenarios is often playing out:

In case 1, you are a beginner. You have an XY question. But you've never heard of an XY question. You also probably couldn't identify your question as an XY question if you had heard of it. Trying to educate people about XY problem is pointless, because beginners don't see it or understand it or have the ability to apply it.

In case 2, you are advanced enough to avoid asking XY questions. You are instead stuck on some esoteric question which you can't resolve by exhausting all your skills. So you post a question on the internet. But you are only asking a question because it is a really hard question. Chances are that most other people cannot answer the question either. So, the only responses you get are people suggesting that you have an XY problem.

nullc4• 7 years ago

Trying to educate people about XY problem is pointless,

I haven't found that to be the case.

Neophytes are perfectly capable of asking questions with a reasonable amount of context: they just need to be taught to do so even when it seems unnecessary to them. Polite explanation to this end works.

to avoid asking XY questions. You are instead stuck on some esoteric question which you can't resolve by exhausting all your skills. So you post a question on the internet. But you are only asking a question because it is a really hard question.

When you describe the context and the things you've tried so far you will signal to people that you're operating at a level beyond their ability to help.

Asking a high quality question is a universally good move for both newbies and experts alike. It takes more effort, but when you are asking strangers who owe you nothing to put in effort on your behalf it can be a demonstration of good faith to put in obvious effort on your part.

DanielBMarkham2• 7 years ago

There's a money side to this that's also important.

Had a friend call me a few months back with a technical problem. "How do I fix Y on my server?"

Now I knew that server wasn't bringing in a lot of money, and the Y he wanted to fix could be accomplished the same way with say, 100 bucks of purchasing something else. But I figured it was better if he figured this out himself.

So I asked him what X he was using it for.

That was it. We never got past that, mainly because he was dead-set on spending a week or two of effort and tens of thousands of dollars on something that was (pretty much) trivial.

So after circling the barn a few times on the phones, I gave up. Told him that yes, based on what he was saying he had a fix for Y that was fine. It was just going to take a long time and cost a lot of money.

"But there has to be a quicker, cheaper way!"

Argghhhh.

Yes, there is. But if you can't figure out that you're solving the business problem first, the technical problem second? There might not be one. Technology exists for a reason, and technical problems exist because business needs are not being met. Fix the business problem. It'll drive the tech decisions.

It was quite a frustrating call. For both of us. The X of the XY problem might be critical. Or not. People who ask dumb questions get dumb answers.

fwip• 7 years ago

Maybe just be nice to your friends.

DanielBMarkham2• 7 years ago

I was. We were nice to each other. He's a dear friend. I don't think either one of us were being unkind to the other. I stayed on the call as long as I could. I wanted to help him out. He was just trying to scope down the conversation at the wrong time. I understand his motivations and reasons, but it was premature.

I worked with a really smart leader of a large agency many years ago. Not only a nice guy, he was also brilliant and on top of everything.

I could see several projects he had were going off-the-rails. Why?. Because he kept asking technical questions and getting technical answers. The questions were fine but context-free. The answers were good but context-free.

There was no X. He kept asking Y questions and getting really smart, detailed answers. It was a terrible mess. If you have no context for a technical answer, the only thing an answer is going to do is confuse you.

It's like the old joke about the guys getting lost in a hot air balloon. As they passed over a building, they saw some people on the ground.

"Where are we?"

"In a hot air balloon!"

Now that's a joke, but it hides a truth: without context and a shared language you can get answers all day long to questions and not be any better off at the end than when you started. Probably worse off.

mycall• 7 years ago

I know solution to hiding OS from nmap. Use exotic OS, like ReactOS, XTS-400, Genode, etc.

lxe• 7 years ago

Unfortunately, the assumption that "someone who's asking for help with something doesn't really know what they want or need " breeds horrible culture.

This is evident by the examples in the article and the ones in the cited sources. If someone is asking for help, don't try to shame them into explaining what "they really want". What they want is help with a morsel of a problem they ask, nothing more. If it's not the right path to a solution, they will decide, not you.

jayd16• 7 years ago

shame them into explaining what "they really want".

You're already starting from a toxic situation where asking for help is somehow bad. Besides, you have the same "I know better" attitude in this case, just on the opposite side.

It might get on some nerves to start having to answer "why do you want to do that?" when you're in a siloed culture but eventually people adapt and it just leads to more thought out plans.

Asking why just adds more context and it doesn't need to turn into a second guess of the solution. Its better to breed a culture where asking for context is considered a good thing with no emotional baggage tied to it.

chrisg3• 7 years ago

Exactly.

And this isn't helped by the fact that reading internet text generally leads to misinterpretations of the authors intent.

Note, from the article in the second example:

"Then ASK FOR WHAT YOU WANT!"

Likely, the person who wrote that text was not angry, nor frustrated when writing that, but it very much seems that way to a reader.

cheerlessbog• 7 years ago

What else is intended by the capitalization? To me it means shouting and I don't do that unless I'm mad.

TeMPOraL• 7 years ago

This applies to conversations on the Internet in general (and probably to meatspace ones too) - unless there are obvious signs suggesting a different meaning, taking what someone wrote in any other way than at face value is disrespectful and rarely leads anywhere productive.

luhn• 7 years ago

It doesn't need to be a negative, condescending thing. Taking the time to ask "What problem are you trying to solve?" can alleviate quite a bit of frustration and wasted time. It's a simple question. It's not assuming they don't really know what they want or shaming them—You're trying to better understand the problem they're solving so you can give them a better, more holistic solution.

czr• 7 years ago

I agree that shaming / confronting people is not good for culture, but I disagree with the conclusion. "Y can be done (see below), but if you're actually trying to accomplish X here's a much more convenient way" is the kind of response that I appreciate most–not only has someone answered my question, but they've also helped correct whatever misunderstanding prompted me to ask about Y in the first place, and they've given me an easier way to solve X.

For me, at least, a response that solves both X and Y is (if done politely) far preferable to one that only solves Y.

emilfihlman5• 7 years ago

You are in the minority.

marblessubtext• 7 years ago

If it's not the right path to a solution, they will decide, not you.

This is a dangerous position to take. I have a junior dev on my team frequently asking for help on Z when X is still a problem. If I actually answer his question, what he puts out is a disaster.'

Like he was having trouble with generating prime numbers. I could have answered his question, but why is he generating prime numbers? Long story short, to create an "optimal" number of threads to insert to a database because inserts were slow and he had 100k to do. Dude. Batch insert. Why are we threading, absolutely not, no. Stepping back Z, Y, X, and V and finding a better solution. 1/10th the code, 1/20th the time, better for the DB.

When you have someone on your team doing things like this, absolutely ask about X.

parentheses• 7 years ago

The conflict between describing X vs Y is also about a few things:

- the size of signal: asking a pointed (Y) question saves typing/reading/etc. in some situations - saving the person time and showing that you came to them after some thought (rather than with the original problem)

Asking Y can quickly become about X even if you begin by asking Y.

e.g.

Y: where do I find the foo.bar config value

X: I need to implement feature to _k_

drngdds2• 7 years ago

Is it just me or is the XY Problem really poorly named? It's unlikely that anyone could hear "XY Problem" for the first time and have any idea what it means.

arithma• 7 years ago

Maybe calling it "ButWhy?Problem" would help

333c• 7 years ago

What about the "Oz problem," as a reference to the man behind the curtain in the Wizard of Oz?

appleflaxen• 7 years ago

I completely agree.

Maybe the "presumed solution problem".

anoncake• 7 years ago

n00b doesn't actually want the last 3 chracters in a filename, he wants the file extensions, so why ask for the last 3 characters?

n00b doesn't actually want the file extensions either, he wants to <insert purpose of the program he's working>.

n00b doesn't actually want <insert purpose of the program he's working> either, he wants not to get fired from his job.

n00b doesn't actually want not to get fired from his job either, he wants to acquire money.

n00b doesn't actually want not to acquire money either, he wants to avoid starvation.

Looks like the answer to "How can I echo the last three characters in a filename?" starts with "Buy some seeds".

iovrthoughtthis2• 7 years ago

Give people context and explain your end goals.

micimize• 7 years ago

While it's an important phenomenon to be aware of, its name fails to carry any semantic meaning. Should be named something like "falsely known solution" or "context stripping" or something

drngdds2• 7 years ago

I'd just call it "asking the wrong question."

jacinabox• 7 years ago

It makes me think that whole software projects (and popular ones at that) have fallen prey to the XY problem. For instance, are containers really necessary, or are they shoring up a lack of robust dependency management allowing different software programs to use different versions of the same application simultaneously.

davidivadavid2• 7 years ago

Yes, the most common type of XY problems I see is people trying to solve made up problems. It happens to everyone. Lately I was browsing the Google X website [1].

Here's the kind of stuff you find there:

WE CREATE RADICAL NEW TECHNOLOGIES TO SOLVE SOME OF THE WORLD’S HARDEST PROBLEMS

(Why would you solve the hardest problems as opposed to the most valuable ones? Oh well, let's keep going)

How can balloons deliver the Internet to rural and unconnected places?

(I don't know, but are we sure using balloons is really the most sensible solution to that problem?)

How can kites be used to generate electricity in unexpected places?

(I don't know why we'd want to bring electricity in "unexpected places" as opposed to places where it's required, or why kites would be the best solution for that either)

Etc. It's very easy to create very hard problems if you impose unnecessary constraints.

[1] https://x.company/

burlesona• 7 years ago

I just want to point out that the example is really poor behavior on the part of the teacher: Shouting in all-caps to ASK FOR WHAT YOU REALLY WANT is more likely to result in the newbie not asking at all next time.

If this is your work environment you need to make it okay to ask questions, and to dialogue about a problem without shaming the learner. (Really that should be civil behavior in any environment.)

fishtoaster• 7 years ago

One thing I hate is when the XY problem gets encoded into a knowledge base:

1. It's common for people trying to do Y to ask about X, which is a bad way to accomplish Y. 2. I'm trying to accomplish Z, for which X is a perfectly good option. 3. Every answer on stack overflow for "How to X" is people assuming the XY problem and answering "here's how you do Y instead".

londons_explore• 7 years ago

Can someone explain the rationale of setting up a domain for just one blog post?

As well as seeming like quite a lot of effort, the fact domains expire means that content probably won't past 2 years unless you fancy keeping paying for the handful of visitors it'll get.

koala_man4• 7 years ago

My rationale for doing this with linuxatemyram.com was so that people would be able to offer this FAQ to someone on IRC or irl without having to Google up that one blog post they once read on that topic.

It also helped focus the message by removing all the notion that this is one of many articles on a general tech blog.

kentonv3• 7 years ago

It's pretty lame that the examples on the page show the answerer not just answering but also talking down to the questioner. XY is a problem that happens a lot, but disparaging people who are trying to learn is not the right response.

Here's how that conversation should have gone:

---

Q: How can I echo the last three characters in a filename?

A: If they're in a variable: echo ${foo: -3}

A: However, if what you really want is to get the file extension, it's probably better to handle extensions of any length. Try this instead: ${foo##*.}

A: That expression says to find the longest prefix of `foo` that matches the given glob pattern, and remove it.

Q: Oh wow! That is actually what I wanted, but it hadn't occurred to me that such powerful pattern-matching would be built into the shell. I already had a solution involving piping through sed, but it was pretty slow and ugly. I noticed that all the files I'm operating on have three-letter extensions and I figured there'd be an easier way to take a slice of a string, since that's a common thing to have built-in. But your answer is the best of both worlds. TIL!

---

There's really no need to admonish the questioner for starting with the wrong question. That won't help. It will only make them afraid to ask questions at all.

sgillen• 7 years ago

I think it is worth telling (politely) the questioner that they really should start by telling people what they really want to do, in order to get better feedback in the future.

kentonv3• 7 years ago

I think the most polite way to tell them is not to tell them what to do, but rather ask: "Could you give us more background on what you're trying to do? It might turn out we know of an alternative solution that avoids this specific problem."

This doesn't suggest that the questioner did something wrong, but still helps them learn to provide background details when asking questions.

phkahler• 7 years ago

I will say something like "can we back up a minute? Why are you trying to do this?" And yes, my second question should be worded differently or spoken carefully to avoid being interpreted as "WTF are you doing?"

stevep98• 7 years ago

yes, surely the questioner is trying to balance “doing as much of the work as possible”, and “breaking the problem down into small steps”, versus “give me the answer to my entire problem”

If you ask too broad a question you’re likely to be ignored, imo.

learnstats22• 7 years ago

Very often, it's useful to frame a question in a way that someone with specific expertise can quickly help you - 'YY' situations come up all the time.

Nursie• 7 years ago

This is a great attitude. I tend to approach junior technical staff asking these sorts of of questions with a slightly different tack -

Q: How can I echo the last three characters in a filename?

A: What is it you're trying to achieve by doing Y? What's the context? You can probably do it this way ... , but let's see if it's a good fit for the problem you're trying to solve.

emilfihlman5• 7 years ago

People are there to voluntarily give help. They also get loads of really stupid questions and venting does help with keeping them providing help.

If your goal is to reduce help, then be my quest.

acpetrov• 7 years ago

People are there to voluntarily give help. They also get loads of really stupid questions and venting does help with keeping them providing help.

Ha, imagine hearing this about volunteers at a hospice. SO boards are infinitely less important, but volunteers shouldn't be losing their calm at the same people they're helping

Waterluvian2• 7 years ago

If you're a seasoned veteran here to answer my stupid question and the cost of answering my stupid question is to condescend then PLEASE KEEP WALKING. I do not want your help.

kentonv3• 7 years ago

Yeah... no.

If you must vent, vent privately to a friend. If you can't help but vent back to the person asking the question, then please stop answering questions, because you aren't helping.

cjaybo• 7 years ago

I disagree with your assertion that the QA community would be worse off if these types of people stopped answering questions. If a person cannot provide a helpful response to a question without being condescending or rude, then I personally think the QA community would be better off if that person reduces the 'help' they are providing.

Very few people have knowledge that is so rare and difficult to obtain that others should endure discouragement (or plain unpleasantness) in order to receive the benefit of their help. And, in my experience, the people who do have that sort of specialized knowledge are often some of the most helpful.

dang2• 7 years ago

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

https://news.ycombinator.com/newsguidelines.html

nullc4• 7 years ago

You proposed interaction assumes that the respondent was able to guess the users ultimate goal. That's often only the case on the Nth run-through of the exact same proxy question.

Asking the wrong question wastes both the asker and the answer's time. By all means be kind, but if you fail to take the opportunity to remind someone to be wary of this general problem you are doing both them and yourself a disservice.

learnstats22• 7 years ago

This is really about two people communicating effectively.

If I saw the original examples happen in the wild, I'd understand the person answering the question to be the problem.

koala_man4• 7 years ago

I'm one of the people who tend to ask why. Sorry to everyone on this thread who are frustrated by it.

From my point of view, I'm really not trying to undermine what you're doing, it's a genuine attempt at getting you the best solution.

Here are some examples:

How do I use apt on Fedora?
How do I unroll this bash loop?
How do I multiply this number by 1024 with sed?

I could have a lot of fun answering each question exactly as stated and if you say "I know it's the wrong tool for the job but I'm doing it for fun" that's exactly what I'll do.

However, it turned out that:

They just wanted to install stuff thinking apt was universal. The better solution was using yum
Their loop was slow and they had read that unrolling was an optimization. The better solution was reducing the number of subshells spawned
They just wanted a result, not to implement an integer multiplication state machine in sed. The better solution was using perl/awk

I know some people are jerks about it, and that sometimes you want answers to weird questions. When people do appear to be going down the wrong path though, it would be a disservice to help them do that without letting them know that there's probably a better way.

Sorry for any false positives.

dmd• 7 years ago

I made this slide about XY at a previous job: https://i.imgur.com/hH97U9b.png

ibudiallo• 7 years ago

I remember in the early days of phone apps, you'd find thousands of different apps that try to solve a trivial problem. It inspired me to make this video: The alarm clock problem https://youtu.be/ebQAM5ADfYQ

dang2• 7 years ago

Thread from 2015: https://news.ycombinator.com/item?id=10023882

amelius• 7 years ago

The inverse also happens:

You: I want to find a solution to X.

Response: you probably don't want to solve X, but rather W. Here's a solution for W, ...

You: sigh

chucksmash• 7 years ago

This gets reposted now, a day (ish) after I saw someone comment "is this an instance of the XY problem?"

I suspect that people are mining data from ("paying attention to") certain posted keywords and then re-posting.

How might I disprove the null hypothesis that people are just repackaging highly voted comments? How might I convince myself that I am wrong or right? Please show your work or cite sources.

hitekker• 7 years ago

Lazy social-signaling drives the over-application of the "XY Problem".

In a private medium, Questioner and Answerer can delve into the wider context and arrive at a well-informed solution.

In public however, any line-of-inquiry is corrupted by its presentation. In the specific context of public Q&A, this means that the motivation of the average answerer is often just to _present_ themselves as knowledgeable without actually being helpful.

In most difficult questions I've seen posted on SO, most answerers have neither an understanding of the context nor the basis with which to compose the "right" answer. Compounding this ignorance is that the Answerer fears losing face and also that he or she is not not personally or deeply invested in the question itself. Thus, a clueless Answerer who accidentally or foolishly interjects in the thread, will eventually default to a simple strategy: negotiating the question.

Less "digging" and more "dumbing", the question is simplified or distorted beyond its original intent so the Answerer can "answer" or rather discount it. They thereby "resolve" their stake in the question posed: giving them (or at least not hurting) the esteem they crave in the larger group. At the very least, they'll have said something and appeared, in their minds, to be just as smart or smarter than the questioner.

StackOverflow's basic design weights against this kind of child's play, since only the Questioner can award the Answerer the checkmark. But I think the more popular Questions still suffer from the transients desperately seeking their ego-fix.

l0b0• 7 years ago

Basically lifted straight from https://mywiki.wooledge.org/XyProblem. GreyCat's wiki is the best Bash resource in existence, but the first example is pretty typical for the IRC channel. There's really no reason to berate people. For example, lots of people have never or very rarely been exposed to anything other than three letter extensions (including most DOS/Windows users), and it's entirely possible that they were trying to do something which only involved three letter extensions. Not every script needs to be a completely generic solution.

rstuart4133• 7 years ago

The problem comes in many guises. A similar problem arises when someone says "Y" doesn't work. After going on a long trek you find out "Y" is just fine - they had some other problem "X", had done some diagnosis and decided it was caused by "Y" being broken.

After many years of being dragged down this rabbit hole my first question is now "what makes you think Y isn't working". If the answer is "because X is doing ...", you've just dogged another hours wasted time in another rabbit hole.

aib2• 7 years ago

I often encounter this problem in the form of newbies asking how to dynamically create variables inside a loop:

"foo1 on the first iteration, foo2 on the second, etc."

As much as I am a fun of weird little endeavors like programmatically modifying the local scope, my answer is usually along the lines of "...arrays."

The XY Problem (2014)

Related Stories