Never ask a candidate to rate themselves out of 10

by zug_zug • 2 years ago

30 75 2 years ago

i'm surprised people bother asking questions this inane. (or at least surprised people think they get value out of this.)

justmedep• 2 years ago

If someone responds with “10” and fails to answer basic questions about that specific topic you know that the person is either lying or is overly confident.

bobthepanda3• 2 years ago

there are so many other ways to figure out if a person is not an expert. like just asking them basic questions without the bs.

doe_eyes2• 2 years ago

If they selected "5" and failed to answer the same question, would they be a hire? If not, what's gained by having them self-assess?

I imagine the answer might be "we wouldn't waste time talking to them", but then, you just created an incentive for a person who would otherwise rate themselves as "5", "6", or "7" to pick a higher number in hope of squeezing through. You then penalize that in the interview process. Seems pretty counterproductive?

function_seven2• 2 years ago

It's bullshit psychological gaming. Obviously 1–6 are right out. Can't pick any of those. 10 is also out. (You're not that arrogant, are you?)

So the question becomes: "7, 8, or 9?"

Still some strategy to think about here. If you answer "7", then you're saying, "I'm not that good but even I know not to answer lower than 7". Answering "9" is similar, except you're saying you know better than to go full self-aggrandizement.

Yup. Nothing of value is gleaned from this question.

bell-cot• 2 years ago

10 is also out.

I recall an amusing HN story from a couple years ago: $candidate was asked about his proficiency with $open_source_program. $candidate was challenged on his not-so-humble answer ...and then suggested that the interviewer compare his name with that of the author/lead maintainer of $open_source_program.

So the question becomes: "7, 8, or 9?"

I'd throw in a few plausibly-relevant things that zero-ish candidates will have meaningful proficiency at, as an honesty test.

Suppafly• 2 years ago

Reminds me of the self assessment part of the yearly reviews at my job. No one is going to give themselves a below average rating on any question and generally score themselves higher than normal hoping their boss agrees on at least a couple of the questions.

scottlamb4• 2 years ago

It's bullshit psychological gaming. Obviously 1–6 are right out. Can't pick any of those. 10 is also out. (You're not that arrogant, are you?)

Context matters. Back in 2007, a Google recruiter asked me to rate myself 1–10 on maybe 20 different areas. I gave a bunch of 1–6s, probably a couple 7s, maybe an 8 (don't remember), no 9s or 10s. My interviews focused on the areas I'd self-assessed most highly on, and I got hired.

They set expectations as described in my other comment here: https://news.ycombinator.com/item?id=40907661

fsckboy2• 2 years ago

from what you wrote, sounds like 8 is a solid answer.

azornathogron• 2 years ago

When I was asked this (early in an application process) the question came with an anchored scale. That is, each number had a corresponding description. I thought it worked fine.

samatman• 2 years ago

This is entirely different, and does avoid the central problem discussed in the Fine Article: is 1...10 to mean ±5σ, or are they deciles? Either way, what's the sample group? It also sucks because there's not a whole-number median answer: 5 is further from 10 than it is from 1, 5 + 5 is 10, 5 - 5 is 0, not 1.

However, ten levels is too many. Five is better. With ten levels, someone is always unsure whether they belong one higher or one lower than the answer they give, so you have a test which combines a real self-assessment with an unknown variable: the personality of the person you're asking.

For a skill of interest, five levels will give most candidates a clear image of where they actually belong, and the differences are large enough that a dishonest answer can be cross-checked with straightforward "I expect a 4 to answer this correctly" questions.

scottlamb4• 2 years ago

As usual, you can do a thing relatively well or poorly. The post is describing the dumbest version of it and attacking it.

iirc, Google recruiters used to (still do? dunno) ask people to rate themselves in a bunch of areas.

* Why are you asking? So we can pick interviewers knowledgeable in the same areas you are.

* How should I rate myself? Here's a reference. 10 means you are the world expert, maybe invented the thing I just described. 9 means you literally wrote the authoritative book or are on the standardization committee. 8 means you wrote a book on it or the like...and so on.

In general, you should expect that you will discuss the things you rate yourself the most highly on with someone at Google who knows them better than you do, or at least (if you really are the world expert) one of Google's most confident people on that subject.

I think this was a useful practice, at least in the context of Google where they have a very wide interview pool and actually sometimes do interview people who literally wrote the book on something.

xmprt2• 2 years ago

But it's still a useless question isn't it? If someone literally wrote the book on the subject then you should probably know that if you're interviewing them. And anything below 8 feels meaningless. How do you differentiate a 5 vs 6 vs 7? Why can't the answer just be "what do you know about X" instead of having to use an arbitrary number?

scottlamb4• 2 years ago

If someone scores a 10 or 9, yeah, ideally the recruiter knows that rather than having to ask, but people are imperfect, and regardless the recruiter will be entering the 10 or 9 into the interviewing system to find the matches.

How do you differentiate a 5 vs 6 vs 7? I don't remember the guidance they gave, but you certainly give yourself higher numbers on areas you know better, and this gives you the opportunity to significantly influence what you will be interviewed about and by whom. Google mostly was (is?) hiring generalists. They assumed if you know something well, you can pick up other things too as needed. But if you bomb the thing you say you know best, that's not so good.

turtle_heck• 2 years ago

* How should I rate myself? Here's a reference. 10 means you are the world expert, maybe invented the thing I just described. 9 means you literally wrote the book. 8 means...and so on.

This is exactly what I'd do, ask what constitutes a "10", and if the interviewer can't answer that question why are they even asking?

coleca• 2 years ago

I do the same thing at AWS, except use a 100, 200, 300, 400 level scale. I am just looking to see what the candidate thinks they are strongest on and weakest. If the candidate says they have never heard of AI/ML topics, there's no point in quizzing them on it. Not everyone can know everything, it wouldn't be fair to judge someone based on not being able to answer questions for a domain they don't claim to have any experience in. Or worse yet, waste valuable / limited interview minutes on fruitless questions.

SketchySeaBeast• 2 years ago

The only way I'd ever answer “How would you rate your knowledge of techonolgy[sic - but c'mon, is this supposed to be a professional blog?] X out of 10?” with a 10 is if I built the technology myself, and then probably not even then as I know there's someone out there who is going to make it do things I never even imagined.

oldpersonintx• 2 years ago

as inane as asking people their biggest weakness

"me? my biggest weakness is that I work TOO HARD!"

kelseyfrog• 2 years ago

"I'm generally content with initial offers."

Bootvis• 2 years ago

I see candidates putting skill level bars, e.g.

    ------- 1 ------- 10
    C++:    xxxxxxx
    Python: xxxxxxxx
    SQL:    xxxxx

I'm not seeing any upside. Are you someone that did this, why?

xmprt2• 2 years ago

I did this. I was a freshman in college, didn't know better, and needed to pad out my resume somehow.

bobthepanda3• 2 years ago

I think at some point this started spreading around on blogs as "a thing you can do to separate yourself."

A lot of resume advice floating around is not good.

advisedwang2• 2 years ago

I don't do this, but I considered it because it allows you to list a wide range of skills (critical for getting past filters) but also tempers expectations in an interview. E.g. I can and have programmed in Java, but if you ask me about less common Java features I'm probably going to give pretty weak answers.

doe_eyes2• 2 years ago

I have seen this used two ways:

1) As a recruiter-level screening question. A recruiter can't evaluate your answers to a coding problem, but they can reject all candidates who select less than "7" on a slider. It's basically a cheap way to go from 1,000 resumes to a more manageable number that's actually shown to the hiring manager.

2) As a "gotcha" on technical interviews: oh, you selected "9" on C++? Well, here's my favorite IOCCC entry, figure it out! You can't? "No hire, the candidate lied." It's counterproductive, but many engineering organizations approach interviewing this way (also see: "brain teaser" questions).

Between #1 and #2, the safest answer is always "8".

kevindamm2• 2 years ago

There was a ratings system like this in early days of Google but it was accompanied by a very explicit guideline for what each number means. IIRC 6 was demonstrated mastery, 10 was "literally wrote the book on the topic."

It wasn't for candidates, though, it was initially for interviewer stats and possibly maybe referred to for team transfers.

(the current version is in a completely reworked system and on a four-point scale that includes "interested" and "currently learning" so really only a two point scale of claimed knowledge)

advisedwang2• 2 years ago

You should ask the following about every interview question you are considering asking:

1. What answers would lead to a hire and what answers would lead to a reject

2. Does this align with someone able/not able to do the job.

Without these the interview question doesn't provide you the information you need as an interviewer.

notRobot• 2 years ago

This is great advice.

influx• 2 years ago

3. Have I tested this question on current employees?

4. Is this a puzzle question with only one answer, or are there subproblems they can still solve and I will get actionable data from?

red-iron-pine• 2 years ago

I like these 2 more

pflenker2• 2 years ago

I would like to politely disagree. Unless you are hiring robots or people working in a robot like role, there is - extreme outliers notwithstanding - almost never a single kind of answer that would make or break the interview.

When interviewing, you should be looking for evidence that the candidate might be able to deal with the problems that they are confronted with in their new role. Questions should be directed to finding that evidence, and answers can basically provide you with evidence, provide you with evidence to the contrary or be neutral. At the end of the interview you hopefully have collected enough evidence so that you are convinced the candidate is able to successfully work in the problem space their new role brings.

episteme• 2 years ago

You don't disagree then.

rhelz5• 2 years ago

chuckle back when I worked at Intel, a question we'd ask candidates for a C++ programming position was, "On a scale of 1 to 10, how would you rate your understanding of C++?"

Anybody who gave an answer of 5 or above was an immediate no-hire :-) Because nobody is more than a 4 out of 10 on knowledge of C++.

digitalsushi• 2 years ago

how did you hire interns then? what a terrible scale! if i was wet behind the ears and giving myself an 8 cause of what i dont know i dont know, it's your job to hire me and teach me how wrong i am!

rhelz5• 2 years ago

OP here. This is probably the most down-voted and dumped-on post I've made on HN.

Answer: We hired interns the same way. Is that really so bad though?

How would you rate yourself on a scale of 1 to 10 on C++: 1 being you know C, and 10 being you are Bjarne Stroustrup or Bruce Eckle. Personally, I started out as a 4 back in 1996, and have monotonically fallen ever since.

pton_xd• 2 years ago

Anybody who gave an answer of 5 or above was an immediate no-hire :-) Because nobody is more than a 4 out of 10 on knowledge of C++.

You'd definitely hire programmers with the same sense of humor I suppose.

I do agree that asking to rate one's knowledge of C++ is an obvious trap question. Any reasonably seasoned person would probably laugh and respond with another question, "... which part?" But yeah they'd have to share your sense of humor.

drewcoo4• 2 years ago

You'd definitely hire programmers with the same sense of humor I suppose.

There are many ways to structure interviewers to hire clones of yourself.

rhelz5• 2 years ago

How would you rate yourself on a scale, from 1 to 10, on C++ knowledge? 1 being you know C, 10 being you are Bjarne Stroustrup or Bruce Eckle? Personally, I was a 4 back in 1996, and have monotonically fallen since then.

scottlamb4• 2 years ago

I wouldn't be so absolute about it, but I certainly empathize. Any time a candidate told me they were an expert in C++, it didn't go well. I think that suggests they are either wildly overestimating their abilities or are too interested in being "clever" and using weird corners of the language that aren't necessary/beneficial for the problem and that no one else will understand. Probably both.

racional3• 2 years ago

Then it's a straight-up hazing question.

And not only that -- it betrays the mathematical (and everyday practical) ignorance of the people who apparently think of themselves as wily and clever for asking it.

It doesn't matter how weird and fuggly that particular language is. The only obvious, intuitive and reasonable way to approach the "How do you rate yourself?" question is always in relative terms, compared to the pool of candidates applying for the job (or perhaps compared to other engineers at the company).

Which either way means a fricken bell curve, obviously and by definition. For which 5 is the median, and the only acceptable answers are "7,8,9".

But instead someone on your time decided they had to be cute and turn it into a "Guess the punchline to the snarky joke question I came up with this morning" question.

This is Intel's idea of culture, apparently.

jltsiren• 2 years ago

There is a lot of cultural baggage in rating scales. For example, if you were a high-achieving kid in a Finnish school and the scale goes up to 10, you intuitively assume that 8 means bad, 9 means ok, and 10 means good. Which means that you should rate yourself 9 in everything you are comfortable using and 10 in things you are good at.

On the other hand, if the scale is from 1 to 5, it implies a bell curve with a low standard deviation. Especially if the scale is measured in stars. So if you took an Uber to the interview and nothing unusual happened, the driver should obviously get a three-star rating.

At one point, I would have interpreted a scale from 1 to 10 in terms of an RPG I was playing at the time. 5 meant a beginner who had been focusing on that thing, 6-7 was basic competence, and 9-10 was an expert, but not a particularly unusual one.

drewcoo4• 2 years ago

Then it's a straight-up hazing question.

Hazing is an initiation rite. This is a barrier to entry.

They turned away anyone who wasn't deemed humble enough.

They were assholes but that's not hazing.

rhelz5• 2 years ago

We did look very unfavorably at candidates who had an unrealistically high opinion of themselves.

But we didn't ask dick C++ questions. Like list all 200 ways you can initialize a variable in C++. And, once somebody has told you they are 4/10 in C++, you can't ding them if they don't know your favorite C++ feature, or if they haven't memorized the next version of the standard.

IMHO, interview questions should not be about what the candidate knows. At all. When I went to Intel, I knew nothing about chip design or EDA software.

Which was OK, because chip design methodologies were all obsoleted by Moores law, and the techniques needed to fix it hadn't even been invented yet.

What is important is what you can learn and how fast can you learn it.

monkpit2• 2 years ago

I think this question is very revealing when paired with another related question. The rating is unimportant and not to be used as a requirement, anyone can lie and say 10.

I feel that it shows insight into how strong a candidate really is and how much they are bluffing, for example “rate yourself in JavaScript” and then asking a JS question. If their answer lines up with their self-assessment (somewhere in the ballpark) then it seems more likely their other claims are true.

However, I would never ask this question and rate a candidate on whatever number they decide to come up with. Lots of times the most skilled engineers rate themselves lower than you would expect.

So yeah, the number is meaningless, but I think the question can be revealing in an interview setting.

aeonflux• 2 years ago

I do ask this question in a initial questionnaire. I list bunch of technologies and just require this input. I do agree with all the points in the article, but my goal is to have relative scale. I am not able to verify all the needed technologies, but I can do verify few. Whatever the scale candidate choose, I assume its consistent. If someone self-scores 9/10 out of Postgres and doesn't understand basic things, I can safely assume his other 9/10 are worthless.

LargeWu2• 2 years ago

Agreed, it's a bozo filter. It's not intended to find good candidates, it's intended to weed out bad ones.

Ekaros• 2 years ago

Hmm, if they ask you to rate yourself out of 10. Should you then ask them to rate various aspects of their company on same scale? Like work-life balance, leadership, team's technical skills, competence of managers and so on?

lesuorac• 2 years ago

Do you not ask interviewers questions during an interview?

Like yes, during an interview you should attempt to learn what it would be like to work there.

l5870uoo9y• 2 years ago

Have been freelancing for ages and in my experience talking about recent books you have read that are relevant to the employment absolutely seals the deal. Implicitly it shows you are 10 out of 10 on a topic.

pflenker2• 2 years ago

This article misses that the asker knows about these limitations as well, and it also misses that the answer in itself alone is devoid of meaning for the interviewer.

It’s the context that matters - for example a comparison of skill A which the candidate rates at x out of 10 to skill b where they rate themselves a y out of 10. Both x and y are meaningless, but the relationship between these two is not.

Self awareness is also important, both in terms of strengths and development areas. People need to be aware of their blind spots because we expect them to take a step back and let themselves be led where appropriate. And people need to be aware of their strengths because we expect them to step up and take the lead if necessary.

A quantitative scale for qualitative things really helps having the right discussions with candidates.

WalterBright• 2 years ago

I was once asked: "Where do you see yourself in 5 years?"

I answered: "CEO of this company"

Didn't get an offer :-)

anamax• 2 years ago

"probably somewhere else" is the correct answer in the US.

drewcoo4• 2 years ago

Also a no hire, though.

On a first date, people want to feel flattered, they don't want to feel they've been truth bombed.

tombert• 2 years ago

Sort of tangential, when I was interviewing for a job, one of the hiring manager asked me list my four (!) biggest weaknesses.

I can think of hundreds of weaknesses on my end, but of course it's an interview so you don't necessarily want to tell them everything, just stuff that either doesn't make you look too bad. It was extremely difficult, and I thought it was kind of a sociopathic question.

Granted, I did get an offer for that job (and took it), so I guess it all worked out, but it was a very strange part of the entire interview process.

neontomo• 2 years ago

a really talented developer i worked with told me that the absolute rock stars in the industry would call themselves 2-4 while the worst would say 6-8. dunning kruger in effect

monkpit2• 2 years ago

This is true, and it provides valuable insight into a candidate’s thoughts while taking very little time in an interview.

legitster• 2 years ago

Eh. If you are an experienced professional, you should be able to triage. Making semi-accurate, off-the-cuff judgements is exactly what you are being hired for. I've never minded these questions.

It’s entirely ambiguous, and depending on which interpretation you give, there’s really no wrong answer.

This is a feature, not a bug. There are going to be other parts of the interview process, knowing where you are confident and where you are not is an invitation for further discussion.

derekerdmann• 2 years ago

When someone asks you if you're a god, you say, "yes!"

why5s• 2 years ago

I've been asked this (on a scale from 1 - 5) and have had some success referencing previous performance reviews (i.e. meeting expectations or exceeding expectations). YMMV obviously.

jiehong• 2 years ago

Meh.

From 1 to 5, rate yourself and explain why.

The value itself isn’t what matters, but the reasoning of the candidate and let both of you discuss where the candidate needs some ramp up and where they want to grow.

At some point, a question is like a metric: once you track it and try to improve it, it ceases to be a good metric.

That’s why multiple interviewers cross check themselves and the probation period helps confirm or affirm the fit in the position and the team.

whitej125• 2 years ago

I ask this question - always - but it goes like this.

Q1: Rate yourself 1-10 on X language? Their answer is Y (and almost always 7).

Q2: What's the difference between a Y and a Y+1?

Q2 is real question with Q1 being there just for framing. Some people simply state that the difference between a 7 and an 8 is "just more experience". TBC, that's not a wrong answer.

However what gets me really excited is when this line of questioning shines a light on what the candidate is curious about, or interesting in learning next. It goes a step beyond just self-awareness that there is always more to learn. To me it's a signal that this person has a plan and a drive to grow oneself.

While I do not make binary hiring decisions on this question or any question; I can vividly remember the answers I was given to this line of questions by some of the best engineers I've worked with.

kohbo• 2 years ago

I like this reasoning but would rather ask this question about a technology rather than a language. That may just be my particular field, though.

eska• 2 years ago

Couldn’t you just ask „what are you curious to learn?“ directly, instead of beating around the bush with awkward questions like „what would you rate yourself?“ that people have to „game“? It’s just another way of asking „what’s your biggest weakness?“ with the obligatory „my perfectionism“ response. I just skip the fluff and communicate openly.

fsckboy2• 2 years ago

I just skip the fluff and communicate openly.

that doesn't mean people don't game the answers.

readthenotes1• 2 years ago

We used to ask people to rank themselves on the page-jones scale of expertise (detailed here https://medium.com/@helen.taner.nc/how-to-estimate-your-soft...)

Generally, people who ranked themselves as Masters were subject to the dunning Kruger effect (and/or arrogance)

milliams• 2 years ago

It can too easily go this way: https://www.youtube.com/watch?v=IkYUDQCYGHA

karmakaze• 2 years ago

"I'm an 11 on the Tufnel scale."

kevindamm2• 2 years ago

I think the correct answer should involve a calibration, some reference to what you interpret the value to mean.

I wouldn't ask this question to a job candidate, but in any context where it might come up, if the response is "10 because I am at least 90%tile on a speed or error rate compared to other candidates" then that's acceptable to me.

It should make sense, though. If the response is "where 10 is having written a frequently referred to text on the entire subject I would score an 8," what does it mean.. Wrote 80% of a book? Wrote a book but it's referred to 4/5 as often as the best book? Neither of those are really practical interpretations.

As usual, it's the way an answer is given that can make more difference than the response per se. If ever asked this kind of "rank on 10 your skill" question then giving a reasonable benchmark for your choice can improve the clarity of your communication, at the very least.

platzhirsch• 2 years ago

I used to have this type of rating on my resume when I started my career. Over 10 years later, I can tell that today it wouldn't communicate anything.

At most you could convey that you're more proficient in language A than language B.

zzo38computer• 2 years ago

I would not ask such a question. If someone asked me such a question, then I might say: Due to the Dunning-Kruger effect, I don't know.

Never ask a candidate to rate themselves out of 10

Related Stories