We analyzed thousands of interviews on everything from language to code style

http://blog.interviewing.io/what-really-matters-in-technical-interviews-we-analyzed-thousands-of-interviews-on-everything-from-language-to-code-style-heres-what-we-found/

by emilong • 9 years ago

261 169 9 years ago

blog.interviewing.io

crobertsbmw• 9 years ago

None of the graphs are loading for me. It says "If the problem persists, open an issue at support.plot.ly" Unfortunately, I have to pay money to file reports...

sappapp• 9 years ago

Same here

emilong• 9 years ago

Looks like this has been fixed.

Jemaclus• 9 years ago

Furthermore, no matter what, poor technical ability seems highly correlated with poor communication ability – regardless of language, it’s relatively rare for candidates to perform well technically but not effectively communicate what they’re doing (or vice versa), largely (and fortunately) debunking the myth of the incoherent, fast-talking, awkward engineer.

My interpretation of this is that interviewees who can communicate clearly about code (whether they wrote it or not) correlate with high technical ability. Does this suggest that rather than having the interviewee write code on the spot, one could give them some new code they've never seen before and ask them to reason about it aloud for 30 minutes, then gauge their technical ability based on their ability to communicate clearly about the code?

In other words, could you replace live-coding with "here's some code, tell me about it"?

soulnothing• 9 years ago

This was the style of interview I ran. I usually picked snippets from an open source project. I usually grabbed pull requests that fixed bugs.

Have them delve through it. Reason aloud, ask questions about it. See how they gain context, and how they troubleshoot. The biggest down side was they sometimes need to reference external projects, or documentation. I would often have to search on my work laptop and then let them should surf the docs.

qntmfred• 9 years ago

this is also a prominent technique I've used in the hundreds of interviews I've conducted.

vonmoltke• 9 years ago

My previous company did this. I have seen three variants over my time: explain what this code does; explain why this code is wrong and how to fix it; explain why this code is inefficient and how to optimize it.

DoofusOfDeath4• 9 years ago

explain why this code is inefficient and how to optimize it.

That sounds like a trick question. I've done a lot of code optimization. Here are a few generalizations I've drawn:

(a) You rarely know what part of a program is the bottleneck until you profile it on an appropriate workload.

(b) You often don't know why it's slow until you dig into performance metrics like cache-miss ratios, branch-mispredicts, etc. Often at the disassembly level.

b0rsuk3• 9 years ago

I think "find the bug in this code piece" questions are dumb! When you program, you don't get a helpful fairy (or Clippy for you Windows users) telling you the code is wrong and needs fixing. The bug manifests itself in production or when you try to execute the code. Bad output, exceptions or segfaults, slow operation, infinite loop. Then you have a vague idea what to look for, or at least what kind of input causes errors. Furthermore, if you're the author you should know what the code is trying to do, and how. I believe programmers in teams are usually told to fix their bugs, not fix someone else's bug ?

I'd rather see a question like "Please modify this piece of code to add X functionality."

lj35• 9 years ago

Those are my preferred types of interviews when I'm an interviewee and a technique I've used a few times on the other side of the table. It's so effective and efficient. I don't know why it isn't used more. Maybe it finally will be, now that there's some solid data supporting it.

lettergram• 9 years ago

That's actually how I conduct interviews. It's been immensely successful at weeding out candidates within 5 minutes; although most interviews are 45 - 60 minutes.

Basically, give them less than ten lines of code, ask them what it does, where are a couple bugs, ask what would you name the function, etc. Then we talk about how to improve it. I'd say, less than 20% of people I interview pass. It's actually amazingly low how many people can find a bug and communicate it.

Half the people don't even tell me what they are thinking. And no matter how many times I try to work with them, act like their buddy, or w.e. they just kind of shut down. They think in their head, don't work through the problem at all.

marcosdumay• 9 years ago

They think in their head

I'm left wondering why this would be a problem.

watwut4• 9 years ago

"Half the people don't even tell me what they are thinking."

Maybe they are introverted or simply need to think before they talk. Around 50% of people are introverted (less in usa). Many people are like that and simultaneously quite skilled. Moreover, some environments punish errors, so people who worked/studied there tend to be conditioned to think before talking.

"And no matter how many times I try to work with them, act like their buddy, or w.e. they just kind of shut down."

You are not buddies, you are interviewer about to decide whether they get hired. Many people shutting down might mean that they are not comfortable juggling "buddy" social role and expectations and "serious job interview" social expectations simultaneously.

sulam• 9 years ago

I'd suggest making it more than 10 lines of code. Similar to kids finding easter eggs -- you want a reasonable number of things out there to talk about. It sounds like you try to get there by keeping it simple, but I would posit that some people's brains may lock up on a single block of code and having it in context or more there gives them something else to look at. Our brains are very bad at having insights when focused on the thing you're trying to have an insight about.

Jemmeh5• 9 years ago

Just curious, if you've weeded them out at around 5 mins, why does it take 45-60 mins?

Also, could you possibly just do an initial phone/Skype interview to save time in that case?

kafkaesq2• 9 years ago

It's been immensely successful at weeding out candidates within 5 minutes

Indeed, it's amazing how quickly people reveal themselves in this way.

No whiteboards, no quiz shows, no grilling of any kind needed. Just: "tell me your story".

jimminy• 9 years ago

"They think in their head, don't work through the problem at all."

Those are independent clauses. Just because the individual is not talking out loud, should not be indicative that they aren't working through the problem. Personally, I am a very visual thinker and layout visual models in my head for problems with high (3-5d) dimensionality that would take longer to draw clearly in 2D space while explaining all the traversals I'm mentally making. I would then reduce a solution, and explain that, or I apologize for zoning out.

It became obvious to my office mates when I was working through and reducing a problem space, because I would "hang" on the thought. Sometimes I would be down, stuck in thought, for 15-30 seconds and according to them they could tell because my face loses emotion and my eyes flutter about.

paganel• 9 years ago

Half the people don't even tell me what they are thinking. And no matter how many times I try to work with them, act like their buddy, or w.e. they just kind of shut down. They think in their head, don't work through the problem at all.

As it happens most of my "aha!" code-related moments come when I'm either under the shower or when I'm washing the dishes. Very rarely did it happen for me to look at a piece code while in front of my computer and then immediately be stuck by what that code does or by some hidden bugs in it. It's all sort of mechanical, at least while I'm sitting at my desk. I'd say it would be very hard for people like me to reproduce that "aha!" moment during an interview, in front of some other people who expect me to have those moments of enlightenment right there and at that precise moment.

gxs• 9 years ago

That's an interesting approach, but I wonder if you're screening for people who can speak and code vs just people who can code.

I'm a product manager and I suspect I would do pretty well in this type of interview unless the snippet is especially complicated and/or esoteric.

In that case you might be screening for people who can talk but not actually code or people who can code but might not be the best communicator.

Interestingly enough, if youre optimizing for specific outcomes, you may not need to screen out the latter.

Engineers who struggle to communicate but are productive can be extremely successful given the right environment - including having someone on the team who knows who to work with various personalities and is technical enough.

Just some rambling, but it's worth pointing out given the scarcity of engineering talent. Not everyone is going to be Paul Bucheit, ace product manager and engineer all wrapped into one.

vellum• 9 years ago

Each moment of silence makes the interviewee feel more tense. Instead of staring at them, why not leave the room while they read through the code and collect their thoughts?

You could even have HR do it. When they arrive, HR hands them the code and instructions. Then, they read it for 15 minutes, until you walk in.

avip• 9 years ago

I'm absolutely not an "introvert" of any sort, but if you give me something I've never seen and ask me to reason about it, you'd better take a very wide step back, stfu, and wait for 10m - if you're to get any results from my slow brain.

notyourloops• 9 years ago

That's great! I really hope interviews like that become more common.

dando3• 9 years ago

When you put people under the pressure of not having a job, a time limit and worst, the pressure of having someone looking over you, bugging you, and judging you, it can be incredibly hard to think straight or say anything at all.

You can act buddy all you want, but people aren't stupid. They know if they take a few extra seconds too long, you're already thinking "this guy sucks"

jghn• 9 years ago

I used to take "relatively rare" a step further and viewed it as a myth. I'd get into debates with coworkers that all one needs to do is have a conversation with a candidate to fully judge their abilities. No need for coding tests or the typical questions. Unfortunately after we brought in a ton of candidates there were a few people who fell into this bucket. I have no idea how it happened but they couldn't code themselves out of a wet paper bag when it came time to do so even they kicked ass in technical conversations. Sadly it became a bit of an "I told you so" moment where the outliers are made up to be more frequent than they actually are.

I like your idea though.

MichaelGG• 9 years ago

Had a similar experience with a remote sysadmin. Interviewing, he seemed to understand everything. He explained how he'd setup networks, details on using SSH for bastion hosts, etc. Seemed to really know stuff.

After hiring him, he was unable to even SSH to a box. As in he didn't know how to give us his private key, or how to configure his SSH client.

Maybe one guy takes interviews under other people's names then lets them mess up the work.

DoofusOfDeath4• 9 years ago

That happened to me once, during a phone interview with Google.

My brain simply froze during the live coding test. I looked like an idiot, even though the coding challenge was one I could have normally handled in my sleep.

My guess is that it was due to anxiety about doing well on the interview. But to this day I'm gun-shy regarding live coding tests.

cookiecaper2• 9 years ago

It shouldn't take much demonstration to see that they are basically competent, and there shouldn't really be a difference between asking them to loosely state the commands they would issue and the process they would take and watching them do it.

People normally perform better by verbalizing the process than they would by sitting at their computer and performing it, since a lot of the concern in an interview can be whether you're getting your code/commands syntax-perfect or not.

The only way I can really see this going badly would be if the interviewer was only asking generalized questions that someone like a non-coder HN reader would be able to pick up from the zeitgeist without actually having ever implemented. Instead of asking about trends or fads, it's probably better to ask detailed questions about implementation processes.

If they really have internalized all the applicable concepts but can't express them in computer language, well, it shouldn't be hard for them to get up to speed on the syntactical fineries. Make sure you're testing for grokkiness of the core concepts needed, not how well they're following the trends or the news.

Personally I use a very small code test and put the rest of the energy into a detailed conversation.

Jemmeh5• 9 years ago

Something I've learned from making Youtube tutorials is that you need to have a deeper knowledge to actually teach something. The point at which I study a topic and have it "click in my head" comes way before I can actually manage to "explain it back clearly". As I'm writing the script for the tutorial I often realize there's a gap in my understanding that I have to go back and work on.

The old saying "If you can't explain it simply, you don't understand it well enough" applies here I think. :)

benjaminjackman• 9 years ago

That has been my experience as well, one level abstracted. Back when I was a position when I last had to organize teaching employees technical domain-specific skills, the best way we found for them to learn was for them to teach others, this was accomplished by having one generation of employees train the next and then reviewing the newly trained employees.

Then for the next round of hiring, advance the employees up the education pyramid. The trainees become trainers, and the previous trainers mentor the new trainers by reviewing the newly trained employees.[1] It was one of the things I think we did we really well and that I am really proud of (even though the business went belly-up).

When training is done is this pattern, hopefully a virtuous cycle of education is established throughout your organization.

1: Note that this was at a quickly growing company, with recruits that had little to no relevant actual know-how to do what we did.

cr0sh• 9 years ago

I've never experienced an interview like this, but it sounds like it would be both challenging and fun - especially if it was done in a language (or pseudo-language) that you'd never seen before (then again, they could lob BF at you or something eso like that!).

Based on other responses to your post, it's heartening to see that others do this kind of interview, so maybe in the future I might encounter it.

jondubois• 9 years ago

Some companies just love fast-talking engineers because they sound like they know what they're doing.

Some engineers spend a lot of effort trying to project a specific image of themselves at the expense of bringing up the morale of others in their team.

On a few occasions, I've seen an engineer tear apart the work of another engineer at a team meeting in order to make themselves look good and more senior in front of their boss - Often, the critique is delivered with such vigour that it sounds rehearsed. I've seen this behaviour in startups mostly.

desdiv2• 9 years ago

When I'm nervous, I don't communicate well.

When I'm nervous, I don't code well.

I don't know about other people, but for me personally, when you measure me under a high-stress situation, all you are measuring is just my level of nervousness.

poikniok• 9 years ago

I must ask, did you get a university degree and if so how? Personally I have found interviews to be trivial compared to exams in school, so while I do get quite nervous for some of my math examinations I have never had this experience in interviews.

javabean224• 9 years ago

That's A Bingo. That's how false negatives are made.

maxxxxx4• 9 years ago

That's what I do. If I can have a meaningful conversation with someone about stuff he did or stuff I am describing most likely they will do well. If the conversation gets very vague or confusing most likely things won't work out.

11thEarlOfMar3• 9 years ago

We do it in reverse. In the first 2 hour interview, candidates learn our tech from 2 or 3 engineers. We then send them home with a set of coding tasks that are all relevant to what we do. As in functions of utilities that we actually would need to write, as opposed to puzzles. They choose one of 6 tasks, write ~100 lines of code, then come back for the next 2 hour interview and lead a code review of the code they wrote. This approach allows them to take the same time and care they would if working for us, rather than putting them in an unnatural state of coding at a white board in front of one to n scowling veterans.

Engineers who are capable can clearly explain their coding choices and their style is evident. If they've borrowed too liberally or had someone else write it for them, it becomes apparent pretty quickly because they can't clearly explain the code they supposedly wrote.

This is a fairly new approach for us, but so far, the candidates have appreciated it and the team reports that this approach gives them a good sense of the candidates ability.

lj35• 9 years ago

So, based on the new information presented in the article that people who can talk clearly about code can also write code, are you likely to change your interview practices to omit the take home task and ask the candidate to talk about code that somebody else wrote? It would be faster, easier and according to the data in TFA just as accurate, yes?

cakedoggie3• 9 years ago

Who doesn't love homework, especially as an adult.

No thanks.

graphememes6• 9 years ago

I do this frequently and I utilize their resume to power the code.

Someone says "I am a 9/10 in JS" I give them something that uses the animation frame api, the dom api, and a few more javascript APIs that I personally am not fully comfortable saying that I am a 9/10 on (been working with JS for over ten years) and see how they decipher what it is doing, what jumps out, and how they would review it / give feedback.

9/10 times they don't actually understand Javascript.

kasey_junk• 9 years ago

Its fairly easy to see how this correlation breaks down. It is almost a truism that someone who is good at communicating with a wide range of people, is likely to make that wide range of people think they are more technically competent than people who aren't good at communicating with a wide range of people. Especially given the virtually random nature who you will communicate with during any given interview cycle.

But, its quite possible for people who are not naturally able to communicate with a wide range of people to be able to communicate with a smaller subset (say people they are more comfortable with, or people working on similar problems to them).

The more interesting question is, could those people who communicate less broadly, in a situation where they do communicate well, perform technically at the same level (or better) as people who are more natural communicators?

Another way to say this is, evaluating interviews is fairly prone to bias towards people who interview well. As someone who is hiring I'd be much more enthused about their results if it correlated to job performance, not hiring results. Its almost gospel at this point that there are people in software who interview well who do not make good hires, but the methodology of this article doesn't talk about this at all.

All told, nothing about this research is persuasive to me on my opinion of "interviews are largely worthless in determining who will be a good hire".

amalcon• 9 years ago

The version of this where that code is clearly unsuitable for its stated purpose is my favorite coding interview question.

The unsuitability should be extremely obvious, no "spot the missing semicolon". Maybe it has a severe performance problem, maybe an obvious security flaw, maybe a logic error. The point is to prompt a discussion of better alternative ways to do what the code is trying to do.

leeny2• 9 years ago

Graphs have been fixed! Sorry about that, HNers.

default-kramer• 9 years ago

You might want to think about upping the contrast and/or font weight too. It's nearly unreadable for me and I have pretty good vision.

Edit - it looks way better if I disable Open Sans. It might just be a font issue with Chrome or Windows.

georgiev• 9 years ago

Order has been restored!

Terr_• 9 years ago

There's something awry about the "fewer code execution errors" graph: I count an odd number of columns when they ought to be in pairs.

Clicking on the graph to go to plot.ly and viewing its "data" tab, it looks like there's a blank X value for:

         text                                                                           y                   x
    0    bucketed_success_rate: -0.02<br>pct: 0.1<br>`Interviewer Would Hire`: False    0.0987654320987654

skylark• 9 years ago

Overall, the data lines up with my own intuition, but I thought I might throw my own interpretation into the ring.

One of the biggest keys to doing well on technical interviews is to completely separate the problem solving from the coding. The strongest interviewers will discuss the problem and solve it at an abstract level using diagrams. Once satisfied with the solution, they'll code the entire thing making few mistakes.

I think this is what drives most of those metrics. Strong interviewers submit code later, and have a higher chance of it being correct because they take the time to problem solve upfront. Their thought process seems more clear because there isn't the iteration of "this should work, let me code it, oh no wait, that's wrong, let me erase this now..."

UK-AL• 9 years ago

Completely devalues working with prototyping though.

omot• 9 years ago

I think this applies to actual software management too. I don't see people with the most code commits as the ones who gets promoted. The ones promoted are usually the ones who really scope out the problem. Once they're sure of a complete solution they code out their plans on a steady pace.

I think Einstein's the one that said: "If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions."

ozim• 9 years ago

Yes guys that do have a lot of commits (and those who don't think through the problem long enough) end up with commits like:

"Fix" "Real fix" "Fix the fix because of fix"

Then you know you don't want those on your team. It really bites on tight deadlines when you have to put something to production but poor dude needs to push that really, really last fix.

alexchantavy• 9 years ago

The strongest interviewers will discuss the problem and solve it at an abstract level using diagrams. Once satisfied with the solution, they'll code the entire thing making few mistakes.

I agree. In general I'm mediocre at coding interviews, but I do best when I have a chance to whiteboard and draw the problem. On the other hand I do absolutely terribly on phone screens with a shared document and no where for the interviewer to see my drawing.

NumberCruncher2• 9 years ago

> An average, successful candidates interviewing in Python define 3.29 functions, whereas unsuccessful candidates define 2.71 functions. This finding is statistically significant.

The "average" is too sensitive to outliers and should not be used for such a comparison...

[Edit] Being bored I calculated the Kolmogorov-Smirnov statistic based on the chart. It is between 10%-10.5%. The number of defined funtions seems to be a significant but weak indicator.

leeny2• 9 years ago

We actually did the KS test as well, but we omitted the results for narrative clarity. Our KS test statistic is also < 0.05.

NumberCruncher2• 9 years ago

I would recommend you to put also the more sophisticated results (like K-S tests etc.) in the blogpost. For people interested in technical interviews and charts they may be more important than narrative clarity.

hexane360• 9 years ago

Soon to be a clickbait article: "7 simple ways to hack your interviews. #1: Define 10,000 functions. That makes you a good programmer"

dkersten4• 9 years ago

4 functions will do. You just need to be above average. 5, just to be safe.

snissn• 9 years ago

What is the blue bar on top of the page?

FLUX-YOU2• 9 years ago

pace.js or something similar to it

bovermyer• 9 years ago

There's just one problem: this assumes that code challenges are present in all (engineering) interviews.

pklausler• 9 years ago

I like to ask one question that probes basic analytic ability and a second question that probes programming aptitude. Generally, the first question either takes 3-5 minutes or the whole 45-50. It's usually a problem of the form "write a predicate (Boolean-valued) expression that is true when..." applied to something simple, and it's a basic test of being able to use relations and logical operations to characterize a situation. It's depressing how many great-looking candidates with awesome degrees, resumes, and phone-screen performances get stuck trying to describe how to tell whether two calendar entries (just start/end times) conflict with each other.

janwillemb• 9 years ago

The title is quite clickbaity: "We analyzed thousands of technical interviews on everything from language to code style. Here’s what we found."

What's wrong with this, I think, is that a (journalistic) title should give an ultra-condensed summary of the main point of the article. This title suggests that the authors gathered a lot of data but didn't find much.

(I find myself quite intrigued by clickbaity titles somehow, sorry for that.)

FLUX-YOU2• 9 years ago

If you filter the interviews to only interviewees who:

- liked the person

- rated the questions 3 or 4 stars

- gave the interviewer 3 or 4 stars for being helpful

Do the trends still hold?

How are those trends compared to only looking at interviews with:

- disliked the person

- rated the questions 1 or 2 stars

- gave the interviewer 1 or 2 stars for being helpful

javabean224• 9 years ago

Here is a hint. If you aren't a fresh graduate avoid companies making you code in a browser under a time pressure.

thenanyu• 9 years ago

Judging by this graph in the article, and somewhat counter to the claim in the article: https://plot.ly/~aline_interviewingio/952.png?share_key=Htks...

Looks to me that interview length is correlated with success rate. If your interviewer stops before 60 minutes, there's a bias towards successful interviews. It seems like the interviews that end up being "no"s tend to get hard-stopped right at the 1-hour mark.

zebraflask2• 9 years ago

A lot of this article rings true with my experience. And I agree with the comments dinging live coding tests - those are the worst. I can't code effectively unless I'm calm and can concentrate, and these things are almost designed to get you off-balance.

Even more galling when you have a healthy GitHub portfolio that they refuse to even look at in favor of a quiz (this has happened recently).

graphememes6• 9 years ago

To note a few things regarding the interview and hiring process (I do a lot of it, have done about 20 in the past two days), the reason for the quiz and not using your portfolio (unless its extremely outstanding and you are fine doing a live coding questionnaire) is so that we can judge all the candidates equally and fairly.

It gives a common baseline to judge. Each candidate does the same thing and we have a good idea of what we are looking for, the rest of it tells us how you think, how you approach your work, organize your work, and best of all? You can compare that to how others do it.

Now, not to say that's everyone, that is what we use it for.

---

Before, as an Engineer / Manager, I hated doing "live coding" tests when it wasn't relevant. For example doing "algorithms" or "palindrome" or "sliding window dns" or "O(n)" examples when you're doing front-end or a management position screams to me that the people doing the interviewing don't know what they actually want.

Instead quizzes or live coding that are relevant like "tell me how to access all the elements in this particular element and traverse the children to apply some styling" is much more relevant and will show me the thought process, their ability to retain information, and their recall. It also shows communication ability when they get stuck and ask for help or use me as a sounding board.

It's not always about your implementation, but how you handle the situation and communicate

zebraflask2• 9 years ago

I agree with you about irrelevant questions, and out of the dozen or so coding quizzes I've done over the past several years, mostly with start up companies, those are unfortunately usually what I got. One company was advertising for a front end role and, sensibly enough, asked front end-related questions - that was the exception.

And the thought that quizzes provide a fair point of comparison comes across to me as putting process ahead of substance. Interviewing isn't meant to be fair to everyone - only one person gets the job, after all - so it's not like handing out cookies and stickers in middle school. It's meant to see, in part, whether the person is capable of generating working code. If you have a person who can provide samples to prove it, requiring an artificial quiz really is a slap in the face to a lot of good candidates.

sroussey• 9 years ago

A lot of tech interviewers fall back on college experience of the pop quiz. Or what made them feel smart (which is why standardizing such things across managers is important). Really smart engineers that want to relive the A grades from school will try and be clever and undo such standards.

That all said, the correlation between great whiteboard / coding tricks and them being a successful part of a team isn't so great. It's not their job to code short programs under pressure of being watched. It's a proxy. There are better ways.

Joboman555• 9 years ago

These people are not particularly good at interpreting statistics.

jaclaz• 9 years ago

These people are not particularly good at interpreting statistics.

Maybe that is a tad bit too harsh, but surely the use of "big difference" and of "significant" seems like not being justified by the actual data:

On average, successful interviews had final interview code that was on average 2045 characters long, whereas unsuccessful ones were, on average, 1760 characters long. That’s a big difference! This finding is statistically significant and probably not very surprising.

An average of 1760 vs an average of 2045 indicates a general average-average of around 1900 lines, so that would be 1900+/- 7%, and anyway the difference in ranges is so little that anything could cause it.

To have more or less 200 characters, merely calling variables a, b, c, etc. vs. FirstUserChoice, DefaultArrayIndexingField, you know what I mean, would be enough.

Same goes for:

On average, successful candidates’ code ran successfully (didn’t result in errors) 64% of the time, whereas unsuccessful candidates’ attempts to compile code ran successfully 60% of the time, and this difference was indeed significant.

As I see it 60% or 64% as an average are almost exactly the same number, and bear very little significance. Maybe it is just me missing some sensibility ...

Radim• 9 years ago

"Poor technical ability seems highly correlated with poor communication ability"

Yep. Articulate attention is the name of the game (where "communication ability" sounds a little nebulous).

If you can't organize your thoughts, bring them to the forefront of your attention, name them, you're likely bad at handling abstractions. And abstractions are at the core of "technical ability" -- the ability to name things, find the appropriate abstraction boundaries, chisel structure out of chaos.

Articulate speech is the greatest human invention for a reason.

Testing for that (plus conscientiousness -- can you pay attention to details and get shit done?) during interviews makes perfect sense.

tptacek• 9 years ago

One of the language results doesn't make sense. It claims that it matters, significantly, if you solve interview problems in Java when the hiring company is a Java shop, but not when the hiring company is a C++ shop.

But that's reversed. It is in fact fairly difficult for a high-level language programmer to pick up C++, and facility with C++ (or at least C) is a common, accepted goal for C++ hiring shops. A C++ shop that hired candidates without regard for their aptitude at C++ would have real problems.

blacksmythe• 9 years ago

The data suggests that it is easier for a C++ programmer to get make a good impression in a non-C++ shop, than at a C++ shop where they likely to test you on C++ edge cases.

Interesting that this effect does not show up for Java programmers.

dkersten4• 9 years ago

I think this makes sense, because C++ is seen as this difficult beast. Non-C++ shops will be impressed that you know any C++, while a C++ shop will want to dig much deeper and make sure you are sufficiently advanced at the language to do the job. Non-C++ shops won't dig as deeply (either because it doesn't seem relevant -- you're not going to be using C++ anyway, so you just need to show programming ability and not C++ mastery -- or the interviewers don't know C++ well enough themselves, since they don't use it in their jobs) and won't ask you about in-depth edge case language features (because why would they if they don't use the language), but in a C++ shop, they will care about all of these things and have people who use C++ and can ask in-depth questions.

Why isn't it the same in Java? I'm not sure, perhaps its because Java has less gotchas as a language (certainly a lot less undefined behaviour and weird memory-related gotchas, no templates, no multiple inheritance etc) and C++ has this "its a difficult language" prestige which Java doesn't have.

SonOfLilit• 9 years ago

They give P ~= 0.36 to this claim, in an article that I think makes more than 1/0.36 ~= 28 claims, so I would suspect that maybe that's just a coincidence in their data?

We analyzed thousands of interviews on everything from language to code style

Related Stories