More whiteboard coding won’t build you a better team

http://blog.hackerhires.com/2012/09/more-whiteboard-coding-wont-build-you-a-better-team/

by oohaba • 14 years ago

59 42 14 years ago

blog.hackerhires.com

dollarpizza2• 14 years ago

Heresy!

dollarpizza2• 14 years ago

Umm, polite sarcasm, anyone?

The fact of the matter is, the hypothesis that "whiteboard skills == mondo real life developer skillz" is pretty much an unspoken, and utterly unquestioned article of faith out in vast stretches of startupland.

paulgb2• 14 years ago

The best interview experiences I've had have started with whiteboard coding but turned into a discussion about the code, assumptions, requirements, theoretical possibilities vs. practicalities, etc.

The worst have been ones where it was clear the interviewer had pulled questions out of the company's interview playbook and didn't have the depth of understanding to discuss them.

jpdoctor• 14 years ago

> The worst have been ones where it was clear the interviewer had pulled questions out of the company's interview playbook and didn't have the depth of understanding to discuss them.

I'd argue that also belongs under the "best" list, because it told you exactly the level of the people you'd be working for/with.

paulgb2• 14 years ago

Very true.

In one such interview, I was asked for a way to implement Rock Paper Scissors in OOP without using a conditional. I came up with a neat way to do it with modulo operations and array lookups, but that wasn't The Right Way. So I came up with a way to do it with exceptions, but it wasn't The Right Way. Finally I came up with the solution the interviewer had in mind, which used Java's method overloading (my background is not Java).

I'm not sure if I "passed" the interview, but it certainly didn't make me want to.

gordonguthrie• 14 years ago

The key thing is that 'you don't want to hire people to write code'. Code is inventory, code has a cost, code-not-written is what you want (as much as possible).

So you need to hire a developer who knows 'how not to code', 'when not to code', how to write as little code as possible (libraries, reuse and designing things that don't take much code, clean, concise, elegant, and well-structured).

These sort of developers are indeed the best to hire - but hard to find. And, as the article points out, is not what most technical interviews test for.

wtetzner• 14 years ago

Even better, a developer that can take an existing code base and shrink it.

chris_mahan• 14 years ago

you mean get rid of it. Or, failing that, replace it with 30 lines of python.

zeteo• 14 years ago

"Advanced whiteboard coding" does have its uses:

1. It checks that you still know the basics of what's required for a CS degree.

2. Quite possibly, it also checks your capacity to (re-)learn some things for preparation.

Both of these are good indications of your ability to adapt to a new technical environment.

griffindy• 14 years ago

I agree with the ability to learn part, but do you need the basics for what's required for a CS degree? I say this as someone who does not have a CS degree, but I also don't know that many places working directly with binary trees and linked lists, though I'll admit I may not have found them.

bmj• 14 years ago

I've been doing software development for over ten years without a CS degree. Yeah, I've never had a problem that linked list solved, but I suspect having a CS degree might have better prepared me to solve certain classes of problems. After 10 years, I have a deep tool chest of patterns (and anti-patterns), but early in the game, having a better algorithmic eye would have certainly been beneficial.

columbo• 14 years ago

That question, and the example in the link really threw me for a loop.

At first I understood the question as described, but then looking at the javascript code I suddenly became lost and frightened. It wasn't until I clicked "run" that I realized the code wasn't working "as is" and needed to be made to work. Then it was just a matter of adding cache to the function (at least that's what I did). I had first read the blog as "this an example of caching" not "this is an example of the starting point for the problem".

I still prefer to not have 1 "golden" question but a list of problems the interviewee can choose from. That way they can find something that matches with areas they might be more famalier with (oh crap, I haven't thought of Fibonacci in awhile so I'll take on this robot-battle-engine quiz)

On an unrelated side note I had some fun making quiz #27 pass assertions but fail the meaning of the test:

      for ( var i = 0; i < array.length; i++ ) {
        var d = [];
        d.fn = fn;
        d.fn(i);
      }

munin2• 14 years ago

semi-serious devils advocacy. I don't know if the below is an argument that is correct but I hope for it to be proven wrong. I am pretty sure that accurately assessing individual capacity/potential is really, really, really hard.

"Google is a very successful technology company. While some of the problems they solve are very challenging and require innovative thinking and research, they have employees with decades of research experience and long publication track records to crack those.

New and seasoned developers will almost certainly have plug-n-chug everyday responsibilities, yet their interview process is still a fearsome battery of computer science questions. Their hiring also produces what is regarded as the most skilled CS workforce in the world and the market rewards them richly for it.

Why shouldn't we emulate this? Isn't it working? Aren't advocates of a disruption of this system essentially saying "Go ahead, relax your standards, what's the worst that could happen"? Is it taking your company 6 months to find a decent employee? Maybe that's about how long it should take because there just aren't that many decent employees."

randometc3• 14 years ago

The question is: what is your tolerance for (a) false negatives and (b) false positives?

Rephrasing (a): who are we passing on, are they in fact good potential employees, and what qualities do they have that our new hires might not?

Rephrasing (b): who have we hired that, despite whiteboard coding proficiency, isn't cutting it? And why?

In my experience whiteboard coding sessions feel adversarial, like a trap or a test, and you're basically waiting for the candidate to have an epiphany moment. Either they know the problem, or they don't but they get lucky and hit the "right" answer.

Doing reviews of code samples (or even pair programming against a new problem) puts interviewer and candidate both on the same side of the problem and feels more collaborative - I want to believe this gets you a better sense of what someone would be like to work with, and fewer false positives and negatives.

If anyone has evidence or studies either way I'd love to read them.

munin2• 14 years ago

I think this is the correct question to ask.

in my experience, false positives in early ventures (research projects, startups, project partnerships, whatever) are the kiss of death. If you need a group of people to do work at high capacity, bringing someone who does not 'fit' in when the group is young will probably destroy it. existing high-capacity people will become angry, there is less fault tolerance in the quality of the work and the rate at which it has to be done, etc.

it is usually the case that it is better to have 2 workers who click and are highly productive than 5 workers where 1 is not productive. so in the early days, failing closed seems very valuable.

when you are larger, there could be a higher tolerance. you could afford to hire someone and fire them a few months later if they do not work out, because the work that they are doing for you immediately is not as critical. however, there is also a certain culture that comes along with this and if you want to avoid that culture it seems like you still want to 'think lean' and part of that thinking could be to keep the amount of productive people high.

and it might also not go well with your current group members to say "yes, that hire was bad and we'll terminate them next week" for the fifth, sixth time in a quarter ...

danielweber2• 14 years ago

Rephrasing (b): who have we hired that, despite whiteboard coding proficiency, isn't cutting it? And why?

Has someone suggested whiteboard proficiency is sufficient by itself?

The whiteboard is meant to filter out people who suck. "Add a cache to a JavaScript function" is a dumb question for reasons many others have pointed out. Joel's classic article[1] recommends these as examples:

• Write a function that determines if a string starts with an upper-case letter A-Z

• Write a function that determines the area of a circle given the radius

• Add up all the values in an array

Those are all trivial. You should be able to come up with another half-dozen if you sit and think for a bit. The language doesn't matter, and the syntax shouldn't matter (as compared to the semantics).

I hear lots of scare-stories of "great coders who choke on interviews," but I find it very hard to believe any kind of "great coder" can't add up the values in an array. I might ask "how do you find the median value in the array" just to see what they come up with. There isn't a right answer, but there are lots wrong answers.

[1] http://www.joelonsoftware.com/articles/GuerrillaInterviewing...

groby_b3• 14 years ago

It is adversarial, because it's a gating mechanism.

Sure, I'll be happy to discuss deep issues with you - after you've shown me that you're proficient in basic CS. It used to be that you could screen that over the phone, but now that you can look up the answers on your phone, that's kind of hard :)

bduerst2• 14 years ago

Anecdotal, but I interviewed for a position where they gave me a tab deliminated data set, and said, "Show us what you can do with this in X language". I had two days.

It was one of my favorite interviews.

MisterBastahrd• 14 years ago

Google hires the best because they have the capacity to pay the best. If you're hiring a mid-level developer in Texas and can only pay 70K a year, don't be surprised if your candidates have a hard time working out esoteric brain teasers.

j_baker• 14 years ago

I think this argument is simple, beautiful, and wrong. The assumption this seems to be built upon is that Google has developed a great hiring system that is doing well for us. I think that's a bit simplistic.

The reality is that (like everything at Google), the hiring process is constantly evolving and changing. Google puts considerable effort into improving its hiring process.

So, my counterargument is this: If Google is tweaking its hiring process and trying to make it better, why shouldn't you? Don't think of it as relaxing your standards, because it isn't. Think of it as changing your standards to better suit the kind of people you want to hire.

ianstallings• 14 years ago

I don't care about these tests. I have a very simple interview process - I ask you what you've done and we talk through it. I may even ask you about how you've solved some problems involved with whatever you were working on. I then jokingly ask you about an inner join or a fib sequence and laugh after a few seconds.

Maybe it's because my I come from the family business of construction years ago, but I believe that it should be easy to hire someone and get them involved quickly. Then we can find out if they are worth it or not. Anything up until that point is talk, and talk is cheap. If they don't past muster eventually I get rid of them. Explaining this up front makes it even easier. I'll hire you quick, but if you don't do what I need you're out. It's not personal.

loeschg• 14 years ago

Man... I don't know what I'd do if an interviewer presented that situation up front and very directly. "We'll hire you fast, but we won't hesitate to let you go." I wouldn't be able to stop thinking of the bad news bears scenario where I quit my job, get hired, and fired at the new one within a couple months. I guess it depends how awesome the job is that I'm applying for and how (not) awesome the job is that I'm leaving.

untog2• 14 years ago

Try being on the country on a work visa. Makes that situation 100x more terrifying.

kylemaxwell• 14 years ago

"Competant [sic] developers, even those who don’t specialize in JS, should be able to answer this question in 5-6 minutes. Therefore, this is a quick filter."

This is a false statement. I have programmed for years, but never in Javascript, so I have no clue what "add a cache to a Javascript function" even means. I can quite easily write code "that determines if a number is prime", but the assumption that everybody knows JS even if they don't "specialize" in it is problematic from my PoV.

cgag• 14 years ago

I think it just means add memoization, I don't think there's anything javascript specific about it.

thirdhaf• 14 years ago

If you want to implement it in a general way you still need to know how javascript supports closures though.

seabee• 14 years ago

The question actually has nothing to do with JavaScript specifically, it's not much more than 'do you know what memoization is' if I understood it correctly.

gscott• 14 years ago

I wonder if H1B hires have to go through this process. Does anyone go to India and ask people if they know how to cache an item in Javascript?

funkaster• 14 years ago

I've been on both sides of the fence: being interviewed and interviewing. The times I've been interviewed I felt (most of the times) that either the person questioning me had no complete understand of what he/she was asking --it could be possible he/she was a junior engineer-- or it was a tricky algorithm question. The thing is that not everyone thinks or plans the same way. Analytical thinking is different for everybody. Most of the time I like to solve problems first either by brute force or the simplest solution and then try to improve it, but the author of the post also got a very good point: "why".

When asked questions without a context, you have no motivation and they seem weird. If you want to know if I know what a b-tree or a hash-table is, or what is database normalization, just ask me to explain you what it is. I also, wouldn't expect a candidate to implement one of those things right out of their head... that's just crazy. I would expect for him to be able to use them or at least how they work. If you need to see what I've done or how I perform against a particular issue your company might be having, code review of already code-reviewed code is a great, great way to test a candidate.

Github profiles are also a good indicator: it shows what you want, want motivates you, it can show real-world code that you have written.

Just talking of your experiences and how you've solved problems is also a good thing.

dorkrawk• 14 years ago

I've always thought that a good programmer job interview challenge would be to hand the candidate a laptop (preferably with their OS of choice) and have them build something simple but new. For example: "Have you ever built a web scraper in Ruby? No? Ok, Ruby's installed on this computer. Here's a CLI, a text editor, and a browser with Google open, feel free to make use of existing language resources, build me something that pulls some text off of www.example.com"

I think it's good to see how people learn new things and to see if the candidate looks for existing libraries rather than writing everything from scratch. You'd need to be able to give the candidate some time but I think it would be a valuable exercise.

groby_b3• 14 years ago

We used to do that at my previous company. We'd actually have several candidates come in together, have them form an ad-hoc team, and spend 6 hours on solving a real-world (toy) problem. With one existing team member on that test team, and a rotating team of observers.

It's a great method to hire, but it's expensive - you better make sure that the people you invite in are pre-checked quite well.

It definitely was the most fun I ever had interviewing.

skyebook• 14 years ago

Asking how someone would go about solving a problem is a good question to ask but simply can't replace making sure the candidate can write code. Imagine hiring script-kiddies instead of actual hackers.

As an aside, did anyone else see the irony in the word "competent" being misspelled?

captaintacos• 14 years ago

Yep. We all did. Hence the "[sic]" used by some commenters in here when quoting the author -> "Competant [sic] developers, even those..."

On the other hand. What would be the difference between a script-kiddie and an "actual hacker"? The years of experience? Or the purpose for what the code will be used for?

More whiteboard coding won’t build you a better team

Related Stories