Hiring Great Developers: The Pre-Screening

http://www.small-improvements.com/recruiting/hiring-smart-developers--pre-screening

by pufuwozu • 16 years ago

51 46 16 years ago

bugsy4• 16 years ago

A fine article for hiring adequate or average developers.

You can't really hire great developers using any process involving resumes and ads. Hiring great developers involves recruitment. You already know you would hire the developer before you even contact them. The entire issue is can you offer them enough to lure them from where they are now.

variety• 16 years ago

Or, it should be seen more as a tool for filtering people out rather than for selecting people in. And if they are used, these tests should be (1) done at the very earliest stages in the process and (2) be conducted in a manner that's as time-efficient as humanely possible.

sp4rki• 16 years ago

And just to be sure, we disregarded the results from the screening anyway, and based our hiring decisions only on the three onsite-interviews.

Am I missing something here?

chrisaycock4• 16 years ago

The screening process he's describing is just to throw applications away, not to actually accept anyone.

bigiain• 16 years ago

I suspect it means passing the screening gets you through to the onsite-interviews, but the subsequent decision about hiring is purely based on the interviews, not on any results or rankings from the screenings.

huhtenberg3• 16 years ago

We tried it and while it does help to filter out completely clueless people, however cheating is a real problem. This clearly depends on the industry and specifics of the position, but out of three people that hired, two clearly cheated on the remote coding test. This was for a C position in embedded context, and one would think that no one in their sane mind would cheat, and still many people did.

So, yeah, good idea, but use with caution as YMMV.

chrisaycock4• 16 years ago

> out of three people that hired, two clearly cheated on the remote coding test

What screening did you do onsite?

huhtenberg3• 16 years ago

No, all from other cities. Sent the task over email and wanted to get a solution back in 8 hours or less.

dblumenfeld• 16 years ago

We tried it and cheating happens, but for us it was bearable. We were running candidates through Codility as the very first level of screening (3 programming problems to be solved from home), then through 2 or 3 interviews. People were cheating on a test once in a while, but not badly enough to spoil the whole filtering process. At some point we started sending out a welcome e-mail before the screening session to make it clear that cheating is nothing more than a recipe for a seriously bad interview, that you really do not want to have. Helped somewhat, but still somebody cheats once in a while. I guess such is life.

tav• 16 years ago

Here's a much better approach: use IRC.

I've successfully recruited dozens of exceptional individuals through IRC and am surprised that it's not often used for recruitment. Networks like freenode are filled with amazing individuals who are quite amenable to interesting startups. Even with my first company back in '99, when I didn't even know how to code, I found that the developers hired through IRC performed magnitudes better than the ones who came through more traditional hiring processes.

Just hang out for a few months in the IRC channels of open source projects that you find relevant; get to know the various individuals; once you find someone that you believe you can work well with, engage them about joining your venture. Easy, effective and the social bond even lasts for years afterwards.

trotsky• 16 years ago

But I am still surprised at how few companies do this

Because it flags you as a mill, uninterested in the applicant and likely a firm that treats development as a commodity. I'd be willing to bet that the 30% who didn't bother to respond to the challenges included some of your best applicants.

You'd never hire an accountant by insisting they prepare a quarterly for you before you ever met them and you wouldn't make prospective truck drivers arbitrarily drive hundreds of miles to an interview.

If you want to screen for the absolute basics of rote capability that's what those various certified professional type credentials are for. But just like your little tests, you'll be pre-screening out the best as well as the worst.

praptak2• 16 years ago

"Because it flags you as a mill, uninterested in the applicant and likely a firm that treats development as a commodity."

My impression is exactly the opposite. It flags them as a company that cares about technical competence.

"If you want to screen for the absolute basics of rote capability that's what those various certified professional type credentials are for."

Their test screen for more than that. And of course those professional certifications are useless if you go beyond the very basic level, just like "CV-years of experience" are useless for determining competence.

smikhanov• 16 years ago

It flags them as a company that cares about technical competence

I bet the tasks being sent out are most probably something in the vein of parsing a CSV file or (in case of Atlassian) writing a simple JSP or trivial XSLT or what have you. Not the best check for competence.

yoak• 16 years ago

Most of the most talented won't have those credentials and, in my opinion unlike getting sample problems, being asked for the credentials is such a red flag that the best won't apply.

perfragemann3• 16 years ago

I tried to address most concerns on my post. But I missed your comment about the accountant and the truck driver. I'd never ask a truck driver to travel hundreds of miles.

In fact, bringing in people to an interview right away without at least calling them IS like asking the truckie to drive a long distance just for the sake of it. I'd avoid that. But I would ask him to send me a photo of his current truck (if he claims he has one). I'd maybe also ask him to name five roads he'd avoid between his main destinations (say in SF, if he claims to be from SF). Someone who's actually a cab driver in NY would find that hard to answer. :-)

jchonphoenix4• 16 years ago

There's a major issue with your method. The best developers know they can get jobs wherever they want. What incentive do they have to bother with the extra work?

cageface• 16 years ago

When I see application processes like this I'm generally more interested because it signals to me that the company cares about hiring and that I'm more likely to have colleagues there that have been selected for ability rather than having exactly the right checkboxes on their CVs.

ajg1977• 16 years ago

Because, a) the 'best developers tend to love beating challenges so see it as a fun exercise to do one evening, and b) good developers want to work with good developers and seeing the screening process is a benefit.

Good developers of a certain experience level generally realize that if they apply for a job somewhere and waltz through the interview, or worse, the interview is a basically a pleasant chat with the tech director, then the chances are the hiring standards really aren't that great .

Oh and c) - if a developer does feel they're too good for your screening test, or such a thing is enough to put them off your company, then weeding them out at such an early stage may actually be a positive thing.

I do think it's crass to send out a problem sheet without at least verbally talking to the candidate briefly though. We currently do -

5-15 minute chat with a recruiter or PM who is savvy enough to filter out the people who really aren't experienced enough.

3 hour timed problem (debugging, coding, math, domain knowledge)

Technical phone screen

In-person interview (6-7 hours, all expenses paid).

bugsy4• 16 years ago

What you say is true for mediocre to average developers.

Before I had my own company, every place I worked whenever I would talk to clients or go to conferences I would get on the spot job offers. Principals would ask "Would you ever consider leaving XYZ?" I would say I'm not looking at the moment. They would always then hand me their cards and say "If you're ever looking to move, give me a call first."

In addition, I would get calls at home to my unlisted number because of articles I had written or patents I held.

When it was time to move on I would have dozens of open offers already to chose from. Why the hell would I or any other great developer bother dredging through monster hoping to find the 0.001% of ads that are both legitimate and for a place than isn't some web shop with slave hours and low wages? Answer is I wouldn't. Any company that only relies on job ads and resumes is not a place that great developers work for.

jchonphoenix4• 16 years ago

Sure, but keep in mind people have busy schedules.

Recently a startup I was somewhat interested in required a 4-5 hour coding project that was similar to the labs I do in school. I'm already bogged down in work interviewing, doing normal classes, writing a senior thesis, and preparing my research for publication.

I decided to forget about the company and look elsewhere, for I already had offers from great places like Google and Facebook and could always fall back.

I'm not saying its bad to screen candidates with interesting puzzles. I'm saying that you risk filtering out the better candidates that just don't have time to deal with puzzles from a company they don't know.

Additionally, keep in mind that I come from a competitive university in the academic world (although I figure competition is pretty bad at most universities). Thus, with my current 100 hour work weeks, I would just rather not bother with solving tedious puzzles.

jamn• 16 years ago

I second this. I just graduated and as I was looking for a job and I interviewed for a bunch of companies.

One of them asked that I solve some programming challenges. Sure, I like puzzles and I like being challenged. However, at the time I was already being challenged by my thesis work, problem sets, interviews... it's hard to agree to make a huge time commitment just to get the right to a first round interview.

Since other hot startups/ cool companies (Google, FB) don't require this type of activity, it makes sense to me to think that a lot of people simply won't have the time to apply because they're busy solving more interesting puzzles.

ig1• 16 years ago

Facebook are well known for their application puzzles,

http://www.facebook.com/careers/puzzles.php

And Google are notorious for having a long drawn out interviewing process.

praptak2• 16 years ago

I'd be glad to do the extra work (well yeah, within some limits, I won't code you a website for free :) ) The benefit to me as a developer is that this reduces the probability I'll work with incompetent people.

jchonphoenix4• 16 years ago

One major argument I'm hearing is that by giving these coding projects, competent developers should be more interested because this guarantees the people they work with will be competent.

My response is that you can find out the competence of an individual without wasting 5 hours of their time on a pre-interview puzzles. If they pass your phone screen and interview, you should be fairly confident that they are a competent programmer. I'm hard pressed to believe that someone who passes famously difficult interviews (for example Palantir) would find the puzzle anything more than busy work.

torial2• 16 years ago

At my previous job (which I enjoyed greatly until the president embezzled funds and ran off with a woman), I asked to do a programming task so that I could prove my skill and ability to do what they needed.

sciboy2• 16 years ago

I simply give them a call and ask them what the most interesting papers they have read over the last year are. Without any forewarning of course. People who are passionate about things know their stuff; and it's not from blog posts or forums that they obtain this information.

And I can tell you, if I learn something while on the phone with them then it's a guaranteed good hire.

perfragemann3• 16 years ago

I believe that with your approach you are guaranteed to only invite great applicants, so that's cool. But I worry that you also dismiss many great applicants who are not that snappy, have a bad day (minute, I guess!) I couldn't come up with a great paper. I read blogs, sure, but papers?

If I needed a developer who's great at answering tricky questions in front of an audience, your approach is perfect. But I'd also love a developer who solves hard questions with a bit more time to ponder the problem space. I worry I couldn't spot them in a such a tough phone interview. Oh well, next post! :-)

sciboy2• 16 years ago

No offence but discussing the most recent/interesting paper you have read shouldn't be a tricky question.

BweezySearch• 16 years ago

Very cool. I'm thinking it would be great if elance-type of sites, or even experts exchange, would have an area where a prospective employer can submit small coding questions to potential employees.

tomjen3• 16 years ago

> I am asking applicants to write some unit tests, to summarise the key point of a Google I/O video presentation that's actually fun to watch. I am asking them to learn enough about a web framework to be able to write a very basic program. I am asking them to review some really dumb code. And since I don't need code-monkeys but people with opinions, I am asking them for their opinion about my protoype: What would you improve?

Shit you are kidding right? Or do people just really value their time that low?

VladRussian• 16 years ago

it isn't about process, it is about people. As Steve Jobs put it : "A players hire A players, B players hire C players."

http://www.folklore.org/StoryView.py?project=Macintosh&s...

marcin• 16 years ago

Wrote it in the post comment, but will write again. The company I helped get traction (Codility) would address many issues people have with remote testing: manual work, some of the cheating and qualitative vs quantitative dilema. I may be biased, due to my former involvement, but it costs nothing to try it, so you can see for yourself: http://codility.com/demo/take-sample-test/

kranner• 16 years ago

Bunging in my own http://codeboff.in in the same spirit.

brlewis• 16 years ago

Upvoted you both for on-topic self-promotion. HN needs more of that.

bengl3rt• 16 years ago

An interesting side note to gather from this is that it confirms something I've long suspected: LinkedIn recommendations are useless. FWIW I have a couple good ones, but I don't think they've ever even helped me get an interview much less a job. I don't think anybody who actually makes hiring decisions has time to look at them; seems the only purpose they serve is to let people who casually cruise your profile know that you are competent.

xiaoma• 16 years ago

Interspersing all kinds of rebuttals to complaints others have made about the post inside the post itself looks really defensive. It disrupted my reading.

benaston• 16 years ago

Posts like these reenforce my dismay at the total and utter devaluation of the education system when dealing with recruiters for full-time positions (15 years minimum of full-time education, but you still need that one final test).

That said, I don't have a problem with these tests if the setter understands that they are not there to filter in or out the competent, but instead serve as a filter of convenience because dealing with 60 resumes is easier than 600. This seems an acceptable pragmatic approach.

When rubber hits the road a business needs to make money by being competitive. If having read a book on JUnit is your metric for hiring competitive employees, so be it. I can't help but wonder if there is a sizeable amount of latent competitive advantage in the complement of this set.

Hiring Great Developers: The Pre-Screening

Related Stories