How I ended up conducting successful tech interviews with just 1 question

http://www.nicolasbize.com/blog/how-i-ended-up-conducting-the-most-successful-technical-interviews-with-a-single-question/

by bubblicious • 12 years ago

261 133 12 years ago

nicolasbize.com

insaneirish2• 12 years ago

I can't take seriously someone who doesn't capitalize letters appropriately.

sho_hn6• 12 years ago

Most alphabets don't have letter case. Its utility particular in written English is questionable at best.

I'm not saying you should stop capitalizing, mind you -- there's value in standardized orthography. But it's a good idea to set your horizon beyond it. I've noticed that I've become much more flexible and less dogmatic about things like this since looking at a bunch of other languages and writing systems and recognizing written communication as spectrum.

heurist• 12 years ago

I learned the other day that before the printing press all characters were 'upper case' but were squashed down to be able to write quickly and use less space. Printing press creators copied what they saw in written text by creating small characters alongside the normal large characters and case rules developed out of that. You can play around with squashing handwritten uppercase characters (write them fast and small) and see why lowercase characters look the way they do.

nawitus2• 12 years ago

Most alphabets don't have letter case.

But the latin alphabet is apparently the most common alphabet. I don't really see your argument here.

insaneirish2• 12 years ago

He or she is writing in English. English has rules. Some rules are made to be broken. Capitalization is not one of them when it comes to writing coherent prose.

archagon• 12 years ago

I think it lends a sense of humility to the writing.

sliverstorm2• 12 years ago

To me it just lends a sense of laziness.

ldng• 12 years ago

It's some French hipster fad. My ex-boss did it too. It can b annoying at times but once you get accustom, it's bearable. I don't even notice now.

ianstallings• 12 years ago

It doesn't help that it's pretty long-winded to just make a simple point, taking it from just annoying to painful. I think I could fit the point into a tweet to be honest - 'What was your most interesting project?' is the best interview question.

And the fact that we're discussing it here in this thread makes your point stand - it detracts from the message trying to be conveyed.

asuffield3• 12 years ago

The flaw in this method is that it assumes there is only one kind of developer: the lone architect, who builds fine tents out of whatever is lying on the ground and then departs.

What it won't get you is breadth of experience, operational background, full-stack thinking, or people who can look at somebody else's work and say "here are the ways in which that is going to blow up in your face, two years from now". (Anybody care to add to this list?)

If all you ever do is bootstrap new projects from nothing then maybe that's the sort of people you want to hire, but it's probably not enough to build a sustainable product.

Spearchucker• 12 years ago

It doesn't assume that at all. You as the interviewer get to listen. You can ask probing questions to get the insight you need, or to steer the conversation. It's a great question, because it relaxes the interviewee as she gets to talk about something she knows well, probably enjoyed, and - if she's as good as you hope - has good depth and breadth of knowledge about.

howeyc• 12 years ago

I admit I just skimmed the article, but I don't see a requirement that the project you choose to talk about be a solo effort.

I think the point of the question is to find people who are passionate about their profession (in this case programming).

iamben• 12 years ago

I'm not sure I agree - from reading the article, I got that the author found people who were excited and passionate about what they did, people who were excited to code. It may have been the lonewolf project that lit them up, it may have been the massive team effort where they found a novel way to help an entire team. But it was someone who really believed in something, rather than someone who was simply working for the weekend.

Goladus5• 12 years ago

I don't think that question precludes all of those things. Anyone who is a "full-stack thinker" with broad experience should have at least one project in their past where they created something they're proud of.

asuffield3• 12 years ago

I'm just going to pick one person to reply to at random since there seem to be half a dozen comments misunderstanding the same thing...

Yes, a person with all of those things can answer this question. So can a person who doesn't, which is the point. Answering this question correctly cannot identify people with these highly desirable and uncommon skill sets, so the expected result of using it as your only interview test is to hire the most common subset of people that can pass it.

On further reflection, I have another objection to this interview approach. I would never work for this company because it fails my personal red-flag test of the hiring process: "Do I want to work with the worst imaginable person who could pass this interview?"

smtddr• 12 years ago

The thing you call a flaw, I call a strength. Anyone who would interpret the question that way and just shuts down instead of asking for clarification or making any kind of attempt to describe _some kind of_ project they've been involved in... is a red-flag; you shouldn't hire that person.

iSnow• 12 years ago

I don't think so. If I had been an interview partner of him, I'd describe how I (as part of a three-person team) re-architected a 10-year old JEE application and converted it to a Jersey/Angular single-page app - while retaining parts of the old code that dealt with scientific calculations, replace other parts with a math library and found the flaws in the ORM configuration that made it take 30s for a page view.

After we were done, the same page took less than a second - but the thing that would "light me up" was the process of finding the performance bottlenecks and gradually arriving at a better understanding of exactly why bizarre flaws existed in the way the old application configured the ORM (lack of experience by the original developers and most likely external consultants that told them that using an ORM was professional but not teaching them enough on how to work with it)

GrinningFool• 12 years ago

    What it won't get you is breadth of experience, operational background, full-stack thinking, or 
    people who can look at somebody else's work and say "here are the ways in which that 
    is going to blow up in your face, two years from now". (Anybody care to add to this list?)

This is true - because that's a byproduct of experience and not passion. Experience can be soon on a resume.

Arguably, passion for the work leads one to make more of the years of experience - ensuring it's "10 years of experience" vs " 1 year 10 times".

bubblicious4• 12 years ago

There were some (a rare few unfortunately) who would be thrilled to talk about the projects they had done professionaly and were really proud of. They would talk in great details about how perfectly designed their asset pipeline was, the paradigms they had put in place, etc. No matter what the project was, if they were excited to tell you about it, it was always a great sign.

reality_czech• 12 years ago

Everyone knows that the only question you need to ask is "how is this an issue?"

http://27bslash6.com/interviews.html

Goladus5• 12 years ago

tl;dr:

The question he arrived at is: “... will you please tell me about the best project that you’ve ever created?”

Then watch for enthusiasm and pay attention to the details. The goal seems to be to identify someone who really enjoyed and takes pride in a program they've written.

wildpeaks• 12 years ago

The "project you're the most proud of" might be a more precise way to phrase it because "best project" could mean either:

- created the most value for your employer even if it was a soul-sucking nightmare / merely maintenance project with no challenges.

- you learnt the most from it / overcame challenges you didn't know you could / came up with moonshot solutions to issues no one even realized were there.

spoondan• 12 years ago

People can be proud of either of those accomplishments as well.

Honestly, I doubt "most proud" versus "best" makes a difference at all. Candidates generally explain their answers to these open ended questions. If they (for some reason) don't recognize the implicit, "And why?" part of the question, you just follow up with it.

The conversation will never be just, "Foobaz, next question." One way or the other, it will be, "Foobaz because..." It saved the company $2M/year. Or users really loved it. Or it improved the engineering process and made releases faster and lower risk. Or it was a really tough problem that required a lot of creative thinking. Or it taught me a lot about 3D game programming.

warcher• 12 years ago

It's always surprising how often interviewers fall back on "API Jeopardy" when trying to make hiring decisions. Maybe it's just because my career hasn't really allowed the luxury of getting deep on any technical stack and staying there, but I just don't keep the details of obscure function calls in my head all the time. And I have to wonder at the utility of testing a developer on having memorized the kinds of things that they will always, ALWAYS be able to look up when they're actually working.

coldcode• 12 years ago

Thank you, finally someone who gets the idea. Though I like to ask two questions, one the best project you ever worked on, one the worst project you ever worked on. You can learn a lot about a programmer by the details they relate in both cases.

chippy• 12 years ago

ooh, that is a good one! I wonder if some people would mention the same project?

danmaz74• 12 years ago

The "tell me about your best project" question is great, but I always combined it with one apparently simple programming question, like giving the quadrant from the (x,y) coordinates. Maybe the OP is able to understand how logic-minded his candidates were just from talking, but many times I was disappointed from the actual ability to write down some code (even discounting the stress of the situation).

ruswick• 12 years ago

The problem is that this method disproportionately hurts people who don't have the time or energy to effectively hold two jobs (one full-time position at their day job and one as an independent developer or open-source contributor by night) because of family, friends, or some other facet of life beyond their laptop.

I think it's totally unreasonable to expect that everyone spend every minute of their lives coding or to have some kind of deep, personal connection to the code they write.

This thinking is ridiculous, and doesn't really appear in any other industry. Do you care whether your accountant's idea of an enjoyable Friday night is sitting at home making more spreadsheets? Would you demand that your eye doctor go home and craft her own lenses in her garage for fun? Competency doesn't require fanaticism, and no employer should expect that their employees devote their entire lives to their occupation.

If I want to code for 8 or 10 hours a day, go home and enjoy myself in the little free time I do have, then wake up and do it again, I don't see why that makes me an inferior employee.

This just seems like more misguided, unjustified cultural absolutism that is so prevalent in the industry.

EDIT: For those saying that the article doesn't necessitate spending massive time outside of work writing code, the author clearly conflates the two. "i have always been convinced that those who love code do not restrict their coding activities to their work. they take home that love and continue to create for fun as a hobby." Personally, I think passion could be a valuable heuristic in hiring, but the author seemed to imply that passion is only measured by your willingness to work outside of your day job. At the very least, that seems to be his expectation of good candidates, and his hiring process clearly disadvantages people who can't or won't code 24/7.

eps• 12 years ago

Accountants don't create something out of nothing. It's an inherently supporting activity.

Also, the question doesn't imply in any way that it has to be an off-work project. Its only point is to see if a person actually cared for any code they wrote in the past or if they treated coding just as means to an end.

nawitus2• 12 years ago

The question in the article was 'will you please tell me about the best project that you’ve ever create', which excludes most work projects, since work projects are typically not single-developers projects.

rwallace2• 12 years ago

I agree with you that if someone is spending forty hours a week writing code at work, it makes no sense to expect him to write more code in his spare time. That's the bad part of the article.

The good part of the article stands, though. In a good programmer, there will be a spark, and he has the right idea about the interview question.

A perfectly valid way to respond to it would be to refer to something you did at work. "Oh boy, that's difficult because the code I've had to deal with has been legacy stuff that... well let me see, there was this big ball of mud where it took us a week to track down an intermittent bug, the debugger was useless... but after that, I figured out a way to put in logging code so we could at least see what was going on and we'd have a better chance next time..."

You're not looking for spare time programming or some particular answer. You're looking for a non-rote answer. You're looking for the spark.

PhasmaFelis• 12 years ago

The first thing that popped into my mind when I read the question was a data-processing script I wrote at work in three days that saved hundreds and hundreds of person-hours of manual work, a task that had stumped a couple of better programmers than myself.

If Nicolas Bize would accept that sort of thing, I think it's a good test. If not, yeah, it's problematic.

Edit: I've just asked him about it in comments under the original post.

keyme• 12 years ago

"The problem is that this method disproportionately hurts people who don't have the time or energy to effectively hold two jobs"

No it doesn't.

I'm a guy who has the "two jobs" by your description. More precisely, it's a job and a hobby. However, my "best/coolest project/work" has always been something I have done for an employer. I can't imagine coming close to that "level" of work by just playing around at home.

If you can't answer the question in the post with a project that you've done at work, and you don't program as a hobby, you simply do not qualify for what OP is looking for.

fixedd• 12 years ago

The code I write between 9 and 5 on weekdays is also way higher quality than the stuff I write the rest of the time.

snambi• 12 years ago

Sorry this is just excuse

the_cat_kittles• 12 years ago

I think one of the points of the test is to see how intrinsically motivated the applicants are about the job. You can throw as much money and perks at someone as you want, but it probably wont work as well as intrinsic motivation.

seba_dos1• 12 years ago

You can still talk about some project from the past. If you never had any hobbyist pet-project (during studying? while learning first language(s)? before getting a job?), then it's quite likely that you're not the person the author is looking for, no matter if you have or don't have time for a hobby now.

nthj• 12 years ago

The problem is that this method disproportionately hurts people who don't have the time or energy to effectively hold two jobs (one full-time position at their day job and one as an independent developer or open-source contributor by night)

“Oh! A few months ago I created a basic budget tool based on graph theory—so I can create 'branches' like in git, but for my personal budget, and then forecast how certain changes affect my budget overall. It really helped my wife and I visualize how certain decisions play into our plans over the next couple of years. Would you like to see how I implemented the recursive SQL lookups in ActiveRecord?”

I hacked out a basic MVP for the above in 4 hours one Sunday afternoon. I don't hack on side projects every Sunday, or even frequently: a lot of weekends I want to explore the city, sleep in, and work on projects around the house. But I love programming, and I love how software can affect our every day lives, and sometimes little itches like the above come up and I scratch them. And this one story would be more than enough to pass the author's test with flying colors, and is /nowhere near/ holding down two jobs.

lifeisstillgood4• 12 years ago

Without sarcasm, yes I would like to see how you branched - it's not an easy solution - and to be honest the hard part of this (and I have a github project for budgeting too) is getting data into a form where you can easily do a projection

What was the household budget in originally - a RDBMS? Mint or quicken ?

Not meaning to be aggressive - just wanting more context - and yes, please let me see code :-)

mamcx• 12 years ago

And can do merge? This could be great for a sync engine for sql! A way to do branch-merge

slang800• 12 years ago

This thinking is ridiculous, and doesn't really appear in any other industry. Do you care whether your accountant's idea of an enjoyable Friday night is sitting at home making more spreadsheets? Would you demand that your eye doctor go home and craft her own lenses in her garage for fun?

Actually, it does appear in other industries. Many of my friends who are designers do various sorts of art in their free-time, whether that's drawing or even the same type of photoshop stuff they do at work. One of the best CNC machinists that I know has come into the shop on weekends to experiment with making ornate snowflake ornaments, folding knives, and motorcycle attachments on the mills... His job consists of machining parts for cable laying machines, but he enjoys the challenge of figuring out how to manufacture complex parts so much that he does it in his free time.

I don't know any accountants or optometrists very well, but I wouldn't be surprised if the passionate ones spent time outside of work learning about the latest advancements in their respective fields.

Competency doesn't require fanaticism, and no employer should expect that their employees devote their entire lives to their occupation.

True, _competency_ doesn't require fanaticism, but if you want to _excel_ at something (and don't want to end up hating your life in the process) you had better really enjoy it because you're going to have to put in several thousand hours of learning. Spending all that time exploring your particular field of study won't be a problem if it's something you love, and you're more likely to continue exploring it and keeping yourself up-to-date if you're interested in it. Thus, passion makes a pretty good predictor for success among high-achieving employees.

doorhammer• 12 years ago

True, _competency_ doesn't require fanaticism, but if you want to _excel_ at something

This is along the lines of what I was thinking.

If I'm looking for someone who is really passionate about what they do, they're probably going to be associated with it outside of their nine to five. This could be reading books, magazines, talking to people, participating in discussion boards, or trying to figure out problems.

That doesn't mean every employee is like that, or that you have to be like that to be a worthwhile employee, but it's not crazy to think that someone in almost every field will be actively involved in something related to their profession in their day to day, outside of work.

Even with accountants, you might not be doing someones books all the time, but you might have an active interest in financial news, forums, current scams, common mistakes, loopholes you could use, etc. etc. (I don't know anything about accounting; don't judge me :))

cpwright• 12 years ago

I don't know any accountants or optometrists very well, but I wouldn't be surprised if the passionate ones spent time outside of work learning about the latest advancements in their respective fields.

My grandmother was a corporate tax specialist for AT&T, she certainly seemed to enjoy doing her personal income taxes (and those of some of my uncles and aunts). This was way before Turbotax, Taxcut, and the variety of programs that have made it less unpleasant.

roguecoder• 12 years ago

I don't think excitement or passion or pride require fanaticism either. When I ask my friend who paves road about a project he's particularly proud of, he'll have an answer and it's not because he's paving roads in his spare time.

What is different about some other professions is that there are ways to develop that passion during apprenticeships or entry-level jobs while learning from those with greater experience. Mastery is far easier to acquire with a teacher on hand who can give timely, guided feedback and correction.

AndrewKemendo3• 12 years ago

Competency doesn't require fanaticism, and no employer should expect that their employees devote their entire lives to their occupation.

Very true. The only problem with that is that when judging between two job candidates, the one who is devoting their entire lives to their occupations is going to be a more desirable hire.

At the very least, that seems to be his expectation of good candidates, and his hiring process clearly disadvantages people who can't or won't code 24/7.

Just as the field of competitive Olympians disadvantages people who can't or won't train 24/7

akurilin• 12 years ago

Also luckily there are not that many people out there who sustainably only do coding every hour of the day. Eventually real life catches up to you and you can't do that any longer.

Also, there are only so many gold medalists every four years. In the IT industry, there's no theoretical upper bound, so it's not nearly as dire, you don't really "run out of slots" as fast.

staccatomeasure2• 12 years ago

the one who is devoting their entire lives to their occupations is going to be a more desirable hire

Not true. The more productive and effective candidate is the more desirable hire.

ufmace• 12 years ago

I'm a bit skeptical of the idea that a person who devotes their entire life to their occupation is a more desirable employee than the one with a balanced life. Maybe if you were doing something completely rote, but who devotes their life to that anyways? To be a productive developer, you should also know how to:

Schedule and prioritize your work to align with business goals Interact with non-technical users and domain experts and understand their problems and techniques Work productively with and integrate your work with the work from related fields, like DBAs, designers, artists, etc Work with a team of developers productively, including making your ideas heard without being intolerant or offensive

That's stuff you don't tend to get from doing side projects. Maybe if you turned a side project into a real business, or were a senior contributor on a large, long-lived open-source project, or something like that.

Come to think of it, I was more sympathetic to the article's idea at first, but writing this makes me reconsider whether just having passion for something that you have built is enough to base hiring decisions on.

grecy• 12 years ago

The problem is that this method disproportionately hurts people who don't have the time or energy to effectively hold two jobs (one full-time position at their day job and one as an independent developer or open-source contributor by night)

No, not at all.

You're assuming that "will you please tell me about the best project that you’ve ever created?" has the words "Outside your previous jobs" - which it clearly doesn't.

If you've made or contributed to something great at a previous job - talk about that!

As soon as I read that question I paused and thought about how I would answer it - of the 10 possibles I came up with, about 5 are "personal projects" and about 5 are "from previous paid jobs" - the thing about the paid jobs ones is that I get to talk about all the awesome people I got to work with, all the stuff they taught me, all the conflicts I had with that Project manager (!!), all the trade-offs we had to make so the project actually shipped, some of me reservations about so much technical debt being created, etc. etc.

There is absolutely nothing wrong with answering this question about something you were involved in at a previous job. The point is to find people that are passionate.

themartorana• 12 years ago

Not to mention that sometimes paid jobs lead to getting to work on large projects that have millions of dollars of input or output, or on hardware scales you never could as an individual. I have to imagine if I worked on some software for the Mars rover, it would trump everything not because it was the best software I ever wrote per se, or it was the greatest environment, but my software is on f-ing Mars!

I'm incredibly proud of some independent projects, but they're definitely informed by my days working on million-dollar projects, some of which were pretty awesome.

collyw3• 12 years ago

And if your job has been maintenance programming?

click170• 12 years ago

Disregarding the conflation, I think the key point here is passion.

I want to work with people who have passion for what they do. You don't have to keep doing it when you get home at the end of the day, but if all you're doing this for is a paycheck and you aren't really passionate about what you're doing, I find it impacts the quality of the output and the speed with which you absorb new information.

pedalpete• 12 years ago

Many commenters on this thread point out that people don't have time to code, or have other interests, and question what other profession expects people to give up their free time to continue work.

This is not unrealistic in other professions as well, maybe just not as easily recognizable. I have friends who are Ad execs and love advertising as an art, they'll often discuss ideas they've had, things they've been thinking, etc. etc. which is outside the scope of their work. It may be the client they'd like to bring in and the ideas they have to improve that client's business, etc. etc. What about doctors who vicariously keep up on all the latest knowledge in their field and are actively collaborating with other doctors on discoveries and research (my sister does this as a veteranarian, and is also randomly helping people in the park on a Saturday morning who have questions about their pets, as well as doing a spot on the local news about caring for your pets). This is unpaid/hobby work, but it isn't as concrete as what we do, and therefore, people don't consider it.

We aren't the only group that is passionate about what we do, and we aren't the only one's who work on our craft outside of work hours.

I think a better way to take what the author is saying is that he is looking for people who look at software development as a craft, and people who are so passionate about it that they are constantly engaged in their craft even outside of work hours. They love what they do so much that they just can't stop.

At the same time, I think it is important for employers to recognize when and if they need somebody who meets this criteria.

collyw3• 12 years ago

Your ad exec is about the equivalent of me discussing new libraries / languages / frameworks out of work hours. I never heard anyone ask for that in an interview. I have seen a fair amount of ads asking to see my side projects / github.

I am reluctantly creating a side project at the moment, as I want to get into freelancing, and all my previous work is in house. Even without kids I find it difficult to find enough hours to put into it (trying to use a free hour here and there leads to almost zero productivity, I need a minimum of three at a time).

djur2• 12 years ago

One other tricky scenario: if someone is really invested in a particular field of programming, like crypto, they may be in a situation where any work they do outside of their employment is at risk of having their employers claim it as a work product or as a violation of a non-compete contract.

I've met several very brilliant programmers in my day that liked to work long, super-focused days on the problems they were presented with at work and then would go home and do anything but look at a computer. I'd be hesitant to suggest they had any lack of love for their chosen profession.

jakejake• 12 years ago

I don't think you need to work on side projects to be a great programmer. It's just that there are very few great programmers who don't have a variety of side-projects. It's a natural side-effect of the desire and curiosity that leads people to become great. They don't see side-projects as some sort of drain on their free time.

I don't think it's a bad thing if you don't aspire to "greatness" though. There's plenty of jobs where a decent, reliable programmer is perfectly fine. 49% of us are below average, after all.

avalaunch• 12 years ago

The comparisons you've made are with non-creative fields. For those I know that work in other creative industries (graphic design, photography, music, theatre, copywriting), they almost always spend a good deal of their free time exploring their passion. If you love doing something, you probably won't be satisfied if the only time you get to do it is under the strict direction of another.

In any case, if there are enough candidates to select from, and you have no better way of determining which are the better candidates, why not choose the ones that code for fun in their free time over the ones that don't? Those candidates almost certainly have a passion for programming while the others it will be harder to tell. Plus those candidates will be constantly learning new skills that I as the employer will benefit from.

mark-r• 12 years ago

The stated goal is to eliminate false positives ("hire the wrong guy and you will be stuck with that person forever.") That necessitates you will have an increased number of false negatives - see http://en.wikipedia.org/wiki/Type_I_and_type_II_errors. The consequences of missing a great hire are less than the consequences of hiring a loser.

My own problem with the question is that I wouldn't know what to answer - I have too many things to choose from. Many are too far in the past to remember lots of details about.

bubblicious4• 12 years ago

Yes, my article might be a little misleading, as I was really looking for the excitement more than anything, work-related projects fully included. The most important objective of my question was to put the candidate in a full comfort zone. No surprises, we talk about your stuff, the stuff you prefered. And we go deep into it (we would discuss it for 30 minutes). I would be respectful and genuinely interested, I could ask any technical questions I wanted because it was their grounds. We would discuss architecture, design patterns, technical choices, etc. (so it wasn't just one question in the interview... more like one question to lead to a situation where i could get the most info out of).

bigiain• 12 years ago

FWIW, my eye doctor happens to be a fantastic wildlife and nature photographer.

I think the article author has a good point - in some cases. For a programmer working in a strict waterfall project, who's "just coding his allocated items of a per-determined feature list", passion might not be important (and may even be contra-indicated, they'll likely leave when they work it out). But if you need creativity from your coders, some level of passion in _something_ is important. I suspect if an interview candidate lighted up telling the story of their quadcopter or bamboo bicycle or burningman project or anything where they can demonstrate solving problems within a set of constraints - even if it didn't involve a single line of code - it would have been looked upon more favorably than somebody who can't demonstrate passion about _anything_.

matwood• 12 years ago

I agree about the passion indicator. I over interviewed and was told to prepare a 15 presentation on anything I cared about. It did not have to be technical, but I needed to be prepared for questions.

The interview ended up being one of my best interview experiences though I turned down the job.

joncalhoun2• 12 years ago

With hiring I feel like it always comes down to two potential paths to pursue:

1) Hire all good candidates as well as a fairly large percentage of bad candidates.

2) Reject all bad candidates, and hire a fraction of the good candidates.

Which route a company takes generally depends on what the company needs. In this case it sounds like his company needed to minimize bad candidate hires more than find all good candidates.

So yes, what you said is correct. This approach will filter out a large number of great candidates without extra time, but assuming the company is finding enough candidates to hire, this isn't a problem they need to address.

I believe this is the same reason why many startups can get away with trial work periods and other interview tactics that large companies cannot. The number of candidates willing to do this is significantly smaller, but it is seemingly large enough for the startup using the tactic.

I suspect that until the industry finds a significantly better way to filter good/bad candidates that this will not change. So if you are looking for a startup idea - this is a great area to get into =D

yason• 12 years ago

I suppose the work the candidate is most keen to discuss doesn't have to be recent which makes it fair.

All good programmers I know have at some point spent significant amount of their free time programming on their own, that's why they've become good programmers. They might have kids now or too much work, but I would be amazed to meet a good programmer that has only ever programmed at work.

Despite that the fact that I have kids and a job that does take its time, I do program as a hobby. It mostly kicks in when things are slow at work or when work is less about coding and more about something else as if I had a weekly or monthly coding quota that I need to fill in. But I do have the urge to write programs independent of my work -- if I'm lucky I can abuse my work to fulfill that urge.

freetshirt• 12 years ago

Yet nobody seems to care about reducing the initial hiring pool to some statistic and only hiring college graduates who think HTML5 is a programming language.

logicko• 12 years ago

There's another group this method hurts: people whose personalities are closer to Mr. Spock than to McCoy. Same goes for the method another commenter suggested about "tell me about your best bug..." (Though perhaps it doesn't hurt too much, because such people usually develop hacks to lessen their natural disadvantage across the board.) Speaking strictly for myself, I don't really feel pride in my code. My identity is completely separated from things I do. When I finish something, there's a sense of accomplishment, but it passes very quickly and I move on to the next thing. If that something was "hard", on reflection it feels easy, because I already did it. And it's not like I've never known excitement and such, or never feel it now. I thought programming was the coolest shit too when I was 14 and I felt the excitement of making a website the color I wanted, and using a for loop to eliminate tons of HTML. I just don't get excited over stuff I feel is "easy", which is everything I've ever done and everything that looks close enough to those things. If you want to make my eyes light up a bit, ask about things I want to do but haven't gotten around to yet. But it still has the highlighted problem of selecting for people who are most enthusiastic about whatever they're talking about, not for people who can best do the job.

All I want from a programming job is to get paid money to spend a lot of time with a computer writing software, and spending time with other humans strictly as needed to advance the computer project. I don't want to sit around a campfire (or water cooler) and trade stories, and I don't have a memory for things like "best bug" and other events people tell stories about. Unless at that event's particular moment I think "Maybe this bug (or whatever) would be good to use as an example for a future interview." and I make a reminder to add it to a file of such life events I review before interviews. It's one of a bunch of little things I have to do that some others don't, which sucks, but considering the whole messed up wasteland that is programmer interviews where pretty much everyone has to do a bunch of BS prep work unrelated to the actual job just to stand a chance, I don't think it's worth complaining much about. Hence the throwaway -- time to move on to other things.

badman_ting• 12 years ago

Right. The point here is actually reducing the pool of candidates to those enthusiastic enough, and with enough time, to do projects in a way that fits the template this interviewer is looking for. I don't doubt they had success hiring people this way, rather the problem is in the false negatives as you point out.

kika• 12 years ago

That's almost exactly what I was doing for years. Two differences: 1. I sometimes ask 1-2 technical questions, for example if the candidate claims exceptional knowledge of some important (for us) technology, but his/her resume doesn't show extensive use of it. Like: "You say you know CouchDB inside out, but you've used it only once and not for long, interesting.... Can you tell me how the _changes feed works - if I listen on this feed do I get just the changed documents IDs or whole documents?" 2. Instead of "what's your best project" I ask more aggressive question - "imagine I give you 1 million dollars right now and in return I want to be a part of what you use it for - a share of profits, credits in the movie, etc. What would you do?".

Edit: ah, the almost mandatory third question: why do you want to leave the job you're currently having?

cottonseed2• 12 years ago

The best advice I've seen on how to structure an interview came from Nick Corcodilos of Ask the Headhunter [1] fame, from his book Reinventing the Interview to Win the Job [2]. If you want to show someone you can do the job (or see if they can do the job), do the job, or as close as you can get to it in an interview setting. Anything will be selecting for indicators that will be more or less correlated with job performance.

[1] http://www.asktheheadhunter.com/articles.htm

[2] http://www.amazon.com/gp/product/0452278015/ref=as_li_tl?ie=...

cottonseed2• 12 years ago

Coincidentally, the most recent Ask the Headhunter article [1] is "The Single Best Interview Question... And The Best Answer". Hint: It isn't about your hobby project.

[1] http://www.asktheheadhunter.com/habestinterviewquestion.htm

lifeisstillgood4• 12 years ago

Absolutely love that ! Weirdly it reminds me of ramit sethi briefcase "technique" which is roughly research the prospect, come up with some proposal that shows you can as value and pull it out of your briefcase at the opportune time.

djur2• 12 years ago

This is specifically about technical interviews as one of several steps, though. "What is your plan for being effective in this job" is the kind of question that, in the OP's three-step interview, should be asked by the manager.

My current employer has candidates do a brief presentation about a project they are proud of. It's useful but imperfect (some people just aren't good at speaking to a group, which is okay for this job; some people aren't legally permitted to go into a lot of detail about their earlier projects).

xur17• 12 years ago

From an interviewee's perspective, these are my favorite interviews too. I get to talk about something that interests me, and have a discussion about the different decisions I made. If the interviewer wants, they can dig into specific details, or ask more pointed questions about the programming language. I find trivia questions off-putting, and tend to limit the depth of discussion into my skills.

And to the people saying that this necessitates projects outside of work - I don't see why it does. You can talk about school projects, projects from previous jobs, etc.

recalibrator• 12 years ago

TL;DR Please tell me about the best project that you’ve ever created

Tomis02• 12 years ago

Give it a couple of months, it will start not feeling right again.

lifeisstillgood4• 12 years ago

Everyone seems to have missed the most important part of the article "this was in France, and you can't just fire someone so if you hire badly you are stuck with that person forever"

The US has the most liberal/right wing/psychotic labour laws in the Western world, and as such the best approach is to hire anyone not an idiot and fire them once on the job experience teaches you if it's a good fit.

This results in massive turnover, and little incentive to improve the interview process

I'm not sure where I am going with this but I am amazed that the local labour laws are not mentioned on this three (afaik)

Renaud• 12 years ago

Even in the US, where labour laws generally don't favour the worker, it wouldn't make much sense to set the bar too low and count on being able to fire people to eventually get the right one.

Hiring people is resource consuming and is always a large investment for a company. Unless you are hiring low-skill workers who can easily be replaced, a high-tech worker will require time before she's productive on the job. Those weeks/months, cost money and figuring out that someone is not working out after 3 months of investing in her could cost a lot more to the company than just the cost of re-hiring someone else.

It's always a better strategy to be diligent early and avoid wasting company resources just to shorten the hiring process.

That being said, being able to easily part ways with an employee is important for a company since the cost of taking a risk is much lower. In France, employers are very reluctant to take risks as they can get stuck with people who are either bad or don't fit in.

There is a middle ground between a system where you can fire people for any reason, whether related to their work or not, and a system where you can't even fire people who don't work out.

lifeisstillgood4• 12 years ago

Tokenadult has a regular post that has grown to the size of a small essay which basically says the only way to tell if someone can do the job is to give them the job to do.

How long does it take to become obvious someone is not a good fit? A couple of hours? No, if humans could not fake it for a couple of hours no one would ever go on a date again.

A couple of days - maybe. My own pet theory is when I hit a new contract is to commit a bug fix by the end of the day and push to production by the end of the week.

But no one sane is willing to give a week to an interview, and few a whole day.

I think in the end we judge people by their public output - in other words GitHub reall is going to be our CV

vorador• 12 years ago

This "in France you can't fire people" meme needs to die. Most software engineer contracts have an four or eight month no questions asked firing provision.

randomfool• 12 years ago

In these free-form discussions I find it really important to politely disagree with some technical decision of their project to see how they take the feedback and defend the decision. Red flags are when they either cannot accept disagreement or come up with questionable justifications. Correct answers include clear explanations of what the trade-offs would be and clear rationale for their original decision process.

clairity• 12 years ago

this is a good point. one developer i hired had the spark but not the maturity to deal with disagreement objectively. he has trouble communicating and working with others. he has potential, so i'm hoping he grows out of it over time.

bubblicious4• 12 years ago

Yes, those open questions could tell you a lot about how the person communicates as well, even though I was very careful as the context of an interview is always biased towards people who are extraverts.

crazygringo• 12 years ago

On the one hand, this resonates with me... the last time I was doing a round of interviewing (being interviewed at several companies), one person asked this question -- and it was the one point when I really "lit up" and really enjoyed being interviewed.

On the other hand, I wonder if it really produces results that are any better. At least, with a quiz, you can give it to current employees who work great, and discover the quiz turns out to be worthless. (As the author described.)

But with a open-ended question inviting open-ended answers, how can you tell if it's really working? Of the many coworkers I've had before, I can think of one who certainly would have aced the question -- a programmer through and through -- but who was fired after a few months due to poor work ethic and sloppiness. Because while he had enthusiasm for programming, he had no time for "standards" or "teams" or "business needs" -- things like commenting code, communicating well with others, following through on commitments, etc.

So I wonder if the author is going to discover, after a few more months, that there are a few more questions than just this 1 question that also matter just as much...

Kaivo• 12 years ago

I recently went through an interview where they asked me somewhat similar questions, where I was able to talk about my personal projects and let them see how much I love doing that.

Before the interview, they told me they went through my Github and looked at my projects, and actually called me for an interview partly because of that. My Github served me as a portfolio. Artists have to hand in example of their work, and I believe programmers should do as well.

That should be enough to see most of those who don't comment enough, don't follow rules and standards, etc. It still comes back to the issue of having time to work on personal projects and having it displayed somewhere, but the same could be said for Artists.

bubblicious4• 12 years ago

To answer your question, I worked for another year for that company and must have done about 100 interviews with that method. It wasn't perfect, but I found that it gave better results than the other stuff I had tried. Also, I found that the more I would bring the person in his comfort zone, the better appreciation I had for their qualities as a programmer. Also, that one question would always lead to a 30 minute discussion about their project (hobby or work). I would of course ask a bunch of other questions, but it was on the things they were the most comfortable with. I didn't have to ask them if they knew this or that, they would tell me all that they knew and we would discuss it. And wherever there was excitement, there was an indication of a good hire.

nwhitehead• 12 years ago

If you are limited to one short question, a better choice is to ask them: "Teach me something technical you've learned recently." You can evaluate their answer on technical depth and correctness, communication skills, and timeliness of the topic. And if they can't think of anything they've learned recently, that tells you something as well.

dpark• 12 years ago

I suspect you're largely filtering for people who are aggressively interviewing. When I'm deep into a meaningful project, I spend less time learning interesting technical bits. When I'm looking for a new job, I spend a lot of time learning random things that might be useful as a clever answer in an interview (because that's what so many interviewers want).

fillskills2• 12 years ago

Thats a neat question. Will keep it in mind for the next interview in addition to the OP

sheepmullet• 12 years ago

My best project was 12 years ago and I hated it at the time.

If you had asked me about it while I was working on it, and I didn't sugar coat, I bet you would have said no hire.

Secondly, it has just taken me 10 minutes to remember the details in enough depth to have a proper conversation about it. Again that would probably be "best work behind him...", or "doesn't seem to know the details....insincere" etc.

This question is easy to game and disadvantages people who aren't prepared for it.

sumedh• 12 years ago

If you know body language you can figure out if the candidate is honest or lying.

zeroonetwothree• 12 years ago

It seems that every week there's a post on HN about the "right way" to do interviews. It's too bad there's never any research done to actually test these strategies. I don't believe that you can get a good answer just from personal experience.

rdtsc• 12 years ago

Ok this is good. I have been facing the same problem. I did fizzbuzz, we have the quiz I also often ask about he "favorite" or most challenging "project", but I don't dwell enough on it. But now I think I should. I like this, I will have to steal this idea.

fiatjaf• 12 years ago

I see this kind of passion about personal projects and excitements about doing something only in the programming field, maybe some other fields, but never fields like dentistry or civil engineering.

Am I missing something? What prevents people from other fields from having "personal projects" to be excited with?

rbobby• 12 years ago

Shows up to the interview and then spends 5-10 minutes quietly reading the candidate's resume and making notes. Not very courteous and makes for an awful first impression.

vehementi• 12 years ago

Said commenting, not silently reading. "Ok you worked here... and here... you worked on this? Ok..." without saying "explain your work here" etc.

chazu• 12 years ago

Great article. Despite the comments decrying this 'one question' as unfair, I think we can all agree that having a real passion for programming is the single most important thing in making it from nooblet to competent and confident developer.

ScottBurson• 12 years ago

I like to ask a variant of this: "what's the most interesting project you've done?" If they come back with something that impresses me, I take that as a good sign.

dj-wonk• 12 years ago

I really hope that the magic question is useful, and I agree that many boilerplate Java-specific questions are misguided and less useful. Still, but I find it unwise to bet everything on one question.

Did the author collect data once he perfected his technique. Is there data?

In my experience, the best tests have many questions, including "experimental" ones, because you want to be able to test your hypothesis against various data points.

I'm also not saying that everything needs do be quantified. Open ended questions are very useful. But why not at least collect the data and tally it to the best of your ability.

kabdib• 12 years ago

One of my favorite questions is, "Tell me about the best bug you've ever found."

It's a great touchstone. Sure, it's subjective, but it gives valuable insight into someone's level of skill, how they approach problems, how they do diagnosis, and so on. Are they scientific? Do they hate on people and their code? Do they follow through with testing?

I've gotten answers ranging from "I don't know" (which is a fail, by the way) to full-stack expositions that boil down to bad code generated by the compiler, to someone finding and solving a fundamental design problem in a years-long project.

Any sufficiently senior engineer will have a tale of a bug that they tell in the circle around the campfire when the kids are tucked away (and probably still listening anyway). And if you're not a seasoned vet, I'd still like to hear about the race condition / double free / syntax error that took you a while to find.

YZF• 12 years ago

I think when you're just starting that first race condition or memory management issue are exciting. After a while it's not that exciting any more. I got asked that question and the first thing that popped to my mind was a very unexciting bug I was chasing in legacy code I was maintaining. I don't think the interviewer liked that answer because it was mostly grunt work and not some some amazing feat of engineering.

I don't think it's a particularly good open ended question for more experienced developers and an interview is not a camp fire circle... It could be part of an investigation in some past project, e.g. what problems did you have and how did you solve them...

halisaurus• 12 years ago

I've had this question asked of me before, and it seems valuable for the same reason as the question on the article: It's an opportunity to show passion/enthusiasm for programming work. I think even novice programmers who love to code will have that one bug that got them. Even if the story is anticlimactic ("I missed a semicolon"). iMo, explaining how they got to that conclusion in a logical, positive way, didn't give up or just copy/paste the answer, and ultimately moved on in the project with a new lesson in syntax is the real "passing answer."

trentnelson• 12 years ago

One of my favorite questions is, "Tell me about the best bug you've ever found."

I've started an initial push at Continuum to open source our interview questions... and this is the first question :-)

https://github.com/ContinuumIO/interview-questions

gregwebs• 12 years ago

Others have noted that the best way to interview is to see if someone can do the job is to have them do the job or the closest thing to it. For the typical developer not available for freelance I combine this approach with the approach in the article.

Show me some code from a project of yours and I will ask you to add a feature, fix a bug, or maintain it. The project can be something really simple. It is hard to learn new technologies without creating some kind of simple project.

This is a second interview, the first interview is by Skype to figure out the logistical/cultural fit, go over resume, and then spend some time talking through a technical issue.

This worked very well for my first hire. The downside of this approach, and a big reason it is not done is it requires an hour of preparation time for the interviewer to understand the candidate's code, run it, look for bugs, and figure out what to ask about. I think it is a much less arrogant way to conduct interviews though and a much more rewarding process for the candidate: they get to delve further into something they already have interest in.

capkutay• 12 years ago

With all the APIs and frameworks that make churning out good looking, usable apps a relatively simple process, I don't think this is a good question by itself.

dzink• 12 years ago

To add on to the "Tell me about a project you are most proud of?" question, having gone through different stages from burnout to re-kindling of the hacker within, I can see it measures attitude well, which is crucial. For aptitude, however I would go a few layers deeper. "What was the toughest problem you have worked on or solved?", "How did you solve it?", "What did you learn?" the choices of problems tell you as much if not more than the answers about where the candidate's passion stands. Are they vague or specific? Do they use industry terms or are they self taught? What types of issues make their eyes spark? etc. If you are hiring entry level people, passion is most important, but if you need to have people who hit the ground running in an area, nothing beats a nice rigorous interview, with some code writing involved, in addition to your question.

cportela• 12 years ago

I'm fairly certain I can't pass this test even though I've tried. Just haven't had the skills yet for what I want to make and the stuff I've made doesn't impress the people I want to be with.

fillskills2• 12 years ago

In addition to hiring people passionate about coding, I have found out that anyone who is really passionate about something (music/surfing/rock climbing etc) is a good programmer. Some of the best programmers I worked with were either crazy about programming or passionately followed through with something else in their lives. The key being that they followed through with their passion. Has anyone else seen this? Or is my data too limited to my personal experience

nardi• 12 years ago

The real problem with this style of interview question is that it is good for those who talk well, and bad for those who don't talk well. I have seen my fair share of candidates who are great talkers, but can't code for crap. The interviewers who didn't ask coding questions loved them, but those of us who made them code knew they sucked.

You should have stuck with programming on the white board. That's the right way to find good programmers.

falsedan• 12 years ago

I like a more structured technique called GRASS, described in Ovid's 'Agile Companies go P.O.P.' presentation: http://www.slideshare.net/Ovid/a-14058644/22

The basics are: ask the candidate to describe a project they worked on (that they enjoyed, listed on their resume, or anything). Ask follow-up questions until you find out: the Goal of the project; their Role on the team; what Actions they personally took to reach the goal; whether the project Succeeded or not; and Speculation on what they would do differently in hindsight.

Repeat until they run out of projects to talk about, or you run out of time.

netforay• 12 years ago

My interview question was similar "What is the biggest program you have ever written" (As I hire only freshers). Problem is, not even 1% seems to pass this test. For those who are talented but lazy till now, I offered another choice, or "Can you do me a small Snake game now"?

How I ended up conducting successful tech interviews with just 1 question

Related Stories