Use Project-based Interviews Instead of “GitHub”

http://ejohn.org/blog/project-based-interviews/

by slig • 13 years ago

54 55 13 years ago

haney• 13 years ago

If AT ALL possible I prefer to contract developers. We pay a candidate for their time and pair program with them for a week. This lets us evaluate not only their ability to code but also their desire to learn and their team culture fit. Since I've started doing this with a new team it's been a lot easier to manage the people that I work with. All the existing developers get to know someone and they all get veto power over whether its someone that they'd want to work with AFTER they're worked with them.

rtfeldman• 13 years ago

It's hard enough trying to find good people; as pleasant as that sounds, I can't imagine narrowing my criteria to "good people who can take a week off their current job to do contract work while we decide whether to make them a full offer."

relet• 13 years ago

This. You have to make your offer pretty attractive, or have pretty desperate candidates for that to work.

flashblu• 13 years ago

That definitely seems tough. Maybe shortening the project time to a couple days (over a weekend?) or a few weeknights would help. From the hiring side at least, the time investment would seem worthwhile, considering you're evaluating a potential team member.

danielweber• 13 years ago

It doesn't work at all for people currently employed.

However, I've given that issue a lot of thought, and I've changed my mind in that there is something really really nice about it: it gives unemployed programmers a lot of breathing room.

You may think that it's really easy to get hired, and in the top-5 cities it might be, but in other markets even very qualified people can't get interviews lined up quickly.

If I were to get laid off tomorrow, I would love the fact that there are employers who, after an initial screen, would take me in for a week at contractor's rates to evaluate me.

1. I would get paid, and after being laid off that would be a tremendous relief.

2. It is much easier to get in the door. Employers always worry that the person they are bringing on won't cut it and will have to be let go. With this test, it is very easy for both sides to try it out without worrying about an expensive "divorce."

spicyj2• 13 years ago

As a complement to the project-based interviews that John mentions, we recently did this at Khan Academy with a candidate and are doing it with another this week. It's difficult because not everyone is free (or is interested) to take a week off what they're currently doing, but we're excited to keep trying this as it's a very valuable signal.

pmiller2• 13 years ago

The benefits don't stop on the hiring side. I would gladly do this for any position it looks like I'm qualified for, for two reasons. One is that it gives at least some idea what the work is like. The other is that I've discovered I turn into an idiot when asked to code in an interview setting while someone else is watching. :P

(Shameless plug: I am looking, if anyone is after a decent Python programmer. Email in profile.)

spicyj2• 13 years ago

We're hiring Python programmers at Khan Academy too, not just JS! I can't promise that you'll get a project-based interview but I promise we'll be friendly.

jyothepro4• 13 years ago

I tried project-based interview, the requirement was very simple build a tic-tac-toe iphone app (player vs device) sometimes device wins and sometimes player wins.

I made this iPhone app(https://github.com/jyothepro/tictactoe) but they rejected me. Reason for rejection was not very clear. I felt I wasted my time :(

visualR3• 13 years ago

Maybe there were a swarm of applicants. You were obviously qualified but they just went with someone else for no reason that reflects on you.

zura3• 13 years ago

Yes, that's the problem with similar open-ended tasks - the assessment is very subjective.

And I have a feeling that I'm gambling - in terms of allocating time vs what I'll get. That's why paid projects make more sense - with defined goals.

In the OP's case - tic tac toe game is completed? playable? that's it - he gets paid, no matter if they don't hire him for some other reasons.

dxbydt• 13 years ago

You were obviously qualified

Not so sure about that. I looked at parent's code submission. In particular, his game logic looks like this - https://github.com/jyothepro/tictactoe/blob/master/TicTacToe...

This is unduly verbose & has newbie programmer written all over it.

Here, I just wrote up the same tictactoe over lunch in the repl - https://gist.github.com/krishnanraman/7591435

Without much thought into optimiziation, refactoring etc., its about 20 lines of actual code. Random TicTacToe is a very simple game. Even if you take that Scala & translate line by line into Objective-C, you shouldn't see an exponential explosion from 20 lines to 400+ lines for game logic alone.

fishtoaster2• 13 years ago

I would argue that take-home project-based interviews are one of the better developer interviewing tools I've encountered. They're realistic work-sample tests, they're low pressure, and they are very high-signal.

They do have the flaw that they might not weed out candidates that are really slow (since they can generally take as much time as they want on a take-home project). They also don't weed out candidates who are willing to have someone else help them significantly with the project.

As such, I've found take-home projects useful in addition to an onsite interview. At a previous company, we would follow up a take-home project with an onsite interview involving a couple 1-hour "code this using any tools/resources you want with no one looking over your shoulder" questions to prove that they didn't cheat on the take-home and that they can get things done in a reasonable timeframe. I always felt it was a pretty useful way of evaluating programmer skill without stupid whiteboard coding.

memracom2• 13 years ago

After the work is submitted the following day, if it looks good then call them in for an interview and ask them what were the weak points of the solution. If the developer is good, they will have worked it over in their head for the next day or two, and will have an answer ready.

Asking questions about the work produced is a good way to identify people who cheated. Ask questions about why and what was your intent with this.

fishtoaster2• 13 years ago

Yep; we did that too. The flow was something like:

- Phone screen with our HR rep to figure out if you're even remotely what we're looking for - Takehome challenge to make sure you can code - Engineer reviews the challenge and, if good, calls you to discuss the challenge and discuss your experience in depth. - If good, come onsite to meet the team, do a couple 1-hour use-anything challenges, and answer a few higher level long-form questions. - Offer.

rayiner2• 13 years ago

I think it's stupid to reject people who don't have an active Github account. Lots of very talented programmers work on proprietary code, especially outside the narrow niche of advertising-driven products where companies tend to produce closed-source software.

jeresig4• 13 years ago

Agreed. (I hope that wasn't in response to the blog post, which was in strong agreement with you.)

rayiner2• 13 years ago

No, just a comment based on Chris Andersen's tweet.

jorde• 13 years ago

Or working on their small startups which might not have time to open source components. It's easier when you have more scale, time and freedom.

visualR3• 13 years ago

What if I have a Bitbucket profile? Or is that not hipster enough?

jeresig4• 13 years ago

You should probably read the blog post. "And to clarify, some people seem to be getting caught up in the terminology – I see many people using ‘Github’ as a placeholder for ‘any OSS activity’, I hope no one is being that literal."

visualR3• 13 years ago

I know, I'm referring to the tweet you are responding to. But let's be honest, Bitbucket and hg do get some irrational hate in certain hacker circles.

adamb_2• 13 years ago

It's interesting to see that employers are essentially requiring more effort from potential job candidates in order for them to be seriously considered, when the tech industry is currently a candidate's market.

saryant2• 13 years ago

I think tech is definitely a candidate's market but the corollary is that the tech market attracts a lot of candidates who can't perform. Real programming exercises are perhaps the only way to weed them out.

Guvante3• 13 years ago

While I 100% agree you need some kind of code based part of your interview practice, there is a large range of exercises that are being spoken about.

It goes roughly from whiteboard problems to sitting at a computer with them over your shoulder to a couple hours before the interview to a full week of work. Everyone knows the lightest form is necessary the big question is how much you can get from increasing the complexity before people get annoyed.

7Figures2Commas• 13 years ago

It may be a seller's market, but it's worth noting that your chances of being hired without going through all of the inane interview processes frequently discussed here are still slim to none at tech giants and hot startups.

The large/hot companies have no problem finding enough candidates to jump through their hoops and, wisely or foolishly, many startups are content to sit on unfilled positions until they find someone they think meets their criteria. So only a subset of sellers have the leverage many think they do.

adamb_2• 13 years ago

I agree that "hot" companies have their own localized demand, but I don't think that's reflective of the majority of companies in the tech field who are now being encouraged to imitate these top-tech players hiring practices.

colechristensen• 13 years ago

I have been approached with the project-interview and it left a bad taste in my mouth leading to me ultimately turning down the interview. Getting a candidate to do work for you in their free time for no compensation is perhaps not the best way to attract and find good talent.

If you want to hire me but want to see some real-world work first, give me an 8 hour contract and a fair wage.

herge• 13 years ago

So you have invested 2+ hours writing a cv and a cover letter, and another 4 to 5 getting ready for, going to and finally attending one or more interviews, and you balk at investing another 4 hours on a project, where your abilities can really shine over the competition?

umsm• 13 years ago

I don't think the argument was about the misc. time it takes to do things. It takes time to commute, it takes time to shower, it even takes time to eat.

The issue is that you're working for a prospective company while not being compensated for it.

yummyfajitas2• 13 years ago

If you want to hire me but want to see some real-world work first, give me an 8 hour contract and a fair wage.

If only there was a website where one could upload and source code one had written for fun. Perhaps even with repo history. Like a hub of git activity or something.

Then employers could simply browse this website rather than demanding these annoying project interviews.

Guvante3• 13 years ago

If you are working at a place that does OSS that puts you miles ahead of anyone working on closed source systems.

Additionally the legality of OSS commits is technically still kind of grey for a lot of people with generic "your code belongs to the company" in their contract with their employer.

steveklabnik• 13 years ago

I read this post as being the same project for everyone, that was extracted from real work, not 'we're going to put this in production once you're done with it.'

jeresig4• 13 years ago

That's correct. We already have an excellent implementation of what we're testing so there's no need to use what the interviewee made (this is simply an interviewing tool).

ateevchopra• 13 years ago

Bravo.! Let me tell you my story. I have been rejected just because I don't have enough activity on my github account. I am a young college senior. My most of "good" projects aren't on GitHub. They are not open source. So does that mean that I don't add up ? No. How can you expect undergrad college students to have 1000s of commits in popular repos ?

nobodysfool• 13 years ago

Come on, a place like Khan academy, who is creating virtual, anywhere classroom, and they don't have remote positions?

jeresig4• 13 years ago

Sure we go! At least 3 people on our dev team are remote, myself included.

zerr• 13 years ago

World-wide?

bitcrusher2• 13 years ago

For a long time, the whole 'Github profile' stuck in my craw, but recently my thinking has changed to more of a 'show us your code portfolio', which isn't much different than a lot of creative job interviews.

When hiring a designer, part of the interview process is showing things that you have done. It doesn't matter if they're real or not, only that you were the one who created them. You also talk about 'your process' when designing those things.

Much the same, I believe that hiring an engineer should be about showing your portfolio and talking to your 'process', whatever that may be. It's a far more interesting way to get to know a person, how they reason and what they're capable of.

l0gicpath2• 13 years ago

Hiring a designer is so much different than hiring a software engineer. And the process of designing should be the least of your worries.

Design is an artistically creative process. Takes talent, I wouldn't hire a designer if I don't perceive the work they've made as beautiful.

But hiring an engineer, there are certain things you can overlook in favour of their capacity to actually build something. I can overlook their code quality, but only because we have a boarding process in which one of the things we make sure a new engineer understands is our quality standard in writing code, the systems we use to enforce that standard from code reviews to pre-commit hooks and lint checking.

An engineer has the capacity to adapt to these changes of both quality and process.

IMHO, you can't teach a designer who isn't talented how to make beautiful things. Albeit beauty is subjective, but there's still a common ground upon which the majority will perceive this piece of art work as beautiful.

bitcrusher2• 13 years ago

I agree that they are different, however it sounds like I'm not communicating my idea correctly.

I'm very much NOT interested in code quality from an engineer. I very much AM interested in how an engineer thinks and approaches solving a problem.

One way to start that conversation is to have a code portfolio that they can talk to. Why did you do this? What were thinking about when you did that? Did you start writing some code then refactor? Did you think about the problem for 3 days before you started writing? Do you diagram on paper? Did you model the data structures before you modeled the application itself?

These are the sorts of questions I'm interested in when I hire someone; They're also the sorts of questions I want someone who is hiring me to be interested in.

Using your words:

To me, writing code is an artistically creative process as well, which takes talent. I wouldn't hire an engineer if I don't perceive their thinking as beautiful

The process for creativity doesn't change just because the expression changes. Compare the process of writing code with that of painting and you'll find a lot more similarities than differences.

PG even wrote a book about it.

pionar• 13 years ago

IMHO, you can't teach a designer who isn't talented how to make beautiful things.

Likewise, you can't teach a developer who doesn't "get it" how to make beautiful things in code.

erobbins2• 13 years ago

I am very much in favor of a low time commitment project interview, something that takes maybe 3-4 hours to complete.

I've written a ton of code in my career, and 99% of it is wholly owned by the companies who I wrote it in the employ of, and I don't have the legal right to show it to anyone much less put it on public display. Some people would say the answer to that is doing OSS/other projects in my spare time... but I have hobbies, and other interests, and don't want to spend my whole life writing code.

Give me an opportunity at no cost to you and low (time) cost to me to show what I can do, and we will both be in a better position to evaluate each other.

aaronblohowiak• 13 years ago

the intensity of the project should ramp with the seriousness of both sides; we start with a 30 minute phone screen then a very simple 30 minute "make these tests pass" and then bring people in for a day of coding/chatting. requiring a project as a first-pass filter is a bit much to ask of candidates and interviewers.

erobbins2• 13 years ago

Yeah, that's essentially what I meant. A coding challenge that takes a handful of hours as a post phone screen/pre onsite screen.. not the first step in the process :)

memracom2• 13 years ago

For all these types of coding interviews I think that the company should state up front how they evaluate it. Is that clicking keyboard in the background someone typing in the code to run it and see if it works? Are they looking for clear communications, do they want you to just solve the problem or is it more important to lay out that whiteboard code exactly the way you would in a source code file?

And what about unit tests? If they give you a problem to code on a whiteboard, do they want to see TDD red green refactor cycles?

And is this the only problem they want you to work on in the hour or is there another one coming when you finish?

Use Project-based Interviews Instead of “GitHub”

Related Stories