Why Resumes and Code Screenings Are Obsolete with GitHire

by heyrhett • 15 years ago

118 84 15 years ago

untog2• 15 years ago

One thing something like this misses in comparison to resumes - what was the effect of the code they wrote?

Sure, I can see that you wrote a kick-ass routine to geocode addresses. What problem was it trying to solve? What benefits did it bring to the end user? Did it have an effect on any measured metrics? This is the kind of stuff people usually put in their resumes.

I'm not that interested in the code a developer writes. I want to know what effect they and their code has on the organisation. It's the same reason I dislike recruiters, because they sell candidates to me on "x years in y!"

MrEnigma3• 15 years ago

To that end, there is more than just perfection of code. It's great to see a developer write perfect 'debt-free' code, however sometimes it's more valuable to get the code out the door in a simpler easier state.

A lot of developers I've worked with have problems getting to a stopping point. For hiring someone the 'perfect' code usually wins out, yet that person might not be the best fit.

untog2• 15 years ago

Absolutely agreed. In fact, these kind of hiring services aren't of much interest to me because I think that a lot of developers are "good enough" coders- but that their personality (be it perfectionism, whatever) is more variable.

I'd prioritise finding a developer that is "good enough" and fits well into a team over a razor-sharp developer that can't work to deadlines or participate in meetings.

dagw• 15 years ago

I often find writing code to be the easiest part of my job. Finding out what the customers are trying to do, what problems they are having doing that and trying to come up with clever ways to solve those with our software is often much harder, and if I was hiring would be a skill I'd care about at least as much as actually being able to write great code. Writing perfect code that doesn't solve a problem people are having isn't that useful from a making money point of view.

jilebedev• 15 years ago

> what was the effect of the code they wrote?

More importantly, the context surrounding the code. I could read someone's code, and find it absolutely abhorrent for any number of reasons: aesthetics, functionality, efficiency, and so on. But if what's written is what the customer wanted, needed, or what was appropriate for the problem, the environment, the cost, or any other constraint that fits into "code context", then by all means it's justified, and I'm unfit to judge the person simply by what's on paper.

In fact, I think a lot of insight about the potential new-hire can be gleaned from their answer to: "Can you describe a situation where you decided that the best course of action was to write 'ugly' code? What do you consider 'ugly'? What was more important than having 'clean' code?"

gcp• 15 years ago

Simple test: myself. Claims I have 5+ years of Emacs Lisp experience. Ahem, I wish. Second test: my boss. Claims he has 3-5 years of C experience (less than me, and less than he has C++ experience). Ahem.

I know these are estimates, but it highlights the flaw: it just looks at your repos and has no insight what's happening. Maybe it would work better if it could detect if your patches are actually being pulled by anone. (That means your code is desirable to at least 1 person).

Eliminating a code screening with this? Come on, don't make me laugh.

rpwilcox• 15 years ago

Same thing happened to me. It claims I have 5+ years of Ruby experience.

But, I learned Ruby in May 2008, because I got a job doing Rails development. (I just looked it up). So, 3 years, going on 4.

peteretep• 15 years ago

I would be curious to know how it gets years of experience so wrong.

sliverstorm• 15 years ago

# Perhaps like so...?

  if ( does_it_compile() ) {
     years_experience++;
     if ( !does_it_segfault() ) {
        years_experience++;
     }
  }

MrEnigma3• 15 years ago

The thing that could be worrying here, is that it could clutter up github with people trying to game it to get a job, instead of actually caring about the project their contributing to.

It seems that in it's current state this could work very well, if this becomes a popular recruiting tool, it won't be sustainable.

thibaut_barrere• 15 years ago

Note to recruiters: don't rely solely on the GitHub profile. I've seen a couple of recruiters miss very good profiles because they were focusing only on that.

MrEnigma3• 15 years ago

Re: Code Screenings are obsolete. I think this is true for some code screenings. However my most recent job, they sent me their own framework, and I needed to make code within that. You're not going to find that on google. Sure I could have had a friend help me, but I think it would have been pretty obvious when I started, and the few that would do that would get weeded out in the first few months.

jmileham2• 15 years ago

Finding exceptional talent on GitHub certainly a solvable problem, but the real challenge for GitHire is the one you can see immediately when you pull up this page:

http://githire.com/order

In the first few reloads, I've seen John Resig, David Heinemeier Hansson, and Yehuda Katz pop up. Maybe they're free to do a quick phone screen?

Here's hoping that there's a lot of secret sauce keeping companies from trying to date supermodels (most of whom are already in a relationship).

mechanical_fish• 15 years ago

That page looks like a teaser. I tend to suspect that the "supermodels" are popping up for the same reason that, when you look at those T-shirt ads on the edges of blogs, every T-shirt is being worn by... a supermodel.

In practice, I presume they offer to try to screen for people who actually want to be hired, not just the ones with big scores on Githubbiness. I note that the very first thing they tell you to do on the "Job Seekers" page is to set your "hireable" flag on Github to true. (I didn't yet realize Github had such a thing, BTW. Thanks to GitHire for teaching me that, if nothing else.)

jmileham2• 15 years ago

I didn't know about the hireable flag either. Cool feature for the genuinely unemployed, but imagine the team morale repercussions of setting your hireable flag to true while in a job you're looking to move on from...

notjustanymike• 15 years ago

Just got "Android". I guess the release of Windows 7.5 really hit the little guy hard...

samstarling• 15 years ago

It looks like they're basically just selling this: http://www.hackdiary.com/2010/02/10/algorithmic-recruitment-...

flixic• 15 years ago

From my estimated experience: CoffeeScript: 5+ years

Ha ha. (the language has been out for 3 years, became known maybe year after release)

tikhonj2• 15 years ago

Actually, that sounds perfect--for every HR post asking for 5 years experience in a brand new language, you can just forward the these results :)

stillinbeta• 15 years ago

I have 5+ years experience with Erlang, a language I know literally not a single line of. It's from a repo (that I forked) that is half/half Erlang, Python.

viraptor• 15 years ago

> Over a million software developers have public repositories on http://github.com, allowing you to see how people use code to solve real problems

That assumes they put code on github. And that they publish the code. And that you know anything about their code.

Then there are those of us who use bitbucket. There's lots of code which doesn't get published because it's not relevant and if it is, it might be a one off project that I'd rather not make public rather than make it look like it's abandoned. Finally, some code just makes no sense out of context - some of the code I'm most proud of were just binary patches to firmware of an old mobile phone with delphi gui for hooking it up to specified functions. Without context that's just an ugly delphi form that does a weird version of "patch" operating on a text file and example files. In context, it's a lot more. Not only I wouldn't expect anyone to understand what that project was doing and why it mattered, it would not get uploaded to github because it was outdated before github even existed.

So while the quoted sentence is still true, I think anyone using this service misses out on masses of experienced developers who simply didn't publish their code on GH, for various reasons. If you want to find a standard <recent language> developer doing lots of opensource work, it might be a good offer though.

The difference might not be clear to the people paying for this service.

maxdemarzi• 15 years ago

I built a similar service that starts with github data and lets you add whatever you want to it. It's more than just language skills you want to hire for.

It uses a nice visualization: http://getvouched.com/visualizations?id=maxdemarzi

Try putting a github username or use the drop down list on the page.

Ultimately you get this profile: http://getvouched.com/users/max/vouches

It uses pagerank + proximity, so if a developer who does have his stuff on github vouches for you, you get all that credit without having to use github.

cruise022• 15 years ago

> So while the quoted sentence is still true, I think anyone using this service misses out on masses of experienced developers who simply didn't publish their code on GH, for various reasons.

To be fair, those are developers that most companies would be missing out on anyway.

techdmn• 15 years ago

I would bet $1 that most companies, like most developers, are not using GH. Most startups, perhaps. Most people who read HN, I don't know. Most companies, that's a pretty big basket.

ErikRogneby• 15 years ago

http://www.workforpie.com checks both github and bitbucket. They also check HN and Stack Overflow to get a rounded out picture. It's an opt-in service though, targeted at the employees not the employers. I've found that their rating system is pretty fair and accurate. I think of it as klout for programmers.

Forking a bunch of repositories isn't going to earn you much there.

dasil003• 15 years ago

GitHub + HN + Stack Overflow eh? I reckon I rank way higher than the likes of Linus Torvalds and other legendary open source hackers. Hallelujah!

LeafStorm• 15 years ago

One major flaw: it assumes anyone you would want to hire uses Github, as opposed to Bitbucket, Gitorious, Google Code, Launchpad, the numerous other code hosts I missed, or hosting their code on their own server.

And if Github becomes "the standard" for hosting software such that everyone looks for code and coders on Github instead of other places, that would be detrimental to the FOSS community as a whole - not only in Github's lack of competition with other code hosts, but also in git's lack of competition with other VCSes.

xxqs2• 15 years ago

don't forget the good old sourceforge and freshmeat.

besides, only about 10%, if not less, of registered github users do actual coding. Most of the users just follow the updates or fork and add patches.

mnutt• 15 years ago

only about 10%, if not less

Is there a number published somewhere?

its_so_on• 15 years ago

I can see when you last checked out freshmeat :)

raganwald• 15 years ago

Tht is not a major flaw of using the service, that is a major flaw of an employer only using the service.

This also true of every hiring source and strategy you can name. For example, what about hiring friends of employees? You miss out on good people who aren't networked into your team.

A good employer will use a variety of strategies for finding prospective employees. This looks like one good way to get a list of five people that might--with further investigation--be worth interviewing.

jrockway• 15 years ago

I don't see public repositories as being a 100% indicator. The information they provide is useful, but I still want to do a whiteboard interview with candidates. There are a few reasons for this. Most of my personal projects are much simpler than what I get paid to do. Many of my repositories contain lots of good code written by other people and it's unfair for me to take credit for their work.

I like the whiteboard interviews because you can observe the end-to-end problem solving. You can see how people behave out of their element. You can see how they approach "an empty page". And you can ask questions that involve more algorithmic insight than "I wrote a Twitter scraper".

mjwalshe• 15 years ago

And in a lot of legal jurisdictions programmers work out of hours actually belongs to the employer if its related to their day job.

DennisP• 15 years ago

You must have a more interesting job than me. Most of my personal projects are way more complicated than what I get paid to do.

fuzzylizard• 15 years ago

As others have stated, how well someone codes is less important than how good a cultural and personal fit they are for the company. I want to know how passionate someone is, how well they do when pair programming, how well they communicate, etc in addition to knowing how well they can code. I can teach someone, who is willing, how to be a great coder, but I cannot teach someone how to fit well into the company culture or with my team.

In addition, I know too many developers who I would hire in a heartbeat that do not have repos on Github and far too many developers that I never want to work with that do have repos.

I also find it curious that on http://githire.com/order, when I refresh the page and see people like 37Signals and Omni show up. Not sure they are looking for new gigs.

smackfu• 15 years ago

What percentage of programmers are doing work out in the open?

psychotik• 15 years ago

I know more than a handful of exceptional engineers who don't even have GitHub accounts. And I know a few really average engineers who are active on GitHub.

As best (if it works like magic even), this only gets you the cream of a small subset of engineers. But if this is the only source you would use to hire, you leave out a large pool of talent before you even begin.

ww5202• 15 years ago

Open source has become a marketing game for some devs.

suyash• 15 years ago

Is this product developed by Github folks?

mattdeboard• 15 years ago

God no.

xster• 15 years ago

I think it would be a great tool but only for a segment of the job market. It's good for finding passionate coding enthusiasts who maintain many open source projects for fun and would be a good fit for a silicon valley type company but for the rest who are just professional white collar programmers who work for big companies, the best code can never be made public and conventional interviews, while not perfect, is the best we got so far.

tikhonj2• 15 years ago

My profile is very flattering: it thinks I have 5+ years of experience JavaScript, Java, Haskell and elisp. I'm a second year undergrad :)

Is there anywhere to submit bug reports? I looked around, but didn't see any email addresses anywhere.

angersock• 15 years ago

So, almost all of the code I've released has been part of a group effort--either something in-house I may not be able to release, or as part of a small team writing OSS.

How does this actually help me? I'm not going to take credit for entire projects, especially not when I'm working on them with friends.

sown2• 15 years ago

This demand that people write tons and tons of code on our off hours is understandable but is getting, I dunno, obnoxious? If not tons, then perhaps something of very high quality. I could write a nifty little linked list implementation but would it really show that I'd be a good programmer for your particular needs?

If you think it does, I predict having code written in off hours is a requirement will result in github being flooded with just horrible code. It's hard to evaluate code in a quick glance. Let's keep it as a bonus type thing so the pressure won't be so high to produce just anything, to torture a text editor until it gives something that pleases the eye and sleights the mind.

Please, keep the industry from moving to make this sort of thing a requirement. I write crappy code in my spare time for fun not because I think it'll get me hired.

----

Not everyone can write code for 50 hours a week and then 10+ more at home, especially if they have outside interests, a family or are just plain tired of it.

Writing code in spare hours is neat, and I do it myself, but I think it needs to be seen as a bonus, not as a requirement.

Perhaps employers are wanting to hire burn-out proof people. I don't think there are enough programmers on the planet to meet employer's demands to hire the best. The implication of wanting 'the best' and asking for code in off hours is that if you don't write code in off hours, you are not the best and not worthy. It's thinly veiled and having been through a few interviews (sometimes I showed them code I had written in off hours, sometimes not) recently, my experiences are probably anecdotal but it makes me wonder.

hello_moto2• 15 years ago

I like your opinion (probably because I'm biased of this trend as well).

Unfortunately, we live in the world where people eat people.

For those who have large number of repos and code written in GitHub, they would rejoice when they see this. Just like Haskell people would feel excited when they see the number of jobs for Haskell programmers out there cannot be met with the supply.

These people will push this idea forward because it is for their best interest (only a few who would keep the purity of the community).

At the end of the day, we're all human beings so we have greeds and egos. We want to have high salary or challenging work or the best work place/culture/free-food and we work to achieve that. Part of the work is to have GitHub repos apparently (used to be StackOverflow karma).

Some hackers dislike the notion of getting interesting jobs/deals because of your rolodex because that would suggest sales/marketing/licking butt mentality. They also dislike it because communication (with real people, not over the internet) is not their strongest suit.

For those who disagree with this wave/bubble, we should create our own. We should push our own ideology. We should figure out how not to be attracted with this wave/bubble.

... or we can stand up and be an entrepreneur (one option, not the only option) and say to hell with this job-seeker mob-mentality?

cydonian_monk• 15 years ago

Exactly. All of it. If you don't like it, fix it. If you can't fix it, replace it (or ignore it). Just because some companies only want to hire people that write code 24/7 doesn't mean all companies have to. And there are some of us that work in closed-source or secure code that simply don't have a public codebase. And an occasional few that are contractually prohibited from writing public code in their "off" time.

(Does having a large codebase freely available make evaluating a potential hire easier? No doubt. I'm not questioning that.)

I'm a programmer by trade but a hardware guy by birth and education. I don't do endless amounts of coding in my off-time, but instead build things. When it comes time for me to move on, if /everybody/ I talk to requires a large open-source codebase for me to even sit in the same room as them - I won't care. I'll find some like-minded folk and create my own job.

oconnor0• 15 years ago

> Unfortunately, we live in the world where people eat people.

I'd be up for changing the world so people stop eating each other.

mcginniwa2• 15 years ago

You are operating under a false assumption. That is that all public repositories (i.e. Open Source) are filled with work by unpaid volunteers. That is simply not the case for a large portion of Open Source projects (and therefore public GitHub repositories).

I've been an Open Source contributor for more than a decade. I would say 90% of my open source contributions have been work where I'm extending or improving an Open Source project's code to suit a paying client's needs.

sown2• 15 years ago

I'm not sure what you mean. I am aware that not all open source is written by unpaid volunteers but that doesn't address my central point.

If the industry demands that everyone write code, be it OSS, personal or just random junk code because they HAVE to, then github/bitbucket/launchpad/sourceforge/etc plus individual portfolios will see an inflation of sorts in code written, some of it good, most of it bad or mediocre. Let's just keep off-hours code as a bonus and not hold it against someone if it is not there. Keep fun-coding for fun! :D

Wouldn't you rather have people contribute to an OSS project because they want to, not because they feel compelled to?

resnamen2• 15 years ago

I'm not even allowed to write code off hours! All my code is belong to my employer. Am I a bad hire?

danielharan• 15 years ago

No, you're working at a bad company.

danssig• 15 years ago

If your company is dictating what you do when you're not working you either need to find out if those clauses are even binding (probably not) or just find a modern place to work. If they really control what you do when you're not on the clock then explain to me how you're not a slave.

The worst part about this is, if companies are allowed to do this they can ensure that some people must continue working for a living instead of becoming financially independent through their own talent.

seltzered_• 15 years ago

I'm waiting for someone to write "Why behavioral interviews are obsolete with facebook" next. </sarcasm>

hello_moto2• 15 years ago

Each community is pushing their own way of establishing the "great developer" presence and calls their solution the best one out there and the rest are obsolete.

Software Developers around the world becomes this victim of a bubble. Be it the StackOverflow bubble (collecting rewards from answering questions) or the GitHub bubble or the blogging bubble.

I'm tired playing the alpha-geek Fire-and-Motion.

bad_user• 15 years ago

I searched for "Ruby" devs from my city. I search for Ruby because I know a couple of good Ruby developers and was curious if any of them popped up.

And the results are pretty bad.

Not only did I get awful results from other countries, the actual 2 results I got from my city are 2 devs ranked as being in the "top 20% of GitHub users by estimated expertise".

One of them has 2 forked projects with NO contributions and is contributing to a third that he just started. He also has no followers. The other is an intern at a local company, with a single project of a few lines of code and no other public activity to speak about. I also searched for them on Google and couldn't find anything else, other than a LinkedIn profile.

Now, I'm not saying that these 2 results are bad developers. But if these are classified as in the top "20%" of developers based on information available online, then the algorithm is doing a pretty shitty job.

To make things worse, I didn't see any personal acquaintance out of dozens that I know have meaningful GitHub accounts and are living in my city. Instead, I got somebody from Istanbul (different country, not even a neighbor) that submitted his configuration files to GitHub and is also in the "top 20% of GitHub users by estimated expertise".

3pt14159• 15 years ago

Same in my city (Toronto). I recognized maybe 4 people that I knew were good and 5 that I knew were junior.

rbanffy• 15 years ago

Like many, my Github repos don't get the love they deserve. Many projects lay abandoned waiting for the day I have the time to finish them.

It seems this system can be easily gamed by putting more commits into your github repos, even if it's not original code, original ideas or even runnable code.

socratic• 15 years ago

Is hiring software developers an evaluation problem or a supply/demand problem?

GitHub provides a great signal for evaluating whether someone is one of the top 100 or top 1000 developers in the world. Just check how many follows or forks they have! But Linus and DHH can work wherever they want, and they're probably unlikely to work in SF for $60--100k plus options in your SaaS startup.

Because of supply and demand, the people who are going to work long hours in that price range seem to be recent graduates. As a result, all of the hiring processes I've seen are geared toward trying to get a vaguely passable evaluation of how good a recent graduate is, using the minimum possible developer time. This leads to the coding interview as we know it: 30--45 minutes long and usually involving a trite programming problem utilizing the most advanced class most undergraduates will have taken, introduction to algorithms.

Personally, I would love if tech companies switched to evaluating based on work samples (e.g., code), but I don't think it makes sense given the demographics that they're targeting as new employees and the amount of time to have a developer genuinely evaluate a coding sample (rather than count followers, etc.).

Is it likely that GitHire and similar will increase the supply of good candidates in the right price range, rather than periodically e-mailing the top developers on GitHub and possibly polluting the ecosystem?

ww5202• 15 years ago

Instead of writing a half-ass project on your free time and putting the source code in Github, build and maintain a product/website/app. It has more creds. It shows you can ship a COMPLETE product instead of some code snippets.

derickson• 15 years ago

Totally agree.

danso• 15 years ago

So...has anyone found a profile that's NOT in the top 20% or better? If I'm in the top 20% for putting in test commits than the software dev world is in dire trouble.

technomancy• 15 years ago

> the software dev world is in dire trouble

I thought we were taking this as a given.

mattm• 15 years ago

I'm in the top 50%

civilian• 15 years ago

On the search page, how do they determine "Hireable"? And why is mine true?

Why is my python professor Hireable: false?

dpritchett• 15 years ago

I'd guess it's the github careers page. You opted in, right?

mcginniwa2• 15 years ago

Hey Githire,

I think the service is good, but I have a couple points of feedback.

When calculating the profile of a user, rather than only looking into repositories under the user's name, you should also take into account the code under any organizations the user is apart of. Most of my coding is publicly committed under an organization repository.

A solution is either to look at all of a user's commits rather repositories or take into a user's organization repositories, too.

Other lesser pieces of feedback:

* The typography and a lot of the images look "fuzzy" to the eye and give off a cheap impression. I would definitely rework those until they give a quality impression.

* Add a "contact us" link in your header or footer or both. I shouldn't have to rely on HN to give you feedback.

aantix• 15 years ago

How do they choose which project to profile? I have many public repos yet they profile the single, oldest project in my profile.

Lots of room for improvement if you ask me.... http://githire.com/profiles/aantix

recursive• 15 years ago

Obsolete? This works only with github, which hosts only git repositories. So the only developers who are hireable use git? Am I getting this right?

erichmond• 15 years ago

Most of the best programmers I know don't even have github accounts because they're so busy writing great closed-source code during the day, and living interesting and varied nightlives that allow them to stay refresh and producing high quality code during the day.

I'm not saying you can't find great developers via github, just that you're limiting yourself to an extremely small pond, even if it seems big.

pvarangot• 15 years ago

Anyone here heard about Ohloh (http://www.ohloh.net/)? This seems like a service they could offer more effectively than this guys, who seem restricted to Github for their data mining.

antihero• 15 years ago

How do they plan to work out if people are jerks or not? Or if they are looking for work, or if they work well in a team? The code itself is a major aspect to a coding job, but not the only thing.

feralchimp• 15 years ago

Best of luck to these folks.

The idea doesn't suck. The current implementation might suck a little. They have their work cut out for them in any case, if for no other reason than the barrier to entry for competition seems pretty low.

They've got two huge things going for them: - $500 is right around the noise floor for a hiring process - Money co-located with mouth. If the client isn't happy, they don't get paid.

resnamen2• 15 years ago

Such saucer-eyed bullshit. Many employers have moonlighting clauses that prevent employees from contributing to OSS projects. All of the companies I've worked for have. If you only go by Github commits, you'll probably be getting only a certain type of applicant and excluding most others.

Github commits might be good supplemental data to have, but if I am to do a fair apples to apples comparison between candidates who might be encumbered by moonlighting clauses, then I'll let the whiteboard coding do the talking.

tferris• 15 years ago

@heyrhett, did you already win some customers? Anyone purchased your 500USD package?

jQueryIsAwesome• 15 years ago

http://githire.com/best : you guys should check if they are actually human beings, (as much as you would like to hire him, android is not a real person).

aidenn0• 15 years ago

Biggest problem here is that the majority of the code I write is not open source, and OSS code I write is very very different than the code I write professionally (so that my employer has no claim on it), so the github version of me is nothing like the real version of me.

Why Resumes and Code Screenings Are Obsolete with GitHire

Related Stories