Hiring Great People (and rules for how we do it)

http://blog.tinfoilsecurity.com/hiring-great-people

by borski • 13 years ago

39 43 13 years ago

blog.tinfoilsecurity.com

tptacek5• 13 years ago

I’m not sure who originally told us about the Sunday test and the Caller ID test, but if they remind me I would certainly like to thank them. These two very simple ideas have kept us from making some very bad hires, and have convinced us to make others that were great. The Sunday test is as follows: if the candidate is in at the office alone on a Sunday afternoon, would you be more likely to come in and work with them? That answer has to be yes. We want to be excited by our colleagues, and not feel like we’re holding them up or they’re dragging us down. The Caller ID test is equally important, and is as follows: if the candidate calls you on a Friday night and wants to spend time hanging out or working, are you more likely to pick up or let it ring to voicemail? If you wouldn't pick up the phone, that candidate is the wrong hire.

Argh. These rules are toxic. You are unlikely to predict your best hires. A lot of evidence suggests that engineers are uniformly bad at making subjective judgements about future performance. The "Sunday" and "Caller ID" rules basically put a deceptive veneer on the most subjective of all criteria: the "gut check".

That's bad enough, but both rules also have a pernicious culture effect. In companies full of 20-something Red Bull drinking 11a-9p gamer nerds, the rule gets you more gamer nerds. In companies full of 30-somethings with kids, you get more people with kids. This is exactly what you don't want to have happen; a real culture adapts to any positive competent person you find. Good cultures figure out down the road how to get people to hang out outside of work. They also accept people who just don't want to spend their out-of-work time with their officemates. Why would you want to filter those people out? That property has no known correlation to ability or performance.

Finally: do you really want to be the hiring manager who says out loud that you have a rule built into your hiring process that involves people working on Sunday? Don't make people work on Sunday.

The sooner we kill the "Sunday rule", the better.

ainsleyb• 13 years ago

This is an interesting take. For us, it's not about age, but about personality. We have a financial controller who is 20 years older than everyone else on the team and we'd bring her on full-time in a heartbeat if we needed a full-time controller. She has a great personality, is fun to have in-house, and knows what she's doing.

How I look at the Sunday test is less of a "are they like me" and more of a "will I get along with them many hours a day"? We work anywhere from 7 hours a day, up to 18 (especially during major code pushes) - we try to avoid this as much as we can, but sometimes (for us at our stage), it's inevitable.

I do have to enjoy working with my colleagues, and someone for whom I won't be willing to come in on a Sunday has a higher chance of bringing me down on a regular basis. That doesn't mean we work Sundays (we're typically in the office M-F), but it's important to be able to get along with people and it's a good litmus test, imho.

tptacek5• 13 years ago

I'm not here to tell you to hire people you don't get along with. But you shouldn't regiment hiring rules that are based on things like "culture fit" or "getting along", because they code for subjectivity and culture bias.

You should certainly rule out people who are disagreeable, or who immediately send signals that they'll be logistically difficult to work with. But you should base those decisions on some kind of objective metric. That requires some up-front thought: what are core hours? Do we avoid meetings and thus require lots of continuous written feedback? Do people need to be comfortable with our chat system?

But as you think of those metrics, start to worry when you get to things like "will this person usually be available to review code prior to deployment on a Saturday night?".

freework3• 13 years ago

I'm the opposite. I don't want to work with people who I "get along with". I want to work with people who will tell me I'm an idiot when I do something idiotic. I want someone who puts all their effort into creating great technology, not someone who puts all their effort into being liked. I have thick skin, so it doesn't bother me to work with an abrasive person.

That, of course, only applies to programmers. On the other hand, if I'm hiring a salesman, I'd want that person to be the kind of person who puts all their effort into being a likable person. The same goes for hiring managers. Being a likable person will get you far in many positions in a business. Programming is not one of them.

RougeFemme• 13 years ago

Unless a person is at one extreme or the other (on a trait spectrum), I'm not sure I'd be able to tell based on an interview (or series of interviews) whether I would want to hang out with him/her on a Sunday. An extrovert who reads as "fun" within a few minutes may be someone you know immediately you'd like to hang out with. What about the person who is an introvert and reads as "fun" within a few hours, days, weeks. . .? Within a few weeks, you would have realized this is someone you could work with, but you would have already made the decision not to hire this person,

georgemcbay• 13 years ago

Having to work 18 hour days is an inevitable consequence of working 18 hour days.

Unlike tptacek, I love the "Sunday test", because it acts as a filter for me -- I'd never work at a place where that question was asked in a non-sarcastic way.

I have to admit though that I don't really get why you'd use the "Sunday test" and the "Caller ID" test, since both are differently worded ways of gauging exactly the same thing. Seems like you could simplify it down to the "Is this guy (or girl) an asshole or what?" test, but I guess just asking it outright like that doesn't give you the same process-hipster cred of having heard of these other "tests".

freework3• 13 years ago

A lot of times when companies say "getting along with us" is an important train in candidates, it often gets manifested as "agrees with us on everything" in practice. In other words, the only way to really fail the "Sunday test" is to disagree with the boss on one too many occasions.

bensedat• 13 years ago

(I work with/for Borski)

Disagreement is one of the things that I think we tend to be pretty open with at Tinfoil. Not everyone has to agree with a decision, but they always get airtime to voice any concerns and for those to be addressed or at least considered/acknowledged.

dools• 13 years ago

I agree completely. I've worked with a range of people across a pretty broad "social spectrum" (from very close friends, to acquaintances, to family etc.) but the one guy I've worked with the longest was from a random job ad and, although we get along just fine, we're completely different people. In fact, because of that I have learned a lot that I probably otherwise wouldn't have.

nkohari• 13 years ago

Fair enough, you've said what not to do. What would you recommend instead?

sixothree• 13 years ago

The rules are incredibly shallow. And to blog about them this way seems narcissistic. I hope the community takes this post as informative so prospective employees so they can steer clear well in advance.

codewright• 13 years ago

I met these guys at a party once. (I'm in Mountain View)

I ended up ranting at the CEO and CTO about how their hiring practices were excluding the best people because they were treating candidates like their time is free. The claim was that they wanted to hire the best people, yet their hiring practices were dissonant with this.

They said it was an acceptable trade-off to not have access to the best people in order to ensure they got people that were going to do whatever they asked them to do. At the time they were trying to hire a designer, but the practices carried over into engineering hiring too.

They want people too foolish to value their time, yet smarter than them. Riding that knife's edge isn't easy.

peacewise2• 13 years ago

How about leaving your employees alone on Friday nights and weekends.

borski14• 13 years ago

We don't force any of our employees to hang out with us (or work) on Friday nights and weekends. Often, I want to hang out with the team, because they'are awesome, but none of it is forced. We don't work weekends except in (rare) major pushes.

tptacek5• 13 years ago

I love hanging out with my team. But I'm more proud of the fact that you don't have to hang out with us for us to spot your ability and extend an offer. And so we have some people in the office who hang out together, and some people who get in early and leave at 4.30. I'm very proud to have a culture that works for both kinds of people. It's one of the nicest things Patrick McKenzie ever said about us: they're friendly, happy people who get social license to join the Dark Side, do smart stuff all day, and then go home while it is still light out.

peacewise2• 13 years ago

In that case, I don't think it is super effective to focus on cases that rarely happen.

Interesting read: http://blog.prettylittlestatemachine.com/blog/2013/02/20/wha...

loteck2• 13 years ago

It seems like this could more accurately be entitled: Hiring Likable People. The tests they are touting are basically: do I want to be around this person? That is, indeed, a good criteria to consider when hiring.

They also claim to want to hire people "smarter" than they are. I am not running a business, but isn't the danger here that "smarter" people take your projects/clients in a direction that your management is unable to provide leadership on, and then just bail, taking the project/client with them?

borski14• 13 years ago

Interesting point. We hire people that are smarter than ourselves in some domain. Typically, I'm still better in some ways than the rest of the company, in certain domains. We just need to be impressed by their intelligence consistently. Does that make sense?

loteck2• 13 years ago

It does make sense, especially for your boutique business. You're looking for folks that bring something to the table, who are likable. While hardly revolutionary, I'm glad it's working for Tinfoil.

SonicSoul• 13 years ago

i think i see where you're going with this. in a small company you do want to work with people you like being around.

the test metaphors can use some work though ;) i don't see how Monday test would be any different than Sunday test? you're still working with that person right?

i've been in a situation where i've made recommendation to hire because the candidate aced the interview, but i knew i had no interest in spending too much time with them on personal level. it was conflicting to say the least, but in the end he ended up making some good work contributions. It is almost impossible to tell how well the person will behave long term (i don't care what Gladwell says).

borski14• 13 years ago

Right, this works for us because we're such a small team. And you're also right that Monday is no different - we didn't mean for it to be taken literally, but it's a way of checking whether the person being a part of the team excites you or not.

epoxyhockey2• 13 years ago

I used to work for a small company where most employees enthusiastically spent time hanging out with the founder on various weekends, going to sporting events and dinners.

The critical point to be made is that the founder signed our paychecks. Until after the company dissolved, you couldn't tell who was doing it for office politics reasons or out of genuine friendship. Today, only 15% of those employees still speak to the founder.

In my opinion, it's best not to approach hiring in the same manner as dating. At the end of the day, it's a business arrangement and it's about getting high quality work done in exchange for money.

stephengillie• 13 years ago

What are your criteria for hiring people smarter than yourselves?

borski14• 13 years ago

They need to have done something impressive. It's honestly not too far from what pg has described looking for in companies to fund - we're looking for people that impress us and we can learn from; not necessarily in /everything/, but at least in /something/. For example, hiring someone who has a /ton/ of experience with administering databases (something I personally am only 'ok' at) would be impressive.

borski14• 13 years ago

For the record, this apparently isn't novel. These tests are very similar to the 'airport test': http://www.asanet.org/journals/ASR/Dec12ASRFeature.pdf

tokenadult• 13 years ago

What kind of work-sample test do you look at for hiring the people you want to be sure are smarter than you are?

https://news.ycombinator.com/item?id=5227923

borski14• 13 years ago

We typically provide a realistic scenario in which we work with the person to see if they are impressive. As in, we work with them on a project unrelated to Tinfoil for a few hours to a day, and see how it goes.

tptacek5• 13 years ago

Do you provide the same work-sample test for every candidate, so you can build a history of how people handle it and then look to see how your best hires did? We did that (it was Cory's idea, not mine, for what it's worth) and it's been immensely valuable.

If I could get one new hiring process adopted in tech, it'd be that one: come up with a graded work-sample test that will generate a history you can mine for data down the road.

epoxyhockey2• 13 years ago

Unless working weekends is a business necessity and asked of all applicants, it is illegal/inappropriate to ask in an interview.

Source: http://www.gsworkplace.lbl.gov/DocumentArchive/BrownBagLunch...

borski14• 13 years ago

Right, and we never ask it. The metaphor is not a literal one.

seanmcdirmid• 13 years ago

IANAL, But just documenting this in a blog post makes it public record and potentially opens you up to lawsuits to anyone you didn't hire in the past.

freework3• 13 years ago

I'm going to tell a story that describes why I don't like "Sunday tests".

Before I became a professional developer, I was a professional pilot who programmed as a hobby. Right after graduating college in 2006, I got a job right out of school working for a flight school. The school specialized in training Chinese people to fly.

Anyways, this was back before the economy exploded. There was a shortage of instructors at my school. I was assigned 4 students that I would be teaching mostly one-on-one. Three of the four students were really cool and very likable. The fourth I didn't like at all. Lets just say we didn't get along well. The guy was very difficult, and hard to communicate with. The other three were a joy to work with.

If I had the luxury of dumping that fourth student onto another instructor, I would have. But this was back in 2007, and the economy hadn't tanked yet, so there was an instructor shortage, and I had to grit my teeth and deal with this guy.

Fast forward to a month later. The tables have turned completely the other direction. The fourth guy who I hated at first was my favorite student. The three students I had liked at the beginning now were the ones giving me the most greif. The extroverted easy to work with likable guys were pleasant to deal with, but they weren't very smart, and I had a lot of trouble teaching them stuff. The fourth guy was hard to deal with, but he was extremely smart and I barely had to teach him anything, he learned everything on his own.

Once the economy got bad, the student/instructor ratio fell. The flight school hired a bunch more instructors and got less students. Every instructor got 2 students instead of 4. I made it a habit to go around to all the new instructors and ask them if they have a student they they don't really like working with. 4 times out of 5 the instructor would tell me "yeah I have this one guy..." My last 6 months at that job I had 8 students that basically taught themselves. That was easy money.

I've found the best way to deal with unlikable people is to just let them go off and do their own thing. The act like that (unconsciously) because they resent you getting in their way.

hkmurakami• 13 years ago

A friend of mine is like the 4th student you describe: very smart but comes off as standoffish and cold until he warms up to you. He was actually let go from a well known (everyone here would know the name) startup in SF/SV wiht a noted brogammer culture after a few months since he "wasn't a culture fit". He's now thriving at a place that suits his character much more.

In the end I think both parties "won" by his dismissal, but companies hiring for culture fit should be aware that they're willingly constraining their talent pool by doing this. (I don't think it's necessarily wrong either -- I expect to have some form of latent character filter if I'm in a hiring position as well. However, I'm much more fond of the introverted, "difficult" types than the average person).

alan_cx• 13 years ago

This is all about "like", what about "respect"?

borski14• 13 years ago

Just updated the post to clarify:

"[UPDATE] Just to clarify the above, we do not mean that we ask candidates these questions. These are questions for the existing team when evaluating a candidate, and it isn't about forcing people to come in on a Sunday, or convincing ourselves that candidates are just like us. It is simply about hiring people we would want to work with, be around, and help / be helped by, no matter what the occasion."

Hiring Great People (and rules for how we do it)

Related Stories