How to get hired (or, 'The silly story of interviewing in the valley')

http://trapm.com/how-to-get-hired-or-the-silly-but-adorable-st

by sgrove • 15 years ago

245 171 15 years ago

Aqua_Geek2• 15 years ago

Are these kinds of interview questions really that prevalent? From the hiring side of the table, I can understand wanting to quickly know whether somebody knows some of the "basics" like data structures and whatnot. But I can't help feeling that you're needlessly limiting your applicant pool to only those who got a CS degree (and paid attention during the process).

Why is so much emphasis placed on the theory of how linked lists work (for example) instead of the practical application of where they'd be used?

Edit: Big O notation is great and all, but shouldn't the emphasis be more on shipping than optimizing? You can optimize your MVP to death, but if you never finish it what good does that do?

robobenjie2• 15 years ago

We use these types of questions to allow an applicant to show us that they can, in fact, write a simple program (see http://www.codinghorror.com/blog/2007/02/why-cant-programmer...) and we use programming puzzles to make it more interesting for the bright candidates.

Stwerner• 15 years ago

Are there any good sites out there that have a lot of these types of problems that are asked in interviews? Sites like http://spoj.pl seem to have problems that are more longer form. But it would be fun to start practicing solving the shorter ones that are asked without the hassle of interviewing to get them.

ejames• 15 years ago

Yes, interview questions like this are common.

The problem with "practical application" questions is that the answer almost always comes down to "it depends". The correct optimization or fix is specific to the actual thing you are doing; that's what makes it practical.

In order for an interview question to be useful, both the interviewer and the interviewee must be able to answer it. The applicant doesn't know the interviewer's product well enough to make accurate technical diagnoses. The person doing the interviewing doesn't know about the applicants' products either. Bogging the interview down in specifics to fill in the background is usually not helpful - plus, it gives lots of opportunities for the applicant to bullshit the interview by listing a bunch of technical details that might not actually have been accurate or important.

Theory questions are favored precisely because they are not dependent on nuts-and-bolts practical details. "Reverse a linked list, writing your code on this whiteboard" is general enough that most applicants can answer.

I understand that people who lack CS degrees might not know the answer. Interviewing means accepting that you don't get a 100% accurate result. I would only switch away from the "basic CS question" problems if I was convinced that some other question has a higher accuracy rating, and I'm not sure of that.

Aqua_Geek2• 15 years ago

> The problem with "practical application" questions is that the answer almost always comes down to "it depends."

Agreed. Talking about application of theory is way too vague and leaves way too much room for bs. But I think my original point was unclear. I'm not suggesting that interview questions be "When would you use a linked list?" but rather "Let's hack on something using a linked list (something beyond just simply reversing it)." Like the article mentions, once you've built a linked list before and have optimized (and reversed) it to death, there's little to be gained from doing it again for somebody else.

qdog• 15 years ago

Yes, most interviews are like this. "Here, solve problem X on a binary tree on the whiteboard, you have 15 minutes, go!"

I'm not good at it, I have to study for a while before going on interviews. The whole "study ahead of time to look like you are a genius coming up with the best answer thinking out loud" routine seems to work, although it feels dirty to me.

When I am the interviewer, I ask people simple questions about what mistakes they have made and how they worked through them, because in my own experience the things I do to solve problems I have created for myself are usually the most enlightening. My take is if you have some obscure bug you spent a few days/weeks solving and you can relate it to me, I tend to believe you, but if you don't have stories about this then you probably weren't really doing much programming.

Big O is a little odd, because it can be very important or not very important, depending on what you are doing. The ability to say "I know this has bad runtime" and then deciding if it's either worth fixing now or noting for later is important.

cullenking• 15 years ago

Because these things are immensely useful, but only if you know about them and can recognize when they will help in solving a problem. For example, I ended up implementing a linked list recently to efficiently allow editing of an ordered, traversable set of lines on a google map (a drawn route). There are other ways to solve the problem, but a linked list was the simplest as well as most performant.

Anyway, moral of the story is fundamental CS concepts come in handy if you know them enough to recognize how they will make a problem easier. Bonus points if you can whip it out in a few hours, which may be difficult if you have never done it before.

gcp2• 15 years ago

But I can't help feeling that you're needlessly limiting your applicant pool to only those who got a CS degree (and paid attention during the process).

Why is so much emphasis placed on the theory of how linked lists work (for example) instead of the practical application of where they'd be used?

Edit: Big O notation is great and all, but shouldn't the emphasis be more on shipping than optimizing? You can optimize your MVP to death, but if you never finish it what good does that do?

I feel the third paragraph here conflicts with the second one. In effect, understanding Big O would help you identify where linked lists are applicable. You seem to want to be given an exhaustive list of where they should be used. Sorry, but that's not how programming works.

Secondly, I take issue with the "limiting your applicant pool to only those who got a CS degree". If you're going to work in the CS field, I think its only normal you are asked question to see if you've got a grasp of the foundations of your job. You don't need a CS degree for that, there exists such a thing as self-study, and I think its a very reasonable thing to require if you're trying to transition between fields.

jaylevitt• 15 years ago

And I take issue with "If you're going to work in the CS field." You're assuming that all programming/coding/software engineering/whatever jobs are in the CS field. There's a lot more out there.

philwelch• 15 years ago

Isn't the MVP stage a little early to hire developers?

abahgat• 15 years ago

I guess that one of the advantages of such an approach is standardization, in particular if your company is big. Once you get a significant number of candidates, it may become more important to have some "objective" and quantitative way to rank your candidates by performance than to get a true understanding about how likely they are to get things done (how would you "measure" that?). At that point, it becomes more important to ensure uniformity of judgement across the pool of interviewers. Isn't that why they often ask programming puzzles: so that they can compare performance. If the price to pay for that is that some valuable candidate won't get an offer, they can live with that. At least, that's how I see it.

Of course the argument for startups will be different :)

overgryphon3• 15 years ago

I'm not sure standardization of interviews is desirable in a large company. My experience is that large companies hire by team, so interview candidates get widely varying experiences depending on what team interviews them. This way candidates can be considered not solely based on technical knowledge but also on how well they fit with a particular team.

hkarthik• 15 years ago

I think this is more typical of positions being filled in Silicon Valley, where it appears that more programmers tend to have CS degrees. Or there are enough people with CS degrees that the rest of the community feels compelled to know and understand at least second year level CS theory (Big O and Data Structures).

I work in Texas, where this is not the case, and I've never been asked to implement a Linked List in any language. I have a CS degree from 10 years ago, and I can count on one hand the number of CS graduates that I've worked with in the past 6 years.

dangrossman• 15 years ago

It's legal ageism. Asking questions whose answers will be fresh in the minds of recent CS graduates is a way to legally screen out everyone else, no matter how irrelevant the Big-O runtime of merge sort is to the job.

jrockway7• 15 years ago

Personally, I always have to wonder how often someone programs a computer if they don't know the asymptotic complexity of a merge sort. It seems like something someone would either pick up at some point in their career, or be able to figure out on their own at the interview.

(Come to think of it, I never took an algorithms class in school, and I know the answer to this question.)

andrewflnr2• 15 years ago

It's hard for me to imagine knowledge of Big O notation being a major limiting factor. I gleaned at least a superficial knowledge of it cruising Gamedev.net in high school, well enough that they didn't throw me out when I had to use it for an internship interview. It seems reasonable to expect people to have at least seen it. But perhaps I was an aberration.

overgryphon3• 15 years ago

Few of these types of questions ask anything beyond the first year of a decent CS curriculum. If interviewers wanted to limit themselves to CS students, they would ask about virtual memory and require candidates to write proofs of algorithms concepts on the whiteboard.

tikhonj2• 15 years ago

I certainly agree that writing proofs and algorithm concepts is a heavy bias towards CS grads. The issue there isn't so much with the concepts involved with the particular problem as with the structure of a proof and the approach to those problems in general. Once you've learned about induction formally, it's much easier to use it in a proof, even if you can understand what it is anyhow.

On the other hand, I'm not sure virtual memory is such a concept. I'm under the impression that virtual memory is a topic tons of people are talking about and that it's hard to get away from it even if you're a programmer without a CS degree. Of course, this could just be the result of my being a Linux enthusiast.

timr2• 15 years ago

"Then, acting as though this was the first time I'd seen this problem, I would ask if it was ok if I thought aloud as I worked my way through the problem on the board. I'd mumble to myself about moving-this-piece-over-here and-now-we're-going-to-get-this, and lo-and-behold, I accidentally solved it in constant memory space, in C - a language I didn't even claim to be particularly good at! Only someone with amazing problem solving skills would be able to do that their first time thinking through this problem so earnestly!"

All this tells me is that tech interviewing is horribly broken. What we have here is a story of a guy with a little bit of experience who has (if his story is true) successfully learned how to game the interview systems at dozens of startups. If you interview people, this article should terrify you. You can argue that it isn't "gaming" if the guy turns out to be a good employee, but that's wrong. If he can trivially game the system, so can anyone else with the ability to go to dozens of interviews and remember questions.

There's a fix for this problem: make the interview about more than the answer to the stupid question. If the candidate answers your question a little too quickly, ask something harder. Chase down a detail. Pick an experience on the resume (and not an obvious one), and dig as deeply as you can. Good candidates know the details. Bad candidates don't have details you can chase.

If you're interviewing correctly, no candidate should be able to come in and snow you with some regurgitated C code from previous interview experience, because you'll be able to out-flank them at every attempt to cough up a line. Unfortunately, there's a corollary: you need to know the subject better than the person that you're interviewing. If you don't, you have no hope of screening for good people.

sgrove10• 15 years ago

Yeah, that was more or less the point of the post. For applicants, don't feel nervous, it's a silly game just to see if you know some of the puzzles.

For people hiring, go ahead and use this as a first-pass filter, but realize that it's easy to game (by someone who's trying hard and learning, so there is a positive signal in that). It's going to be important to go through and work with them for a period of time in a let's-date-before-we-marry relationship. As an employee, I always insisted on a contracting relationship before an employment relationship.

Like I said in the footer, Heroku really seems to do this the right way, and it shows. Kudos to them.

And everyone else needs to step up their game.

groby_b• 15 years ago

In all my history of interviewing people, I've always found there's almost no info to be had in correct answers to any kind of question. You learn about a candidate once you guide them into territory where the fail.

Both where that line is, and how they handle the failure tells you a lot.

(Caveat: Make sure you're keeping it on the fail/no-fail edge, and you treat the candidate with respect. The point is not to show them that they're wrong, but to explore the boundaries of their skill set. Without being mean is a major goal here)

And if you can't push the candidate there: HIRE THEM. Right away. If they know that much more than you do that you can't get them beyond their limits, they are a fantastic addition to your team. (Provided they match personality-wise, too)

kenjackson6• 15 years ago

I have some simple baseline questions I ask everyone. Getting it right just means we continue the rest of the interview, but doesn't get you a job.

I almost always have a programming question I just made up 30 minutes before the interview. I make sure that I can easily do it in 10 minutes and then ask it. It really isn't important to me if they get they answer, but I want to see what questions they have, and how they think. Good candidates will usually get it coded, but some may get hung up on something early on, but still make good progress.

If they come in with a precanned response to my question then they're either a mindreader or can time travel. In either case, I still went them on my team.

bialecki• 15 years ago

>> I almost always have a programming question I just made up 30 minutes before the interview. I make sure that I can easily do it in 10 minutes and then ask it.

This is a lot easier said than done. Do you really do this and, if so, any tips on how? (I'm guessing variations on a core set of questions.)

I usually don't mind asking a questions someone may have already heard because there's not really a right answer, but I just want to discuss a problem to see how you think through it.

prostoalex• 15 years ago

Unless the end result is a completely incompetent applicant getting an offer, I wouldn't call the system "horribly broken". Very few companies are in the business of reversing linked lists, and hence make binary decisions depending on the outcome of just that one question.

I don't think gaming the system is that trivial - an interview is a conversation, so things like computational complexity, runtime restrictions, data structures, implementation differences in various programming languages, and other things are bound to come into conversation. Now, if one can maintain that conversation as it moves along, and seem like a pleasant person to work with, it seems that they're no longer purely "faking".

tedunangst8• 15 years ago

Exactly. I've had lots of candidates basically pass the initial question and then sink their own battleship by volunteering some running commentary that happened to be totally wrong. Memorizing enough crap to impress an interviewer about even a basic reverse the linked list question without understanding it is actually quite hard. It's not enough to remember to drop the terms stack and heap, you have to remember which one is which.

bigbird• 15 years ago

I also don't think that what the article describes is "gaming" the system, except maybe for the part about pretending to have not heard the questions before.

Generally, the reason I ask such questions of candidates is to make sure I'm dealing with someone who can actually analyze a problem and get a reasonable result. So if they nail the questions because they took time to study and get good at interview-style brain teasers, that's actually fine with me - it proves the person can execute a plan to get correct results.

I'm not sure I'd call anyone who successfully "games" the system incompetent.

skybrian• 15 years ago

All that is true, but there's a simpler fix. If they're not reusing each others' questions then it should be significantly harder. Nobody should be using "reverse a linked list" anymore - that's a really old one.

steve89186• 15 years ago

That's not a simple fix. If you don't use "reverse a linked list" anymore, then you start asking increasingly harder and harder questions, and the bar gets ridiculously high, to the point where everyone needs to know how to solve NP-class problems within 45 mins. And even then the system still can be gamed by simply memorizing more and more questions. It's a lose-lose situation.

Evgeny• 15 years ago

It's not really gaming the system. He solved the problems, himself. He asked questions, learned and then came up with more and more effective solutions. Aren't these exactly the traits an employer should be looking for? Didn't he put a lot of effort and time?

Here's my own example. I've been recording some of the problems I have to deal with at work, and some of the things I come across in my spare time in the form of a blog, for the last 3 years (with some gaps here and there). I recently started looking for a better place to work. I included the link to my blog in my resume. I got an offer recently, it will be a substantial increase in salary. I only interviewed with ~5 companies to get an offer, that's my easiest offer and shortest job search ever.

Now, would you say that I 'gamed the system' because I had a blog, and most don't? I guess you can say so, because when I started my blog, I had that in mind - it may be useful next time I look for a job. But I did not mindlessly type stuff in - I made sure I understood it as much as I can, and quite often writing the solution down helped my understanding. Same with this guy - he did not just memorize how to do it.

I had one YC company ask me to create a linked-list structure in ruby, and then reverse it. I did it recursively, mentioned that it would blow the call stack, and then re-implemented it iteratively. I also remember there was some problem where I used symbols rather than strings and that would cause some bad memory leaks over time in a long-running process. - From this passage, he might not be the "top 1 percent" everyone tries to hire, but at least top 5%, that I'm sure of. Or maybe he gamed my impression too.

kamaal4• 15 years ago

The definition of gaming the system is different here. These days unless you are implementing a library yourself. You are probably never ever going to even reverse a linked list ever in your life. Or deal sit down with a math text book when you code. Those days are probably gone in the 80's.

That means asking these questions is akin to you taking an approach to judge an candidate on things which have nothing to do his job. That's why so many programmers don't about all this. The reason is for 99% of the work they are likely to do they don't need to know it.

Under such circumstances when you ask these sort of questions the only way candidate can deal with them is not by actually learning how to solve them. But learn them just for the interview sake.

So you end up hiring some one on a criteria doesn't matter. And the candidate manages to beat you by system of work which defeated your original means of judging him.

I think this is what gaming the system means.

hboon• 15 years ago

Would that be considered top 5% already? I thought it's pretty obvious. Honest question.

kakuri• 15 years ago

It sounds like in this case, this guy might be a good candidate - look at all the effort he put in and his ability to remember things, think critically, and learn.

On the other hand, there are plenty of competent programmers not willing to put up with this nonsense, and the companies are going to miss out on them.

huhtenberg• 15 years ago

Going through the details is the way to estimate candidate's true knowledge of a subject. Ask an easy question, then ask a very hard one, repeat depending on answers, i.e. basically do a binary search:

  - Do you know what SSL/TLS is?                       general
  - What's the maximum size of SSL protocol record?    specific
  - What's a cipher suite?                             general
  - How is DH-anon suite is set up?                    specific
  ...

  - What is a linked list?
  - What is offsetof (and how is it relevant)?
  ...

  - What is TCP?                                       general
  - What is TCP/Vegas?                                 specific
  ...

This is the quickest way to converge to an actual level of expertise, and it really helps with filtering out people with bullshit resumes and those who cheated their way through an offline problem solving. We would say something like "You are not expected to answer all questions, we are just trying to understand what you know and what you don't know."

arghnoname• 15 years ago

There's always a downside to every approach though. Some people get flustered if they have to say "I don't know" too often, so instead of bouncing from tough to easy, I'd probably prefer to progress from easy to tough.

People will feel confident until they hit their limits. Their limits are what you are looking for (presumably), not their ability to think clearly when they are scared they just blew the interview on a question you didn't really expect them to answer.

spiffytech• 15 years ago

I recently interviewed for a position with this sort of technical interview. I felt it worked very well. The interviewer was able to cover a wide range of topics in varying depth, and could see what I could easily recall and what I struggled with.

With the right set of questions, and some hints when the candidate struggles, you can very effectively gauge a candidate's limits on a subject, and see how they try to puzzle things out when they don't know an answer but know enough to take a good guess.

The key distinguishing factor of this method, IMO, is a candidate cannot memorize enough answers to bluff their way through a good questions list. If a candidate really is an N out of 10 with technology $FOO like they claim, they should have no problem answering most/all of your 50+ N/10-level questions for $FOO (my technical interview consisted of ~2 hours of this sort of interrogation for a straightforward junior web dev position. Interview length, in this interviewing style, is important for ensuring you've exhausted a candidate's book knowledge).

nadam2• 15 years ago

"you need to know the subject better than the person that you're interviewing. If you don't, you have no hope of screening for good people."

I think it is not necessary and not possible that the interviewer know more about every subtopic than every candidate. It is enough that they have comparable knowledge in most topics. Good candidates can very easily know more about some parts of the subject (and maybe less about other parts). Being a good interviewer is a really tricky business, because good interviewers should see the strength of candidates and should see if the candidate know even more than them about some subtopics. I've seen some bad interviewers who asked only the very narrow topics they knew well, and basically wanted exact copies of themselves.

ajays• 15 years ago

Tell me about it. I have nearly 20 years of experience in CS, and hold an advanced degree. I have 5 years of experience in a very niche field which is relatively hot. I was approached by Facebook for this specific purpose. They invited me in for just a "chat", asked me if I'd be interested in working with them.

Obviously I said yeah, I'd be willing to talk. They set up an in-person interview. Just 45 minutes they said.

So I show up, fully expecting a discussion about that area, my background, etc.

Instead, I get a random coder who wasn't even making eye contact when asking questions. And the questions? Elementary CS stuff that any undergrad would know.

Now, obviously I know them too; heck, I taught Algorithms courses in grad school.

But why waste my time with this crap? You can hire almost any fresh graduate from a top-40 school who would do a halfway decent job at answering those questions. What does asking a PhD "how to print the levels of a binary tree, one per line" tell you about his work? (That wasn't the question, but that was the level of questioning).

The sad part is: I can waste a couple of weeks of my life, go to Glassdoor etc. and basically memorize these questions. I'll probably stand a good chance of blowing the interviewer away by pretending to solve them on the spot. But what does that tell you about how I am at real-life problems?

jrockway7• 15 years ago

The cost of hiring someone who can't do this stuff makes it worth the risk of putting off someone like you. Sometimes you have to ask rather than be faithful if you haven't personally worked with the person you want to hire. A shitty programmer will kill an entire team.

tedunangst8• 15 years ago

I've interviewed a few people who thought the elementary questions I was asking were beneath them. I insisted, politely, that they humor me, because it's something we make everybody do, nothing personal. They failed. Hard.

Some people are probably bullshitters, but my theory is that some are just people who used to code (a little) but have since moved on to making architectural diagrams for so long they've forgotten what it's like in the trenches. Unfortunately, if you're interviewing for a job in the trenches, you need to demonstrate the ability to do the job.

It's entirely possible they screwed up, depending on why they asked you in, but once a company starts letting in people they "know" can code without verifying it first hand, things go bad fast.

kamaal4• 15 years ago

I can waste a couple of weeks of my life, go to Glassdoor etc. and basically memorize these questions. I'll probably stand a good chance of blowing the interviewer away by pretending to solve them on the spot. But what does that tell you about how I am at real-life problems?

Most of the so-called algorithm experts I've seen fit into this knows-theory-but-no-practical-stuff category. The fact there are some list of questions and their standards variants. All you need to do is just crawl over interview forums. I assume they even have books, as in pdf ready made for these purposes. Just ensure you read all those questions before hand. May be spend an hour a day to get familiar with them.

Go to the interview, when the interviewer asks the question act as though you've never heard the question. Act like you've been having it tough. But then suddenly like a hero emerging from a crisis present the solution to the interviewer.

You have not clue how many people do this. I've seen candidates, whose only job is this. They change a company every year. Spend half their day everyday studying and collecting salary and interview trends. Apply to a big brand, game the interview collect the x% hike stay for an year and then move on.

These kind of people are an online club. And they tend to hire only their kind.

No regards for the guy's actual work knowledge, his ability to solve real world problems. His knowledge of a programming language, tools, techniques. His productivity all that is irrelevant.

All that matters in these large web companies is a Ivy league brand and theoretical knowledge and some arcane facts memorized at college. Close to 98% of the so called algorithm experts fit into this category.

The genuine 2% are at places they like to work at and surely they pick the place they want to work at and not the other way around.

andralex• 15 years ago

As a Facebook employee who holds a PhD, I can say we are constantly preoccupied about improving the quality of our interview process. In particular, the topic of how to adapt the interview process advanced degree holders comes about quite often. For example, I recall we recently discussed that we should offer PhD candidates the option to give a talk about their research. (The option is indeed available.)

That being said, I think a dismissive or patronizing attitude towards a coding interview would be mistaken. Allow me to explain.

It is quite well known that the ability to code is poorly correlated with holding an advanced degree or other formal pieces of evidence. A candidate with strong credentials understands this and should, therefore, not be in the least offended by being asked coding questions. As a good professional, the strong candidate would handle the questions with panache and should be ready for more difficult questions, discussing advanced topics linked to the question at hand, or discussing her particular area of expertise. Either way, she must be able to code her way out of a paper bag, and I see nothing wrong with the interview process probing for that.

I've interviewed a few dozen engineers while at Facebook, many with advanced degrees (because recruiters pair as well as they can interviewers with interviewees for areas of expertise), some from top 10 universities. I ask 2-4 coding questions extracted from my daily job, starting from undergraduate level and on rare occasions ending with a difficult complexity question. Nobody has ever done perfectly well, and for a variety of reasons holding an advanced degree doesn't correlate strongly with doing well.

All in all, I think you'd be mistaken to believe two weeks of rote memorization would be enough to pass the Facebook interview, which is very difficult. Overall, an attitude of professionalism, modesty, and focus helps a lot in any interview.

zem• 15 years ago

as someone who successfully built a great dev team at a startup, i can say that this is a sign of sheer laziness on the interviewers' parts. which is sad, because in a small company, hiring is one of the most important things you can do. there is no real excuse for having an interview so generic that someone can game it in this manner unless you're so large that you can afford the odd charlatan.

paganel• 15 years ago

> there is no real excuse for having an interview so generic that someone can game it in this manner unless you're so large that you can afford the odd charlatan.

I'm also part of a small start-up (as an employee), and I can certainly vouch for that. I for one put a very high emphasis on the personal/professional projects the guy/dudette in front of me has been involved in, on how best s/he can explain them to me, on how passionate the person seems when explaining all this (and no, I'm not talking about the enthusiastic approach that can be easily faked, I just think that when you're a programmer you can feel what the other programmer in front of you is very interested in).

Also, this mania a lot of interviewers have with the interviewees "always ticking the right boxes and giving the correct answers" sort of scares me. It's probably because I work in a start-up, but I can assure everyone that at least half of the time we ourselves don't know the answers (assuming that we even know the correct questions), and there are lots of times when we seem to be knowing in what direction we're going but truly speaking we don't.

rmc2• 15 years ago

If you interview people, this article should terrify you.

If you're interviewing for programming jobs, the (low) calibre of existing applicants has probably already terrified you.

dunk010• 15 years ago

True dat.

lelele• 15 years ago

Lousy employers who employ lousy interviewers deserve lousy employees.

breadbox• 15 years ago

So ... what exactly is the turing-machine dog question? A google search left me unenlightened.

andrewflnr2• 15 years ago

I figured it was just humorous nonsense.

observer1• 15 years ago

"....this process, starting from ~50+ raw leads whittled down into ~40 initial phone interviews/coding challenges, which went to ~25 next stage interviews, eventually down to ~6 really good offers, all in about ~1.5 weeks."

Does this timeframe seem a little dubious to anyone else? I count 65 technical interviews plus interviews with hiring managers, etc., followed by receiving offers. And all of this happening in about 8 days. This story would be a lot more believable if it had a realistic timeframe.

sgrove10• 15 years ago

Nope, it's true. I woke up early in the morning (easy to do sleeping on the kitchen floor in mountain view without a heater), went through craigslist every morning, had an email template I'd individualize for each posting, and send it out. I'd work through the code challenges as they came in, set phone interviews into the late night, in-person interviews in the city during lunch, etc.

It happened, I took it pretty seriously because of the impending cliff of being homeless.

angelbob• 15 years ago

The 25 next-stage interviews are the only part that sounds even slightly unrealistic to me. The phone interviews are 30-60 minutes each, so 40 of them is no problem (20-40 hours of work).

The next-stage interviews are going to vary from 1-2 hours to a full day, so I assume he didn't do many full-day interviews in that 10-11 days, or probably 7-8 workdays.

But startups are often far more willing to do remote interviews, interviews at weird times, interviews on weekends... It would be a packed 1.5 weeks, though.

Timothee• 15 years ago

Besides the apparent mismatch between hours in a day and hours necessary for all that to happen in 1.5 weeks (well, it's technically possible), what surprised me was just even that 40 companies out of 50 were responsive enough to schedule phone interviews in that time. And next stage interviews after that. And make a decision to hire him. Especially for a self-proclaimed bad coder.

So, I agree with you, but this is not really the point of the post. The point is that interviewing is a game that you can master with practice.

jrockway7• 15 years ago

Indeed. It took me about 7 months from "Hey, do you want to work at Google" to getting an offer. I would expect other companies are similar.

rauljara• 15 years ago

I was flabbergasted that someone was asked implement a linked list in Ruby. My initial reaction was 'That is an incredibly stupid thing to do.' My slightly more in depth reaction was "Isn't that what mutable arrays are for?" And my final reaction was, "Maybe I'm missing something." At which point I tried to figure out the benefits of a linked list over a ruby array. I wasn't able to come up with a good reason to use a linked list in Ruby, but maybe someone could enlighten me.

kenjackson6• 15 years ago

Linked lists and arrays typically have different perf characteristics (although I don't know how Ruby arrays work).

For example if I give you an element of an array/linked list and tell you to insert a new element adjacent to it -- a linked list is a constant time insertion, whereas arrays are typically O(n).

tptacek16• 15 years ago

That's a sensible point, but interpreter and runtime overhead crushes any cost savings you might get from adjusting links instead of the whole array backing store.

(Ruby arrays, at least in MRI, are basically STL vectors).

1000 inserts to the middle of a 1,000,000 element Ruby array happens so quickly you can barely perceive the delay. The same insert pattern to a basic Ruby linked list sets my machine on fire.

tptacek16• 15 years ago

Linked lists are overused even in C, where they have a natural expression.

The reason to use a list instead of an array is that it's cheap to insert/delete anywhere into a list, but expensive to do so in the middle of an array.

There's probably no expression of a list in Ruby where insertion is cheaper than inserting into an array. Perhaps if your "list" is a secondary Fixnum index on a larger pool of objects? At any rate, that's no longer a "linked list".

I think it's a silly question. I hope the expected answer is, "why would I ever do that?"

tedunangst8• 15 years ago

My first guess would be that a FIFO queue is more efficient as a list than an array, but maybe there's something weird about ruby that makes that untrue. I'd love to hear why.

tptacek16• 15 years ago

Unless you have an extraordinarily clever implementation of "linked list in Ruby" (in which case this is still a terrible interview question), no. Arrays are more efficient for every reasonable access pattern. If the links of your list are references to Ruby objects (which every online "linked list in ruby" page uses), they're not so much "more efficient" as they are "tractable" compared to linked-list's "intractable".

DannoHung• 15 years ago

Non-mutable linked lists have advantageous memory/copying properties.

brown9-22• 15 years ago

I think you are looking at this from the wrong angle. Just because it made a good interview question (for the interviewer) doesn't mean it's something you'd actually want to use in everyday practice.

kamaal4• 15 years ago

Why would you ask questions to a candidate which have nothing to do with his job?

Will you ask a marathon runner to prove his worth by asking him how quickly he can sprint?

langsamer• 15 years ago

I think a much better indicator of how valuable a person will be in your company is not if he can solve puzzles on the fly but how munch time he puts into coding. If a person is coding in his free time and keeping up with technology and experimenting even though his current job doesn't require it, that person is more likely to have a breadth of knowledge that will pay dividends over time. Something like that cannot be gauged by him/her solving a "reverse-linked-list" problem.

Furthermore, a person involved in the community is more likely have in-roads with other strong developers, which again might pay dividends in the future in terms of hiring.

gcp2• 15 years ago

In my experience interviewing candidates, the two are very strongly correlated. A person coding in a lot, including in his free time is more likely to have implemented a wide variety of things, and will be much more hands-on to solve these kind of riddles/exercises, than a person with only book knowledge who coded the same kind of stuff for years on an end.

codenerdz• 15 years ago

I had a similar experience interviewing after being off the job market for a quiet a few years: You will be rusty initially, but then you will pick up steam.

Just like in your "real life", solving interview problems is not about reinventing the wheel with 100% custom solution, its about pattern recognition where the pattern is a problem you know a solution for. The more problems you solve(and possibly fail) the more data your internal pattern recognition algorithm(your brain) has to help you with the solution.

I dont see it as gaming the system, I see it as being able to prove that you are capable of problem solving.

sgrove10• 15 years ago

Yeah, it's a positive signal that someone was able to pick up from past failed interviews. It's a weak filter, but it's fine to have. From there, it's best to work with the person on short projects, and then hire them as a contractor to find out if they're a culture match and can solve the problems you're facing at an acceptable rate and in a maintainable way before hiring them on as an employee.

russell2• 15 years ago

I use a similar approach. I spent 20 years as a consultant, so I did a lot of interviews. I would send out lots of emails. I didnt do many tailored emails, but I did a carefully tailored cover letter designed to be amusing to the HR ladies. You have to get by them first.

I figured the first few interviews were throw-aways, just to get my interviewing skills back up to speed and to find what this years interview questions were. Yes, they go in fads. One year the programming task of choice was to reverse the words in a string. See it a couple of times and you get too be pretty proficient.

I keep a stock of questions to push back. I havent issued programming challenges, but I have asked, "What is your pain point? What technical issue is causing you problems?" Usually we get into a long discussion of the challenges theyare facing.

shard• 15 years ago

What did you do in your cover letter to amuse the HR ladies? Please share.

russell2• 15 years ago

Do you remember the Johnny Cash song "I've Been Everywhere... Chicago, Reno, Fargo"? Sometimes I think my resume reads like that.

After I added those lines, I got a 25-30% increase in responses.

brown9-22• 15 years ago

I started asking if I could assume TCO - usually knowing they would say no, but it would set them back a bit because that's an unusual question.

What does TCO here mean? Tail-call optimization?

anonymoushn3• 15 years ago

Yes. For a moment I wondered why he wanted to use TopCoder Open in the middle of an interview.

r00fus• 15 years ago

or Total Cost of Ownership... I guess I'm hopelessly "enterprisey".

sgrove10• 15 years ago

yup.

seank• 15 years ago

It's worth mentioning this as a counterpoint to 'startups don't reply': http://news.ycombinator.com/item?id=3351699

tptacek16• 15 years ago

Programmer interviews are famously terrible. Take a second to think about FizzBuzz, why anyone ever thought we'd need something like it, and how it actually continues to serve a purpose.

Everyone thinks they're better at interviewing than they are. They often have a survivorship bias. In clueful companies, it's not that dead weight doesn't get hired. It just gets washed out. Interviewers tend to remember the people they end up working alongside.

There are dev questions that are harder to study for. They tend to be open ended. A few of my favorites:

* Describe some code that tends to follow you from project to project (I mostly interview C devs, so I tend to ask this as, "what's in libyou.a?"). What's in your bag of utility functions?

* What's the worst library you've ever worked with and why did you hate it?

* Estimate the amount of time it will take you to complete system X. Drill down, both by forcing the candidate to specify components (do an informal design exercise) and then cost each of those components. Estimation is an extremely important skill regardless.

* What's a piece of functionality that tends to crop up on lots of projects that you'd never implement yourself if you had the option of using a library? (Usually, I'm trying to get C programmers to tell me that they would not in fact hand-hack doubly linked lists out of structs with nested struct pointers).

* Describe the last system you contributed significantly to; how did those contributions break down structurally (did you write libraries for X, Y, Z; did you write plugins for X, Y, Z; did you extend the engine in X, Y, Z ways). Now describe the trickiest bug you ran into on that project.

For C devs, I always used to ask, "You're testing a component you just wrote and finding that it crashes in malloc. Diagnose the problem." This is less relevant now, but there are probably surgical questions you can ask e.g. a Python dev that verify that they actually have the experience of working professionally in the language.

steve89186• 15 years ago

These are great questions. The one question I use with experienced hires is "Talk to me about the hardest bug you've ever worked on." I will keep drilling down to deeper and deeper levels to figure out how much they actually know, as opposed to memorize.

zmj2• 15 years ago

I'm not sure I understood you correctly about FizzBuzz, but if you're saying that it isn't necessary then I'd have to disagree. I wish the world was such that FizzBuzz wasn't an effective filter, but it is.

I really like the rest of your questions.

tptacek16• 15 years ago

I'm saying that the fact that FizzBuzz actually works should be an indication of how crappy programming interviews tend to be.

patio11• 15 years ago

Thomas is saying that the hiring process for programmers is so comprehensively fubared that asking the FizzBuzz question to someone who has worked as a senior developer for 8 years at a well-regarded firm provides valuable signal and that if you fail to ask it you might hire people who are completely incapable of programming even with your fancy-dancy recruiting process, which shows that the pipeline and standard interview process are both borked like crazy.

steve89186• 15 years ago

I call complete and utter bullshit.

Each "technical interview" I've been to is about 3-6 hrs. I've yet to see a Silicon Valley technical interview where it's not at least 3 hrs, meeting with 3 people. So right there, fitting 25 "next stage" interviews in 1.5 weeks is bullshit.

Where is he going to find the time for 40 initial phone interviews? That requires talking with the recruiter, and then the recruiter scheduling time with a developer. Anyone who has realistically interviewed in Silicon Valley knows that recruiters are very slow, and things tend to get muddled up very easily.

In 1.5 weeks, assuming they don't interview on weekends, that's 6 1-hr phone screens, plus 3 on-site interviews per day. The logistics simply don't work. It's a complete lie. If he had said 1 month, then it still would have been impossible with 1.5 phone screens and 1 on-sites per day, but it would have been slightly more believable than 1.5 weeks.

sgrove10• 15 years ago

Rough. But you're free not believe, I don't have a problem with that.

steve89186• 15 years ago

Sorry buddy, but please explain the complete discrepancy in what you reported.

How do you fit 3 on-sites and 6 interviews per day? Even if your onsites were 1 hr long and they offered you a job after a single interview, you still have to drive back and forth to each location. How did you have time for the 6 phone screens per day?

jrockway7• 15 years ago

There is a spectrum. The elite companies have all-day interviews (Google). Middle-tier tech companies have half-day interviews (Amazon). Big non-tech companies have a couple-hour interview (Bank of America). Small companies can vary, but I've gotten hired off a quick IRC chat. I have no idea what the average startup's culture is like, but they probably don't have 5 hours of developer time to waste on each candidate. So instead, they have a short interview instead of a long one.

eru9• 15 years ago

I can't post to the site, could somebody who can (or can contact the author in some other way), leave a note to the author, notifying him of a typo in "For people hiring, remember that anyway can pass yours tests with enough experience interviewing, even someone as bad at coding as myself." (It should be `anyone' not `anyway'.) Thanks!

sgrove10• 15 years ago

Got it, thank you eru!

troygoode• 15 years ago

People talking about how he "hacked the interview process" are missing the point: as a hiring manager I'd gladly hire this guy even knowing that so much of what he did was not entire genuine. He took a problem (I need a job), implemented a solution (applied and interviewed at a TON of places in that short of a timeframe), and successfully iterated on that solution until he had completely mastered it! In a very short time period he went from being unable to solve a variety of programming problems (or doing so poorly) to internalizing the how & why of the optimal solutions - this is what good software development is about!

I care a lot more about initiative and problem-solving skills than I do reversing linked-lists - this guy has the former two in spades.

overgryphon3• 15 years ago

Sure, if you're looking for someone who is hard working and willing to learn, then this type of candidate is perfect for you.

But if you're screening for candidates with development experience who you can rely on for technical knowledge, the idea that someone can game your interview process in less than two weeks without previous knowledge is horrifying. But I'm guessing that people looking to hire experienced candidates aren't basing hiring decisions on questions about reversing linked-lists.

kamaal4• 15 years ago

I think the point he was trying to make was. If you are productive enough and serious with a little skills you are can be hired.

The reason being you work hard your way towards success. This is far better than the person who knows merely facts.

Knowing doesn't always doing. But in most cases if you are doing something you are likely to know what it is.

krupan• 15 years ago

A lot of discussion here about memorizing the programming questions, but I think the most interesting part of this is when he turned the tables at the Do You Have Any Questions For Me stage and really interviewed them.

I haven't asked anyone a programming questions like he did, but I did ask some tough questions about the team, work environment, management style, company goals (looking for a quick exit? in for the long haul? etc.) at my last job interview (after some good friends gave me the advice) and it both gave me a lot of insight into the company and team that I was considering working for, and I think it impressed them that I was thinking about those things.

In short, know what you want from a team, manager, and company, and take time to interview the people interviewing you to make sure they have what you are looking for.

systemizer• 15 years ago

I had a similar experience. This past Fall, I was interviewing with multiple companies, and I received questions that were very similar.... some being the same exact thing. I was nice, however, and told them out front that I had seen the problem before and asked for another.

I don't regret doing that; some may have passed the interview by keeping quiet even though they had seen the problem, but those people won't go far in the world because they cheat. They will eventually get called out for it and lose their reputation.

nadam2• 15 years ago

I don't understand why is there such a strong emphasis on these kind of interview questions. I think showing a piece of nontrivial code which the candidate have created in the past tells a lot more about the candidate.

For example this tells more about me than the interview questions I usually got: http://codeclamp.com/ccui.js , because it tells about how I think long-term, how I solve relatively hard and relatively complex problems, not how I survive a quick interview (solving relatively easy algorithmical problems but under very high time pressure.). (By the way, if you have an interesting well paying job for me, I might be interested...)

pnathan• 15 years ago

This article really indicates to me the fatuousness of this sorts of problem/solution interview approach.

I agree with timr in that a more more intelligent approach is to throw a problem at the candidate, then drill down into their thinking about the problem.

It's a bit silly if you're hiring based on regurgitation of algorithms in my opinion.

tikhonj2• 15 years ago

I certainly agree. However, I haven't actually seen any pure problem/solution interviews--and this is just for internships where you wouldn't expect the companies to be as thorough. I had questions ranging from "find and fix the problems in this code" to "how would you design a library to do..." to "propose a solution to this problem and prove it works". This might be because I mostly talked to smaller companies.

tldrtldr• 15 years ago

Isn't there a book dedicated to coding interviews?

http://www.amazon.com/Cracking-Coding-Interview-Programming-...

How to get hired (or, 'The silly story of interviewing in the valley')

Related Stories