Will the real programmers please stand up?

http://www.rethinkdb.com/blog/2010/06/will-the-real-programmers-please-stand-up/

by coffeemug • 16 years ago

112 171 16 years ago

pmjordan4• 16 years ago

Over the last 2-3 years, I think RethinkDB has literally been the only company that has made me think "wow, I think I'd really enjoy working there". (I'm self-employed) I find it baffling they can't find enough people. Are the cool kids really only doing web apps these days? What about all the army of Objective-C programmers that Apple has been assembling?

coffeemug4• 16 years ago

Apply!

pmjordan4• 16 years ago

Hah, I might have to if our product tanks upon launch. ;-) (also SSD related, but workstation/laptop-based, not DB/Server)

jpablo• 16 years ago

Do you guys hire people from outside of the USA ?

GrandMasterBirt4• 16 years ago

Sorry, live in wrong state, and I don't code in C. Though I'd love to pick it up, can't be harder than java or javascript, just some things you take for granted like references vs pointers have to be deliberately thought of until they become second nature. O well lets wait till the kids move out and I have free time to pick this stuff up :)

lani• 16 years ago

Eval!

yummyfajitas5• 16 years ago

A good chunk of the cool kids probably want to make more than $125k/year.

http://rethinkdb.com/jobs/

pmjordan4• 16 years ago

It'd be a step up from what I'm making with contracting (and comparatively expensive), but this is Europe, and I'm clueless with regard to cost of living in the bay area.

Besides, where else do you get to work on problems like that? But judging by the other responses on this thread, my idea of fun doesn't seem to be widely shared.

Alex634• 16 years ago

I haven't worked as a "programmer" or "software engineer" for a few years, but looking at GlassDoor.com for evidence, $125K seems like a pretty good salary. Is that an unrealistic salary for a programmer today? Are you referring to the greater rewards of being an entrepeneur/founder?

retroflexzy• 16 years ago

That makes me sad because I've never made more than 1/3 of that as a developer in Vancouver.

GrandMasterBirt4• 16 years ago

Its about benefits. 125k is quite low for high risk jobs for the awesomeness you require. My job offers the same salary with significantly more stability and not so insane requirements.

djb_hackernews• 16 years ago

It's definitely a gamble. Especially since they only have 1.25m capital. I say only because we can glean a lot from the published salary structure. Throw a senior, 2 juniors, an intern, CEO, CTO and some admin people and you probably don't make it to 2 years.

mpc• 16 years ago

You're not hurting with that salary and look to the right... 2% of a company for a hacker at the seed stage is amazing.

plinkplonk• 16 years ago

"Are the cool kids really only doing web apps these days? What about all the army of Objective-C programmers that Apple has been assembling?"

Something of a tangent, but a lot of cool kids don't work in the USA and have no intention of moving. I know a dozen or so people here in Bangalore who'd breeze through these questions. The flip side is of course that I can think of only a dozen or so people and this in a city (Bangalore) with tens of hundreds of thousands of "programmers" in it.

PieSquared• 16 years ago

Huh, I think it's time to learn something.

I can write C code to reverse a linked list, or discuss the asymptotic efficiency of a bunch of data structures in my sleep... but I clearly don't have enough experience with threading if I had to google what a "condition variable" was, or had to make sure my intuitive notions about cooperative and preemptive multitasking were right.

Always nice to be reminded that you don't actually know anything.

pohl• 16 years ago

I have a tangential question: is there a name for the kind of diagram one sees on this wikipedia page involving rooms & doors? I've never seen it before.

http://en.wikipedia.org/wiki/Condition_variable

lmkg• 16 years ago

It looks like a variant of a basic floorplan, the sort you would use when designing the layout of your office. I've never seen it used with respect to algorithms.

ygd• 16 years ago

It seems they're all some type of "monitor".

roel_v2• 16 years ago

I didn't know about condition variables either, but it seems like the questions are drawn from their own problem domain, concurrency (condition variables, multitasking modes, threads vs coroutines). It's fair enough to ask at an interview if that's what is used in your code, but I don't agree that 'what is a condition variable' is of the same level of 'fundamentalness' as 'reverse a linked list'. A graphics programmer will probably find people who don't know their transformation matrices by heart lacking in fundamental knowledge, too, but in the end it's just one concept that is taught once over a 4 year degree, and after that is only relevant for people in that specific subfield.

The article says 'Because they’re part of a core body of knowledge taught in any undergraduate curriculum worth its salt, and because in some form or another, they came up in our daily work' - I can agree with asking it because of the second part of that, but I don't agree that all the questions presented in the blog post are in the category of the first part.

varjag2• 16 years ago

The scary thing is that I once knew what condition variable is, but forgot.

Easy to say it here, but would be really poor answer at an interview :)

kscaldef2• 16 years ago

I disagree, at least based on my personal experience. I suppose I've interviewed at a dozen or so companies over the years and gotten job offers from all but 1. There's almost always something I get asked during an interview where I just honestly say, "You know, I'm not really all that familiar with that" or "I'm pretty rusty on that area at the moment". It's a better answer than trying to BS something.

Tichy2• 16 years ago

Same here about condition variables and preemptive vs cooperative multitasking. I know the concepts, but not the names. I think mabye C programmers have to deal with that stuff more often? In most other environments, the need doesn't arise that often. Many are designed especially to avoid that need (ie node.js, Erlang),

tomjen32• 16 years ago

I too had too look up wtf a condition variable is - but it was because everybody calls them "monitors" which I did have a couple of course that hit on.

noelwelsh2• 16 years ago

In my mind condition variables are stored under "crap I don't need to know because we have better abstractions". I can understand why RethinkDB will be messing around with semaphores (or monitors aka condition variables) but for most programs there are far better ways to organise concurrency. Java recognises this -- experience showed the synchronized (monitor) approach to concurrency to be too unwieldy and it was superseded fairly quickly by java.util.concurrent and friends. Way back in 2000 we were using Doug Lea's concurrency library, which formed the basis for java.util.concurrent. I certainly could use semaphores but it would certainly take me some time to adjust my thinking to that primitive model.

GrandMasterBirt4• 16 years ago

Had no idea what Condition Variables were. Honestly. However after reading up on them I realized I've been using them for a looooong time, they just seemed so natural a solution to a problem. Most simple patterns can be derived by yourself without any help because they really fit the solution so well that it feels unnatural to not think of them. Some of the more interesting ones are removing complexities :)

JoeAltmaier2• 16 years ago

That's an applications-programmer answer. But who did you think wrote java.util.concurrent? The article is by someone who needs systems programmers, who do algorithmic work, not applications programmers, who put together large state-diagram code.

noelwelsh2• 16 years ago

Doug Lea is my guess. We were writing algorithms, but they were machine learning algorithms so implementing our own concurrency abstractions wasn't a win.

GFischer• 16 years ago

Hah, I suspected as much from the article's title.

Systems Programmers = "real" programmers (for the article's writers)

Application Programmers (or even worse, Information Systems guys like myself) are not "real" programmers to them - and to be honest, I don't believe myself a "real" programmer sometimes as well.

coffeemug4• 16 years ago

This is exactly the kind of answer we're looking for.

lbrandy• 16 years ago

Yes, yes indeed. Looking at it like that, it is great a response, but there's still a problem...

Given the rather adversarial nature of the technical interview setting, there's pretty much zero probability you'd actually get that answer from him.

This presents an interesting dilemma.

ydant2• 16 years ago

C and C++ seem to be the languages of choice lately for the interview complaints. I wonder if that's indicative of the language itself or of the number of people who actually program in those languages.

Of the questions / problems expressed in the article, I think I have a decent grasp of most of them, and I don't program in C or language with similar ideas. I did learn with C (and then expand that knowledge to C++), but that's been a long time ago. To not be able to traverse a linked-list, though, that baffles me. Simply shouldn't be appling for a job programming in C or C++ and not understand linked lists.

habitue• 16 years ago

If you have to traverse a linked list at any time to reverse it, you're doing it wrong.

roel_v2• 16 years ago

How would you reverse a singly linked list without traversing it? (and without knowing implementation details too I guess - doing pointer arithmetic and storing the size somewhere else doesn't count).

jallmann• 16 years ago

It's not about the language, it's about how you approach the problem. Maybe their rationale is that using C discourages the use of built-in constructs or libs that circumvent doing the real work.

For this type of problem, it also hints at your understanding of indirection and pointers as opposed to stack allocated values, etc. Pretty much a pre-requisite if you're doing anything remotely performance-critical, like rethinkdb is. When you're dealing with a higher level language, that's all abstracted away by references.

spatulon• 16 years ago

It's been long enough since I regularly wrote C that I spent more time figuring out the old "struct vs typedef struct" issue than I did writing the reversing algorithm.

starkfist8• 16 years ago

Nobody in America has to do this kind of work anymore. Some do it by choice but usually work on something they are really into, like Pro-Tools or iPhone software, or something where you make a lot more money, like Wall Street (or iPhone software). I have not had to do anything remotely resembling reversing a linked list in C since 1994. Since then, the further I've gotten away from C and algorithms, the more I've gotten paid...

Your best bet is to recruit in Russia/Eastern Europe/India/China.

Edit: before I get downvoted into oblivion, I didn't mean this disparagingly, I am just being matter of fact. It will be easier to hire people from those countries. Most Americans who do this work already have a job. Most Americans looking for a job, don't want to do this, or don't know how. I have worked with guys from Croatia who helped write some database driver code for me and were excellent.

JoeAltmaier2• 16 years ago

I do this kind of work daily. I work in drivers, protocols and other operating system places.

No need to denigrate folks who do (or don't do) this kind of programming. Its just the difference between an applications programmer (who codes state-transitions for large state machines) and system programmers (who create those libraries everybody else uses).

IgorPartola2• 16 years ago

Nobody in America (meaning no programmers) has a reason not to know this stuff. You don't have to do it on daily basis to understand it. I have never in my life had to reverse a linked list, but I know how to do it. I don't see how these are unreasonable questions to ask, especially for a company that develops a database engine.

starkfist8• 16 years ago

They aren't unreasonable demands, I'm just suggesting it's going to be easier to find people outside of the USA.

amalcon4• 16 years ago

Here's the thing: you're not being asked to reverse a linked list because they expect you to implement one. 99% of the time, you'll just use a library (the other 1%, you're writing the library). You're being asked to reverse a linked list because it's bloody easy if you understand the fundamentals (what a linked-list is, how a loop works).

starkfist8• 16 years ago

I realize that but wishing that people applying for jobs actually knew what they are doing unfortunately is not going to help much. You can make a whole career without understanding linked lists. If you need people to work on database internals you either have to hire from the areas I mentioned or figure out how to poach people already doing this kind of work. The pool of applicants who just send in their resume generally does not contain the programmers you actually want to hire.

_cbsz2• 16 years ago

You've probably also been asked to do it at some point during your formal CS education, if you had one. Grading this problem on the final for the introductory CS course was a load of fun, let me tell you... (and yet, I had to do the exercise myself just now, because I haven't touched that stuff in a couple years and I was worried I'd forgotten!)

Tichy2• 16 years ago

I recently started a new job and in the second week implemented one binary search algorithm and a sort of Levenshtein distance thing. I almost felt like celebrating - I don't expect another opportunity to do that kind of thing any time soon (at that job anyway). And these are really simple algorithms.

arghnoname2• 16 years ago

That's bizarre. I happen to enjoy it, and while I would grant you this isn't the norm, I don't think it is exceptional either.

Some people, maybe most people, can't be bothered to deal with the hassles of lower level work.

Myself, I don't have to re-invent every wheel in the world, but when I had a web dev job I found I had gone a year without actually writing anything but glue. For people like me, the feeling is "what's the point?" Don't ask me why the finicky lower level stuff is any different, but for me, for some reason, it really is.

adn37• 16 years ago

That was funny to go pen & paper to do the reversal of the linked list, C style, like in the old days.

(aka, Java coding is tasteless)

brown9-22• 16 years ago

What does this mean? The article also states they ask other questions in which the candidate can respond in any language of their choice.

The language isn't important here, it's knowing the algorithm and being able to put your ideas into code.

You could ask the same exact question in Java and you should be able to get the same answer from people.

substack3• 16 years ago

The language is absolutely important, since in haskell I could just:

    > reverse [1,2,3,4]

or if I couldn't use the Prelude (yikes!)

    reverse [x] = [x]

    reverse (x:xs) = reverse xs ++ [x]

    > reverse [1,2,3,4]

The problem is that linked list traversal is a solved problem. Why not throw a novel and interesting problem at candidates? That way you could gain some insight into how they go about solving problems instead of how good the candidate is at memorizing other people's solutions that they could just as easily look up in a search engine in a few minutes.

jimbokun• 16 years ago

These are the kind of questions where I might draw a blank if asked out of the blue, but I make sure to bone up on before going into any kind of technical interview.

alttab2• 16 years ago

This actually made me feel better about what I do know.

But then again, my course work included a course in operating systems where the main project was to write the multi-threading architecture, system call infrastructure, virtual memory, and file system for a bootable x86 OS. I went to Virginia Tech but the course's origins is Stanford.

Best class I ever took.

scott_s2• 16 years ago

You may be curious to know that's not a required course anymore. It's now a senior level elective. The required junior level course is now Computer Systems (http://courses.cs.vt.edu/~cs3214/), taught by the same professor. I was a TA for it last spring and I'll probably TA it again in the fall.

Although now that I look over the listed courses, I don't see a senior level OS class.

alttab2• 16 years ago

Interesting. I remember the course being very difficult. I wonder if it was "too hard" so they made it optional. A mistake if you ask me, that class made me a better programmer than any other undergraduate course.

Although, the only ones that would take it now would want to.

dkarl4• 16 years ago

Similar experience here. A shocking number of people simply can't code. I started giving people a "simple" whiteboard programming question: write code, in any real language (i.e., not pseudocode,) to print out the permutations of a string, and give me running commentary on what they were doing. The idea was, by watching them complete a simple coding task that would require a couple of iterations of design and a little bit of "debugging" at the end, I could get a feel for how they attack problems.

In about a dozen interviews, nobody did it. The problem was way too hard. By the end of the cycle of interviews I had started giving lots of guidance to the candidates: asking them about permutations that their solutions would miss, asking them how many permutations their solution would print, asking how many permutations a string of length 2, 3, or 4 might have, and when they got stuck, telling them what they had right and when they went wrong. Nothing really helped, though.

Perhaps it's not really so shocking. Maybe's it's just too hard in a one-hour interview. I thought it would be easy, but then again, I've never had to write code in an interview. What is shocking is how poorly people floundered. I'm pretty sure most of them couldn't have even reversed a singly linked list. In the end, I gave the best marks to the candidates who:

1. Asked whether it was okay for duplicates to appear in the output.

2. Wrote up a roughly correct (syntax errors and off-by-one errors aside) implementation of their first idea for solving the problem and explained how it worked.

3. Understood when I pointed out permutations that their solutions would miss.

4. Were a good sport about being asked to solve the problem. (Actually, nobody who failed this criteria did remotely well at the others, so it's somewhat redundant. I mention it because I was surprised that people would get crabby about being asked to write code in an interview.)

Three or four of the candidates, out of about a dozen, managed to meet all four criteria. These were people who passed a phone screen! Even worse, some of the candidates who failed one or more of the four criteria above apparently shined in their other interviews, claiming to have been in charge of designing and implementing significant systems.

lincolnq• 16 years ago

Solving this problem (permutations) in Haskell helped me understand the list monad, which in turn helped me think about nested-loop algorithms in general and how they should be structured in code. So thanks. It was a fun exercise and I learned something.

I am pretty sure I nailed it. But I have been asked questions like this at interviews and screwed them up (e.g., at Facebook, which asked me something very similar and I failed to get it right). The notable thing was that at home, sitting at my desk, I wasn't afraid of sitting silently and figuring it out. It took me about 3 minutes of silence for my brain to organize itself enough to solve the problem. I'm not willing to spend 3 minutes in silence at an interview, and yet if I just jumped in I would go down a lot of wrong alleys, become confused, and screw it up.

Is there room at an interview for someone to silently figure something out, without someone waiting on you, asking to "show me your thought process"?

mechanical_fish2• 16 years ago

Is there room at an interview for someone to silently figure something out...?

Frankly? No.

Not for three minutes, anyway.

That's why either (a) the permutations question is too hard, or (b) the interviewer cannot necessarily expect anyone to get the right answer, but should instead judge incidental aspects of the process. (This is what dkarl seems to be doing, and that's fine.)

If you are asked a hard question, I think it's better to steer on the side of "going down a lot of wrong alleys, out loud". Just be honest about it. If you find yourself lost, go ahead and say "oops, thinking out loud during interviews is not my strong suit and now I'm pretty sure I'm lost". Find your own bugs and laugh at them. Backtrack. Start over. Restate the problem. Sketch a unit test that describes the answer that you can't quite derive yet. Define a simpler problem that you can solve out loud, and solve that first. And, of course, always do the stupidest thing that could possibly work.

It occurs to me that the average student spends all kinds of time in school studying how to solve problems quietly and completely at one's desk, and very little time practicing the art of sketching half-baked solutions out loud while waving one's hands. But you can practice that.

dkarl4• 16 years ago

The people who did best tended to have long silences at first and get more chatty as they went on. The ones who dove right in coding and chatting did poorly. I held that against them as a sign that they would always code first and think about correctness later, which may have been unfair. Maybe I'll make "talk as you go" optional next time, in case it pressures people down the wrong path.

calibraxis• 16 years ago

Not many interviewees will have the comfort level to ask the gatekeeper for some quiet time. I think interviewers should offer it proactively. Conventional wisdom in interviews is to always offer a running commentary, because people like progress bars. Particularly since an interviewer will perceive time far more slowly than the person they're grilling.

A common thought process might go something like this: inchoate thought; more structured visualizations and vague search over solution space; enough insights have accumulated; convince yourself that the solution is correct; nail down details. It's an additional skill to provide running commentary to strangers during this.

The permutations problem -- which I've encountered and did fine, FWIW -- may be a poor question for most projects. I imagine only few require much thinking along those lines.

jules5• 16 years ago

Permutations are much harder than reversing a linked list. Actually I'm interested in what solution you had in mind. I can only think of these three:

1) Do it recursively like permutations(xs) = [[x] + p for p in permutations(xs - [x]) for x in xs].

2) Systematically generate all combinations of letters, only print the ones that are permutations of the original.

3) Systematically generate the permutations in lexicographical order.

All three would require pretty significant effort to get correct on a whiteboard in a language like C (without high level list comprehensions & operations).

waterhouse• 16 years ago

Regarding your last sentence, he did say:

> write code, in any real language (i.e., not pseudocode,) to print out the permutations of a string

dkarl4• 16 years ago

1) is what I was expecting. The solutions I saw were either attempts at 1) or attempts to do 3) using nested loops. There was also one guy who spent an hour trying to solve it with increasingly intricate methods of swapping the letters around in the string, I think because the first algorithmy thing that came to his mind was balancing binary trees. (He kept sketching the string as a tree rooted at the first letter.)

Choosing C would be a pretty big fail, though honestly, people didn't struggle much with language issues. I told them it was fine if they didn't remember exact APIs, and nobody got to the point where they were worried about whether the code would parse. One guy did it in C++ using the string class, and I don't think he was at a big disadvantage. It's (evidently) harder to figure out the details of the algorithm than to write out a solution in C++.

Daniel_Newby• 16 years ago

"... give me running commentary on what they were doing ..."

There's your problem! You are asking people to simultaneously (1) design a divide-and-conquer algorithm, (2) write syntactically-correct code on a whiteboard, an unfamiliar mode of writing for most people, and (3) give a comprehensible oral presentation, which creates a lot of social and emotional pressure for many people. Distraction is the death of reasoned thought.

Break the interview into single-purpose phases. (1) Help them write down a clear description with some examples. (2) Have them cook up pseudocode or diagrams of how to solve it, with review/debugging from you. (3) Have them write code. (4) Have them explain why it works.

As for being crabby, probably a few of them saw, but could not articulate, that the micromanagement set them up to fail. They should handle it better, but it's understandable. If you want to evaluate their willingness to graciously write code, start the interview with a dead simple FizzBuzz style challenge. E.g., write a function that takes an array of integers as input and copies them to an output array in reverse order. Specifically tell them they can ask questions or ask for help, but need not explain anything, that the code will speak for itself.

ajuc• 16 years ago

I would probably fail at your interview. It is considerably harder than fizz-buzz or reversing linked list (where no thinking is needed, so stress won't matter), and I would be stressed that it takes me too much time to think this out, so I better start some coding and talking. Things can only get worse from that place :)

amarc• 16 years ago

yes, but only if you pay enough...

jwr• 16 years ago

One thing to consider is that there are many people (myself included) who are really bad at solving problems under stress, especially next to a whiteboard.

Note that this does not necessarily imply these people are useless in real-life tasks. It just means they are bad at passing exams.

This is why I would never get through Google's interviewing process. Some companies (like Google) just accept the fact they will throw away some possibly good candidates because of the nature of the selection process — and that's fine, as long as you realize that.

Note that I'm talking about solving problems involving abstract thinking next to a whiteboard here. The part about people not knowing the difference between cooperative and preemptive multitasking is something I fully agree with — people who do not know that should not seek a job at a company that deals with anything low-level.

coffeemug4• 16 years ago

We fully recognize that we have a high false negative rate. We work every day to get it lower, but unfortunately there is only so much we can do without compromising the integrity of the process.

carterschonwald• 16 years ago

Hey Slava, I'm sorry to hear that getting good hires has been do tough. Both myself and the friend who I later referred to you guys definitely agreed that your interview process was very well designed (and pretty darn interesting too).

Best of luck!

Andys2• 16 years ago

I have a question for you. I'm comfortable answering those coding questions, but not on a whiteboard.

Would you be OK if instead of writing on the white board, I whipped out my laptop and coded the answer there in front of you? If the answer is no, I would happily walk out of the interview knowing I would not be a good fit for the company.

brown9-22• 16 years ago

Note that this does not necessarily imply these people are useless in real-life tasks. It just means they are bad at passing exams.

This is why I would never get through Google's interviewing process.

I understand that some people are bad at doing these types of tasks on the spot.

But what about training yourself to get better at working like this, by working through exercises, having friends help you out, etc?

Maybe it's unfair of me to comment because I'm not in the same boat, but surely there is a way for most people who are currently inherently bad at this class of problem to work on getting better at it, rather than just always assuming that a company like Google or Rethink would never hire them.

starkfist8• 16 years ago

There was a time when computer programmers were often also very good musicians. People liked to suggest this was so because of a mystical math/music connection, but I always thought it was because musicians were used to practicing to improve their skills.

amalcon4• 16 years ago

There is only one way to do this. Go do some technical interviews. You're not going to impress everyone, but you will grow more comfortable in that interview room.

edanm• 16 years ago

If I were an interviewer, here's what I'd be looking for:

1. You hear "reverse a linked list", you maybe ask some clarifying questions (expected efficiency, in-place or a new list, etc.).

2. You start drawing boxes with arrows on the whiteboard.

I've never been able to come up with any algorithm for a linked list without drawing a bunch of arrows and staring at them. And while lots of people might be too stressed out to come up with the algorithm, any programmer who has even a little experience with lists should know to start with that.

In other words, if you have any experience with this kind of stuff, it shows up pretty quickly by what you do/what you say.

cabalamat4• 16 years ago

> One thing to consider is that there are many people who are really bad at solving problems under stress

I'm surprised that people find writing simple programs and talking about them stressful. I find it quite enjoyable.

jonpaul• 16 years ago

It's not that that's stressful, it's the potential possibility of failing the interview and perhaps not being offered the job that causes the stress.

jules5• 16 years ago

On the other hand, any half decent programmer should be able to reverse a linked list under nearly any circumstances.

pellicle• 16 years ago

There are many decent programmers who've been using high-level languages like Perl/Python/Ruby/etc for so long that they don't even remember the last time they had to think about a linked list.

grasshoper• 16 years ago

If someone doesn't know some definition, they can look it up in a book.

If someone isn't able to solve problems, that isn't something you can fix easily.

I'm tired of all these people who do poorly on tests turn around to blame the test. Everyone is nervous during tests and interviews. Those who succeed do so because they can solve those problems like the back of their hand.

calcnerd2562• 16 years ago

I somehow don't get particularly nervous during important tests or interviews. I get a little nervous if it's really really important, but I somehow get over that pretty quickly. It's nothing that I earned by hard work or practice, either. Now if only I could apply that gift to talking to women...

juicebox2• 16 years ago

"Everyone is nervous during tests and interviews."

Yes, but what you're missing is the degree certain people are nervous. Some people lock up completely due to performance anxiety, which has nothing to do with how difficult the problem is. Those people are also more likely to be found in the computer engineering profession due to the nature of the job.

So if you want a charismatic person with strong social skills, then you're just going to have to accept the bullshit artist who is more PR than talent over the painfully anxious and shy person who knows how to program.

That, or deal with hunting down the very few people who are both social and vastly knowledgeable. Good luck with that.

9oliYQjP• 16 years ago

Exactly. There are great essayists. Then, there are great debaters. Both types are great at the art of persuasion, which is the ultimate goal. I used to believe in interviews like this. But after stumbling through poor results, I came to the realization above.

hernan7• 16 years ago

Making the candidates reverse a linked list over the phone? Either the job market is worse than I thought or somebody is power-tripping something awful.

coffeemug4• 16 years ago

http://typewith.me/

tkahn63• 16 years ago

When you ask someone to reverse a linked list, do you mean 'in place'?

jeb• 16 years ago

I can reverse a linked list, but here's what happened on reading this article.

After seeing the problem, I fired up textmate and started writing C pseudo-code to do it. I started off with

    first_item = l.first();
    next_item = first_item.next();

Then I started writing the loop, and somewhere in there, I got confused. And then a wave of panic came over me - shit, maybe you can't do this. I shook off the thought, deleted my code, and restarted. At some point, I realised that I was always pointing at the first element. My stomach sank and I had this dreadful feeling - okay, you kind of suck as a programmer. Then my mind went blank, and I wanted to switch over and see if the answer was there.

And that's with NO PRESSURE AT ALL. With someone watching me, I'd probably not even have been able to get the loop right.

I have never written such a loop, but it's basic stuff, I can do this anytime. But just a little bit of pressure made me royally fuck up.

Programming is thinking - you need to juggle shit in your head - if at the same time you are paying attention to what a HIRER is doing or saying, it will totally smash this ability to do shit in your head.

Your programming exercise probably selects for calm people, not for good programmers.

starkfist8• 16 years ago

Or it selects for people who do this kind of thing every day.

amalcon4• 16 years ago

People who run this sort of question usually allow for a bit of partial credit. It's not pass/fail. If you write something that's wrong, but you can at least point out why it's wrong, then that's not a good answer, but it's not always an immediate disqualification.

grogers• 16 years ago

Staying calm while thinking through a solution is a pretty useful trait to be hiring for - not just for programmers either. What happens when the shit hits the fan and everything is core dumping left and right - do you really want someone whose head is going to explode? Or do you want someone who can stay calm and diagnose the problem and/or get to a workaround quickly.

prodigal_erik• 16 years ago

This. Every so often something blows up at 2 am and we have to troubleshoot an unknown problem while our employer loses $3k/hour and everyone in charge wants to know WTF. Thankfully it's rare, but I don't want us to have to support your code if you aren't up to offering useful help at that moment.

jast2• 16 years ago

Funny, I also fired textmate and started solving the problem in C :-)

It took me around 20 minutes to write all the extra code and 9 to write the reverse function (including the testing and all).

The problem was not knowing how to solve it but more the pressure of looking at the clock and seeing the time flying away. Because of that, some minor mistakes were done and naturally you start to get nervous and waste more time. In a phone interview I would probably even do more minor mistakes and fail it.

_cbsz2• 16 years ago

What extra code? I've got a struct definition, a print_list function, cons, and a really dumb testing driver that builds the list [1 2 3], reversing and printing it a few times throughout its construction. The line count of all that [EDIT: I meant "everything except the testing driver"] is about equal to the line count of my reverse.

jules5• 16 years ago

The trick is to not fire up your text editor until the very last moment. Big design up front works for (difficult) algorithmic problems, and it's the only way that works.

I'm sure you could do it if you took a piece of paper and drew a linked list with boxes.

calcnerd2562• 16 years ago

step one: solve it in LISP

step two: tell a C compiler how to do it

(although the solution I would write first in LISP isn't actually what I would do in C; in C, I'd start with stack/cons-like linked list and do a slinky-queue shift)

torpor• 16 years ago

Doing something theoretically difficult in code during an interview is one thing - but having a compiler in front of you to help you get through that difficulty is another thing entirely.

The point is to determine whether you are a programmer because you think like one, or if you are a programmer because your compiler thinks like one, for you. Surely, if you rely on your edit-compile-test-hack-edit.. loop to get you through the problem, thats okay: most programmers work like this.

But the really good programmers, ones that are valuable to all sorts of conditions of development, can work out a problem pretty rapidly without needing the workflow-loop to get to the conclusion. These sorts of tests, outside the 'normal environment' in which programmers usually operate, don't really point out the programming-efficiency of the person, but do point out how much they depend on tools to be the programmer they are ..

Myself, I also thought of 3 different ways to reverse the list before I sat down to write any code. And even now, with my compiler in front of me, I'm not entirely sure there isn't some brilliant one-liner to solve the problem. I am, however, sure that once I've compiled a solution, and run it, and seen that it does indeed work, I can walk away from the problem satisfied that I at least got something working, even if it wasn't the best solution. The degree of effectiveness of my approach and my process is what is being measured, here, I would imagine, by the interviewer ..

angelbob• 16 years ago

I used the linked-list question for years as well. I think our phone screen was harder -- it probably only filtered out about 60%-75% of our applicants rather than 95%.

It's a good one. I have an awesome follow-up to it that it sounds like your interviewees would never get to anyway :-)

adbge• 16 years ago

> the interview process is something we won’t compromise on.

Sounds like you're caught up in a big circlejerk, to be honest. Your interview process is terrible and that's why you aren't finding any employees. You won't compromise on your process? What? You should be less worried about your process and more worried about finding quality employees (hint: your process isn't working).

Asking people to write a program over the phone isn't going to work. Have you ever written a program while on the phone? I haven't and I'm not going to. It's so far out of my comfort level that I'd probably end up looking like a moron.

Your test sounds like it's too much about what a person knows and not enough about how a person performs. You'd be much better off looking at a persons resume and, if you're satisfied with that, just talking to the person instead of worrying about whether or not they can reverse a linked list in C off the top of their head. If you want them to write you some sample code, let them do it at their leisure instead of with you breathing down their neck over the phone.

Frankly, reading about your interview approach just kind of pisses me off. What's more likely: all the candidates suck or your interview process sucks?

starkfist8• 16 years ago

In the OP's defense, this is how Google interviews, and they've done ok.

chc3• 16 years ago

Google can afford to be ridiculous. Practically everybody would like to work for Google, so they can afford to have a high false negative rate — even if they reject 99% of good candidates, they can probably still fill their staffing needs. RandomCompanySoft, on the other hand, might only get five good candidates for a given position — so if they only recognize one out of every 10 good programmers, they can't find anybody with any skill.

joe_bleau• 16 years ago

I think it was in Dan Ariely's Predictably Irrational book that the story of some branch of the Israeli military came up. They only accept the absolute best applicants, but then they also accept a few that fail the admissions process. The reason is to provide a check on the process; if those admitted 'failures' work out in real life, there's something wrong with the selection process that needs to be corrected.

Once big difference is that they require a certain amount of manpower, so they can't afford to throw away good material due to an inaccurate screening process. I guess in contrast Google can (probably?) afford to be wasteful.

sgift• 16 years ago

If you can believe Peter Norvig maybe they do ok despite their interviews, not because of them:

"One of the interesting things we've found, when trying to predict how well somebody we've hired is going to perform when we evaluate them a year or two later, is one of the best indicators of success within the company was getting the worst possible score on one of your interviews. We rank people from one to four, and if you got a one on one of your interviews, that was a really good indicator of success."

http://gawker.com/5392947/googles-broken-hiring-process (The first quote)

tkahn63• 16 years ago

They're saying that they use http://typewith.me

And honestly those questions that they ask candidates seem reasonable for the type of work they're doing.

shasta• 16 years ago

> What's more likely: all the candidates suck or your interview process sucks?

The numbers he is giving are very reasonable. Remember, that's 5% of the applicant pool that pass this test, not 5% of all programmers. It may be a bad economy, but the people looking for jobs are still skewed to the incompetent end of the spectrum. That might correspond to looking for the top 25% of programmers.

I wouldn't consider anyone who couldn't come up with working code to reverse a list in an hour anything like a peer, and I'd only consider hiring them for easy work if they were cheap or had some other specific expertise.

jrockway• 16 years ago

I disagree with you and agree with the author. :)

It's hard to find people that can answer programming questions while applying for programming jobs. People generally do OK on the "soft" questions appropriate for the phone. "Explain automatic testing." "What kind of editor / IDE do you use?" etc.

People generally don't do well on simple whiteboard questions, and try to BS their way through. Our questions are very simply, along the lines of: in your favorite language, write a function to: "calculate the nth Fibonacci number" (we explain the sequence if you've forgotten), "capitalize the first character of a string", "reverse a linked list in place", and so on. The people that have been good fits for our team answered these questions without trouble.

The people that hemmed and hawed and didn't get a working answer (or claimed that their favorite language already had built-in functions for each of the problems) simply didn't get hired. If people don't do well on one question, we ask more; trying to find one they can do. They usually can't do any of them.

They couldn't show any ability to program, so we don't hire them to program. It's a large percentage that doesn't show any ability, but the people that do pass the interview are good to work with. I don't think it's unfair to expect people to produce code during an interview for a job where they will be producing code. Even if you're nervous, you should be able to do a little bit of your life's work, if only out of habit. You do this every day, after all!

(Another thing I like to ask: after describing your grandiose project from your last job, tell us what you actually did. Oh, you wrote the build scripts, not the custom hardware for parsing FIX messages? That's fine, but why'd you tell us that the design was "ours" for the last 20 minutes? Did you think we wouldn't ask what you did? We're hiring you, not your entire team.)

So anyway I think we've confirmed the OP's results. It's hard to find programmers.

timwiseman2• 16 years ago

I have been surprised when doing interviews how many people fail the basic soft questions. I have had people saying they were senior programmers not know what automatic testing was and who did not understand what an IDE was at all. (I have also had people tell me they preferred simple text editors to full blown IDEs, but they always knew what they were doing.)

And I concur with your basic concept. Even granted the pressure of an interview, if they cannot do the really simple stuff like fizzBuzz on the board or at least get an answer that only has minor syntax issues than I would be very dubious of their ability to program seriously.

alanh3• 16 years ago

EDIT: Has anyone else had OK experiences with on-the-spot programming as they were interviewed? </edit>

I don’t get all the people saying that most people can’t code under the pressure of an interview. What’s the big deal? It’s one function. I have certainly been asked technical questions in interviews, including to write code on a whiteboard, and it’s no big deal — I have missed a bracket or two, or been slightly off, but if it’s 95% right that’s good enough for the interviewer.

I’m not Mr. Cool, either. Finals, deadlines, and papers all really stressed me out in school (mere months ago). I can just code alright and I understand the tech.

I just don’t get it.

I also don’t understand what the alternative is: just take people at their word they are great programmers?

varjag2• 16 years ago

Depends on complexity. Would you like to write a 4-hour test assignment that boils down to partial sort of dependency graph (true story)? Oh and no pseudocode, it has to compile and run.

alanh3• 16 years ago

That sounds quite rough, but that’s not FizzBuzz or the kind of function described in the article. Good point, though.

cantastoria2• 16 years ago

I also don’t understand what the alternative is: just take people at their word they are great programmers?

We've had great success giving candidates a week to design and implement a small application from end to end. It really let's you see everything all at once. Can they design? Good comments? Good data structure selection?

It's the best of all worlds. The candidate has time to really show off their ability, you'll get a much more holistic view of their skills and it puts them equal footing with the interviewer because they've had a lot time to think about the problem. We found that whiteboard coding questions tend to favor "geek jocks" that are really just good test takers.

alanh3• 16 years ago

Doesn’t a little app seem like a lot of unpaid work to ask of prospective employees?

I’m not so sure about "geek jock" but I do happen to be a good test-taker. (I assumed most nerds were, though.)

arghnoname2• 16 years ago

I've been on interviews where the questions are so basic I've been variously confused and insulted. "This job must pay crap," I'd think, or it's pointless work. Now, thanks to the Internet, I've learned that these things can't be taken for granted.

And I'm not a coding genius. I don't particularly like being put on the spot. I've had problems coding before (personally) and just had to go away until the problem magically became easy. Under the pressure of an interview, that kind of lockup would be disastrous.

So it sucks, but apparently the industry is in a place right now where a lot of less capable people have resumes that suggest they can do the jobs for which they are applying to.

I object far more to the presumption that some company can waste my time for two days of non-stop interviews than ask me to code up some problem for them.

pkrumins• 16 years ago

Don't ask the candidates for silly resumes. Ask for their github repos and blogs. I mean who cares what your education, background, and previous job experience is? All of that is irrelevant. What matters is the code they have written. I want to see the code. The best developers will have thriving github repos and they'll be enthusiastic to write about what they are coding on their blogs. The perfect job application form has to have just two fields `Github URL: [.........]` and `Blog URL: [.........]`. Everything else is irrelevant. That's how I am going to hire.

sghael3• 16 years ago

I generally agree that it matters more what you've actually done or created, not what you say on a resume.

But specifically when it comes to asking for the Github url (or the like): that assumes that the code they have worked on is open source. There are plenty of good hires out there that may not have participated in open source but still have plenty of code they've created. I think asking for a public repo is too heavily weighted towards open-source wonks.

I'm content asking the candidate to point me to something they've created. I work on webapps so presumably for the candidate that would be a public facing website. And then i study it. And then I grill them hard on the details of their implementation.

substack3• 16 years ago

This is a good point but at the same point not many employers have this strategy so I suspect there is plenty of underemployment among people writing open source given the level of skill. The hiring strategy that pkrumins advocates seems pretty sensible for a business that is already deeply familiar with the open source world.

tomjen32• 16 years ago

I don't really see much point in writing open source software - either I keep it to myself (isn't really good enough to release, or only designed to solve my problem) or I spend my time doing contract work.

In addition I find git to be fostering bad habbits (no default push of private branches), and github to be an abomination (stuff that relates to my code and source control should not be accessed in a 3th party website, but in the tool itself).

substack3• 16 years ago

The point of releasing open source software for me is that I am contributing in my own small way to the state of human technological progress and to the massive canon of free software that I benefit from every day. Plus it's exhilarating to see other people become as excited about the code I've written as I was when I wrote it!

nradov2• 16 years ago

What pretentious nonsense. Most of the best programmers are working on commercial or government code that they aren't allowed to share, or sometimes even discuss.

logic• 16 years ago

Interesting (to me) observation: I found I spent significantly more time on the scaffolding for this problem to prove my answer correct, than on the answer itself. (I'm one of those folks addicted to "FizzBuzz"-esque toy problems.)

I probably spent five minutes at most working out a reasonable way to do this in-place with a single pass, but spent 20-30 minutes reassuring myself that I had it right for all cases.

These kinds of problems are always interesting to me: they have simple and obvious (after some thinking) answers, but probably aren't something you've touched in a significant amount of time, because nobody reinvents the wheel like this day-to-day. They're very satisfying in a "back to basics" kind of way.

orp• 16 years ago

When I interview programmers I start off with this question:

Write a function in C (on the whiteboard) that takes an array of integers and returns its sum.

I explicitly explain to candidates that this is a warm-up question to start the interview, and that I'm looking for the textbook solution, no tricks.

The advantages are: 1. It takes a reasonable candidate 3 minutes to write down the answer. 2. Answering a question correctly gives them confidence, always important in an interview. 3. It provides a good basis for a little advanced discussion (overflow, speed, error handling, supporting infinitely large numbers, etc).

Amazingly enough, some candidates have actually failed this...

ivenkys• 16 years ago

This is the right approach.

If an interview is like dating/flirting then you don't start by asking the prospective partner about their life philosophy you start by talking about the weather.

"Amazingly enough, some candidates have actually failed this..." - that is just sad.

timwiseman2• 16 years ago

When I have done interviews, I too was truly surprised by how many "programmers" could not program even simple things.

But I do think you should realize what type of programmer you are looking for. Most of their questions mentioned assume a knowledge of C. I have not used C since high school, even in college it was optional for CS majors, and a lot of programmers majored in things other than CS (I started in math, I know another truly good programmer that started in nuclear physics).

This is great if you are looking for C programmers, but I would not consider them basic things any developer should know.

heatdeath• 16 years ago

  void reverse_list(head) {
    struct node* current = head;
    struct node* last = NULL;
    struct node* next;

    while(current != NULL) {
      next = current->next;
      current->next = last;
      last = current;
      current = next;
    }
  }

wangwei• 16 years ago

Hello, RethinkDB. You get what you pay for. You don't expect to buy a diamond for 10 bucks , do you?

juicebox2• 16 years ago

You need to give a programmer a task, then let him get back to you in a day or two. Give him something that you can't find a solution to on Google, but isn't so hard that it couldn't be done in an hour or two.

Sitting there putting pressure on the programmer working is just a prick move. If you gave the same test to a carpenter and asked him to build a bird house and the outcome would decide whether he could pay rent this month, he would cut his damn hand off. Guaranteed.

Tycho• 16 years ago

Maybe you're doing things the wrong way round? Is there no way you can have potential candidates complete a programming task/test and THEN, if they're successful, send you their CVs? Whether using Codility or something you devise yourselves.

It bugs me that no matter how much I practice programming and know my stuff, my application might get glossed over in favour of some non-programming douche with an immaculate CV.

cellularmitosis• 16 years ago

It seems it would be great to have the inverse of codility, where job seekers take a bunch of coding tests, and then employers start sending them job offers based on their scores.

it was only after completing their warm-up problem that I realized that wasn't the case, and that in order to have access to any of the other programming problems (ie, in order to get to the fun stuff), I'd first have to set about finding an employer who has signed up with their service :(

16s• 16 years ago

If I was asked this question and I could express the answer in C++, then I would say list::reverse.

It seems (to me) that more important than this question is knowing when to use one data structure over another (pros and cons) is more important than being able to manipulate one specific data structure during an interview process.

Will the real programmers please stand up?

Related Stories