Tech Hiring: The Hunt for Highly Technical Superhumans

https://modelviewculture.com/pieces/tech-hiring-the-hunt-for-highly-technical-superhumans

by pyb • 11 years ago

21 11 11 years ago

tonyjstark• 11 years ago

I really hate live coding, for example I could do a small app with re-sizing table cells for iOS but animations I would need to read the documentation. So what? Why can't I read the documentation, am I a bad programmer now?

I had to design some tasks applicants need to do for hiring and it is not so easy to come up with something related to the work we do without being too complex or too easy. Whiteboard or live programming tasks are easier to find because they can be quite easy and the time pressure and nervousness make them hard enough. So as an applicant it probably tells me that the employer is lazy?

And I never understood interview marathons, I have friends in business jobs and they have that of almost every interview. It is exhausting and demoralizing, that also tells me more about the employer than it tells about the applicant. If an applicant does not want to do that, maybe she is not desperate enough or has other offers but maybe the job later is also exhausting because the employer does not think enough about the people at the company.

I don't think of me as a extraordinary programmer but I think if the interview is hard the job needs to be totally awesome to justify it. My experience shows me, it is not like that.

geebee3• 11 years ago

I'd like to start by thanking Anna (the author of this post) for identifying and writing about something that I consider to be a very serious problem in tech. While I disagree about some of the details, I'm in complete agreement that hiring practices (among other things) are a serious in our industry, and I feel increasing frustration with an industry that constantly talks about a "shortage" of talent while tolerating or even perpetuating practices that (in my opinion) clearly drive away talented people.

I recently went through the interview gauntlet, including (at one company) five hours of whiteboard exercises with a short break for lunch. While I can't post exact interview questions, I would say that the difficult level questions from "cracking the coding interview" and other sites are probably indicative of the difficulty level of my interview. For example "find the least common ancestor of two nodes in a binary tree" or "find the sub matrix with the larges sum in an NXM matrix" would be pretty typical questions (though of course you won't know the questions in advance). This is at the whiteboard, no programming environment to try things out, under time pressure, in what can only reasonably be called a stressful environment. Many people who took difficult in-person exams consider these to be among their most stressful academic experiences. In software, we consider this to be a normal part of changing jobs.

I agree with Anna that the amount of preparation is considerable. I brushed up on data structures and algorithms, but I only allowed myself three weeks (I'm employed full time, married, have kids, coach soccer, shuttle kids around to music practices, and so forth, so getting time to study isn't as easy as it used to be). I should have asked for three months. I did get the basics back, so I could implement various tree traversals, code up a hash map, do dfs and bfs on a tree, solve binary problems, but I just didn't get fluid enough to do this in a creative way at a whiteboard (On the bright side, I actually enjoyed the work, and I still study these things now. I'm probably much stronger now than I was when I did my interviews. I actually kind of like challenging data structure problems, it's a side of programming that I kind of missed).

But here's what I learned from the experience: where it comes to interviewing and getting a job, you really need to be exceptionally strong on the basics if you're going to be able to solve these kinds of problems at a whiteboard in a 45 minute interview. I know you can look up DFS, but only if you have a while to solve a problem, under less time pressure. I you need to use DFS to solve a related problem that you haven't seen before, in 45 minutes, at the whiteboard… you just need to know this cold. You need to be read to apply it cold. I'm not defending the interview process here, I'm just trying to explain to people what they need to do to be prepped for these interviews.

Now, I do have a few disagreements with Anna's article, in particular around the solution of the "take home". Although I'm sympathetic to the idea of a "take home" exam rather than an in person, and I see some benefits, ultimately, I don't like this approach. This is partly because of my own personal experiences - I once spent the requested 5-7 hours on a homework assignment, sent it back to the recruiter, and heard nothing back, crickets chirping, aside from checking in with the recruiter politely even week or so, until I got a one sentence call back (again from the recruiter, never did talk to a developer) "we've decided not to move forward with your application at this time". I think Gayle Laakman McDowell wrote a very good summary of how this practice can become abusive - ultimately, her argument is rooted in the way it allows companies to burn a lot of a candidate's time without a comparable investment themselves, here's the full link:

http://www.gayle.com/blog/2013/09/18/companies-who-give-cand...

To me, I think the first thing we need to do is recognize that any "shortage" that tech companies are experiencing needs to be considered in the context of how tech hires. A cynical part of me thinks that this process did exactly what it was intended to do, that it's not about whether a candidate can find all matching subtrees in a binary tree, it's whether that candidate can afford to drop everything to get into such mental/coding shape that they are able to do this at the whiteboard. If they can't, perhaps that's a sign that they have life demands that are incompatible with what tech companies are looking for?

This has been a long comment, thanks to everyone who read this far. I do think tech has a long way to go. The problems run much deeper than the solutions I hear about from silicon valley "leaders". Until tech addresses these problems on a much more fundamental level, expect those with choice (ie., those aren't compelled to study certain subjects and work certain jobs as a condition of getting to live and work in the US) to continue to avoid this field in favor of other professions and trades.

I do think that the recent focus on tech hiring is a positive sign that we're starting to look beyond superficial fixes ("make programming cool!") and more deeply at the process itself.

Here's another very good blog post that addresses this: http://www.fastcompany.com/3043082/most-creative-people/why-...).

NumberSix2• 11 years ago

There are two types of coding interviews that I have encountered in the last few years.

One is a grueling several hour interview on data structures and algorithms taught in college-level CS classes. These are typically data structures and algorithms that are almost never implemented in real work. They have been known for decades and are now incorporated in most computer programming languages, libraries, and toolkits. Experienced software developers with CS degrees typically have not used this knowledge in years and need to brush up. Experience software developers with other degrees or purely self-taught software engineers often never learned these at all -- or in a very limited way.

Thus this first type of coding interview is geared toward hiring new college graduates, very strong new college graduates, with a CS degree. It effectively discriminates against experienced/older candidates.

The other type of coding interview is a grueling test on specific languages, tools, algorithms, data structures etc. that the candidate would use in the actual job they are interviewing for.

Both types of coding interviews don't necessarilly identify the ability to learn and adapt to new technologies or many other skills that are critical in many technical jobs.

The coding interviews tend to favor candidates who have both the time and inclination to drill heavily on the typical coding interview test questions.

Contrary to the writer's comments, there are a large number of non-white "Asians" in software development jobs. Asian cultures and educational systems often favor this type of heavy drilling and practice on specific skills.

tomsun• 11 years ago

I had an interview with Salesforce as a new grad and they spent one round asking basic algo questions, one hour long phone screening asking specifics of Java, Python, C++, Javascript, one onsite whiteboard round designing tests with an in house framework for an in house product. And another round traversing trees and graphs. I didn't make it to the next two rounds, but man these engineering interviews are intense.

danny_taco• 11 years ago

While I agree that having you jump through these hoops is really time consuming and annoying, for most if not all companies it's a necessary evil in order to identify potential employees. How else do you assess a potential employee? It's gonna be either through whiteboarding/live coding with the interviewer(s) or through you portfolio, which your github repos and contributions serve as a good example.

Having said that, some companies take it to the extreme, and honestly they sound like a more traditional office type of environment where I doubt you want to work at from what I can infer from your comments.

I also do not agree with the assumption that only the privileged have time to contribute to open source (aka free projects). You either do it because a) you love it, or b) because you need to beef up your portfolio and you are motivated enough to do so. I'm considered a minority in the tech industry as well, yet I got into it many years ago by working on pro bono projects after going home every day from my telemarketing job at 2.am.

Either in technology or in any industry, you'll get your foot on the door and you will advance faster if you enjoy what you do. This is easily recognized by potential employers when they see you participate in hackathons and that you contribute to OSS.

Sure, I began doing free projects to get into the industry, but I keep doing it today because I love it and that's what makes it so easy for me to find employment as a software developer. In fact I get contacted by recruiters at least twice a week on linkedin and it's somewhat annoying. I hope Anna learns enough at her job to the point where she'll be able to easily find a job and be able to recognize toxic environments that make you jump through excessive hoops.

Just my 2 cents.

geebee3• 11 years ago

How else do you assess a potential employee?

This is a critical (and very fair) question. That said, I'm not sure that the options you listed are the only ones available. I'm not sure they're the best options, but I'm certainly not sure they're the worst, either.

How does it work in other knowledge-intensive fields? Physicians don't have to answer questions about organic chemistry at the whiteboard whenever they change jobs, but then again, physicians are subject to an extremely rigorous and standardized academic and apprenticeship path, which results in a a government backed exclusive right to practice. Actuaries don't have to pass exams on calculus or linear algebra every time they change job, and they don't strictly have to major in math, but they do have to take a long series of difficult exams that cover calculus and linear algebra, along with more advanced mathematics (and other more regulatory issues), and they must gain certain types of work experience. There are many other models out there.

I want to be sure I'm not coming off as endorsing any particular approach here. I do see some benefits to the current system - in theory, anyone who can study effectively and learn to do complex data structure and algorithm based problem solving at a whiteboard is welcome to take the google entrance exams. Or not. There's some good in that. However, I also believe that there may be more harm in it that our industry fully realizes.

Personally, I would be willing to take actuary-style exams, or some kind of board exam, if it meant that I could create a study path, truly devote myself to it, be assured a transparent, fair, and consistent evaluation. For me, the tough part is that I have to re-load all of this into exam ready memory every time I interview, which can be unpredictable, and happens over and over (I wouldn't object to "refresher" exams either).

I willingly acknowledge that what I described above is half-formed, and may have all kinds of problems I haven't anticipated. Personally, I do think that what we have is clearly driving some talented people away from our field, but it might also be attracting others. Hard to say.

What I am sure about is that we must consider any claim of a "shortage" in the context of what I consider to be almost a kind of hazing every time people change jobs. The links I provided below (the JOS one) was eye opening, we really do lose people to our industry because of this. I don't have an answer, but I really do think we need to see this as a problem.

pyb• 11 years ago

Just off the top of my head, a few possible hiring processes.

- Recommendations from people you trust

- Pairing interview

- Give a difficult/open ended homework problem

- Whiteboarding interviews are suitable for some candidates, they're not all bad ;)

- Pick from your list (http://sethgodin.typepad.com/seths_blog/2010/12/whos-on-your...)

- Asking the candidate for ideas as to how they'd best be assessed

- Culture fit ; Gut feeling ; or any conversational interview followed by review of performance after 1 month

- Portfolio review (no one actually does it in my experience)

- Hiring off reputation in the community

I'm sure there are plenty more.

NumberSix2• 11 years ago

The article raised some questions about how much time to spend for free on the interview process and also on free work whether at the whiteboard or "homework" for an interview process.

Here are my thoughts and experience and rules of thumb.

A rigorous interview process is something like:

1. send resume or CV

2. half hour phone or sometimes video screening interview

3. one hour in person screening interview (optional)

4. all day interview or equivalent with at least three different people including the hiring manager (unless there are fewer than three people at the company). This usually includes free lunch with the hiring manager and/or his team.

5. company checks references after the interview

A really rigorous interview is about 6-8 hours of the candidate's time, generally without compensation except for travel and often free lunch during the day.

In most cases, a lengthy interview process like this means about a 1 in 2 or at worst 1 in 3 chance of getting an actual job offer.

My general rules are:

Avoid phone screening interviews longer than 30 minutes unless the employer is out of the area. I once had a lot of 60 minute phone screening interviews that went nowhere. My feeling is if the employer is serious about you and in your area, they can and should meet you in person for a longer screening interview. Longer being over 30 minutes. Despite the many limitations of interviewing, it is easier and more reliable to evaluate each other in person than on the phone or even by video conferencing.

It is reasonable for a potential employer to have a longer screening interview if they are going to pay to fly you to an interview, provide a hotel, etc. Prefer a video screening interview to audio only if you can get a good video connection. Prefer the phone if the video connection is poor.

The potential employer should at least pay for travel including air faire, rental cars, hotels, any public transit. If they won't pay, they aren't serious and you should not pay for the trip either. A per diem or reimbursement to cover meals is nice, but not everybody does this and I would not turn down an interview if everything else was covered.

If they want more than at most eight (8) hours of free work whether whiteboard or "homework" to evaluate you, they should pay for your time. It should be a "try and buy" consulting project for them to get to know you and for you to get to know them. It is not just technical competence, but how well do the two of you work together; that is generally a bigger concern than technical competence.

Actually, I would be very cautious about more than three (3) hours of free "homework" or actual coding (not whiteboard) that they could directly use. If they are serious, they can pay a reasonable hourly consulting rate (e.g. $50-$100/hour) to try you out and vice versa. That is not that different from the cost of the time to interview you in person or cover travel expenses. If they won't pay, they probably aren't that serious and may simply be conning you to get free work.

In general, avoid doing work for free unless you get something else of lasting value to you from the free work such as something for your portfolio, lasting credit and recognition, or valuable experience you cannot get otherwise. Cash is king.

soham• 11 years ago

It's very important to understand three things about (tech) interview process:

1. Hiring at scale is a VERY different problem from hiring slower.

When we were looking for 25 engineers a quarter at Box or eBay, there were very few methods that would scale to that level. Asking data-structure/algorithms questions is one method that scales. Is it perfect? No. Is it the least evil? Yes.

2. An interview is not a test; it's a date.

Interview depends on the interview-er, as much as it depends on you. You may be fully prepared, but you may still fail interviews for no fault of your own. Just the way you cannot expect to get married with everyone you date, you cannot expect to clear every interview. The more you think of it as a test, the more you will agonize about it.

3. You have to prepare

Tech salaries are at peak these days. People on the other end know that. As a result, they are not going to give an easy interview. Especially true of hot companies. This is also the artifact/reason why it feels that most good companies are discriminating.

[About me: I've seen this machine inside and out from all ends, many times over. I now run a bootcamp to prepare candidates for technical interviews: http://InterviewKickstart.com]

geebee3• 11 years ago

"An interview is not a test; it's a date"

I think understand where you're going with the analogy, and I think that your advice not to dwell too much on any one interview is a sound. However, I do think it's important to recognize the extent to which a technical interview really is an exam. I'm not just quibbling about terms or analogies here, I'm bringing this up because I think a lot of the problems associated with tech "interviews" are related to the fact that they are exams, but don't observe many of the practices that support the student or examinee in typical exam-based institutions.

For instance, in college, an exam should be associated with a study path, students should be aware of how to prepare for it. While questions on a good exam should need not be shared in advance and may test creativity and problem solving, they shouldn't be out of the blue either, they should be related to the study path. An exam should be graded consistently and fairly, by people who are qualified to recognize a strong performance, and perhaps an approach that was not anticipated (a grader for a math exam, for instance, should be able to evaluate a proof that differs from the one a professor had in mind when he or she wrote the question). Students often have the right to receive feed back on their performance, and they often obtain a lasting credential, credit for a course, or other artifact that demonstrates that they successfully demonstrated knowledge of or competence in the subject matter.

Unfortunately, I believe that tech exams are very stressful, but don't observe many of these practices that I believe exist in universities and other exam-related institutions for a good reason.

The first step, I think, is to recognize tech interviews for what they are, which is a form of exam.

b_t_s• 11 years ago

What's with the section on volunteer work? Is that a thing now?

Tech Hiring: The Hunt for Highly Technical Superhumans

Related Stories