Recruitment Process for a Google Site Reliability Engineer

http://blog.lambda-startup.com/2014/03/recruitment-process-for-google-job-sre.html?m=1

by blackmac • 12 years ago

224 185 12 years ago

laurenstill3• 12 years ago

That meshes with the interview process my SO went through, at Google, for an SRE role. Similarly, they said he could code in any language (his preference is C) but the interviewer preferred python. 45 min to whiteboard in C, you better have a backup set of markers.

His feedback after all was said and done was "brush up on your scripting languages."

judk2• 12 years ago

Google recruiters give famously nonsensical feedback, sadly.

By comparison, Facebook recruiters give very specific feedback tired to reach specific interview hour.

B-Con5• 12 years ago

Neat, I just interviewed for Google for the SRE position as well. This described my experience very closely.

I found it amazing that for each of the interviews I was given enough information in advance to actually be able to prepare myself.

I had the same thought. To me, Google seemed 100% concerned with evaluating engineering skills and they wanted to do the opposite of hitting my weak points or quizzing me on trivia. I got a pretty good amount of detail on what to expect, loads of links, book recommendations, practice exercises, and even a video SRE interview coaching session by a couple SREs (all of which I combed through, yes, I even bought two of the books). Personally, it was a great interview experience.

I also interviewed with another large software-oriented tech company, but Google put more effort into providing preparatory material.

So the fourth interview (1:30 PM) on that day was the large systems design interview. Unfortunately, I was a blockhead on this one, then got nervous more and more and thus failed it.

It's also the area I felt the weakest in. It was the hardest to prepare for and they gave the least amount of prep material.

(Disclaimer: I start with them in 3 weeks.)

WestCoastJustin• 12 years ago

I got loads of links, book recommendations, and practice exercises beforehand (all of which I combed through, yes, I even bought two of the books).

Would you be willing to share some of their/your recommendations?

mden2• 12 years ago

I imagine they (Google) doesn't consider this information private so here's the bulk of an email I got listing prep material and topics: http://pastebin.com/iXGzfkBL

Edit: To clarify, this is for SE, not SRE.

BorisMelnik2• 12 years ago

yes also would be interested in this. Also even a vague example of what one of the questions might look like (not asking for an exact interview question)

objclxt• 12 years ago

There is a now classic Steve Yegge blog post that gives a recommendations and advice (a lot of Google recruiters actually point candidates towards it to prepare). Although it's over six years old now it's still pretty much on point.

http://steve-yegge.blogspot.com/2008/03/get-that-job-at-goog...

zevyoura• 12 years ago

I just started at Google two weeks ago (as a SWE, not SRE, though), and I found Cracking the Coding Interview to be the most helpful book for preparing for interviews in general. I also used Programming Interviews Exposed, which is comparable, but CtCI has a little more content.

raldi7• 12 years ago

A bit outdated, but still excellent: http://steve-yegge.blogspot.com/2008/03/get-that-job-at-goog...

P.S. It worked for me. :)

Good luck!

driverdan2• 12 years ago

I don't understand the prep part. If you have to do a lot of prep for the interview doesn't that mean your current skills aren't a good fit? And if they don't want to test your skills but your ability to solve problems wouldn't they want to test something you're already skilled at?

enneff2• 12 years ago

Anything you can study up for in a couple of weeks is also something you can learn on the job. What's the point of throwing a perfectly good candidate into an unfamiliar situation without any prep? When you actually get the job, there are plenty of people around to help you learn new skills.

B-Con5• 12 years ago

The interviews cover a lot of potential material in depth and I think the perspective is that not everyone works with all of those topics frequently and may need a refresher. Since Google knows what skills they want the interviewees to have, they share that information so the interviewee can round themselves out so that Google can identify their potential fairly. Like I said before, my experience was that Google tried very hard to avoid turning people away because they accidentally hit a weak spot or quizzed them on some bit of trivia.

For me, the preparation was primarily:

* honing the skills I did have to make sure I was comfortable using them in a tense situation

* refreshing and re-honing some things I hadn't seen in a while (some since college)

* filling in some gaps of knowledge (so I could connect the dots in an in-depth explanation better)

* preparing my thought process for the types of questions I'd have

The last one is subtle but important. If the interviewee has a good idea of what is expected of them, they can converge on a good path quickly and avoid going down rabbit holes or wasting 15 minutes trying to get on the same page as the interviewer.

As a side point:

doesn't that mean your current skills aren't a good fit

Don't they really care about your skills at the time of being hired (which they approximate by measuring them at the time of the interview)?

raldi7• 12 years ago

If you were casting actors for a play, would you rather hit each candidate with a surprise audition five minutes after they got out of bed, or give them time to review the lines and stretch their vocal cords?

nanidin• 12 years ago

One of my valuable skills according to a previous employer was my "google fu" - or my ability to condense any problem into something searchable and come out with a working solution.

This covered everything from design patterns to failing SSD's to static init order problems in C++ to arm assembly instructions...

Sadly every interview I've been in usually involves being a human compiler and key/value store for algorithms rather than being a human problem solver.

millstone• 12 years ago

I had the same question. I spoke to two Google SWEs about it, and also went through their interview process.

I concluded there's a number of factors at play:

1. Google views software engineering skills as fungible. If you can learn domain X, you can equally well learn domain Y. Hire smart people, and put them on something they find interesting, and you'll get high quality work out of them. Computer science / problem solving then becomes a sort of proxy for whatever skills are ultimately necessary. If you can learn that, you can learn anything.

2. Google wants to quantifiably improve its interview process, which requires that it be standardized. Tailoring the interview to the candidate too closely makes it harder to compare interviews across candidates. Computer science and problem solving are reasonable baseline skills for a SWE.

3. Having a famously difficult interview helps Google cultivate a reputation for exclusivity, which in turn attracts candidates. Same as why universities like to show off low acceptance rates.

But also some less charitable factors (which are by no means limited to Google):

1. NIH syndrome. Skills acquired outside Google are viewed as less relevant precisely because they were acquired outside Google.

2. Ego. I want to pretend my job is more about sophisticated algorithms and problem solving than tracking down null pointer exceptions.

3. Ageism. Whether chosen consciously or not, structuring the interview around topics taught in school / programming competitions, instead of on the job, slants the process towards recent graduates. Google has one of the youngest workforces among established tech companies, outside of Facebook.

When I've asked Google engineers about it, they've admitted that their interview process is imperfect, but also think it's the least bad approach known. At a minimum they deserve credit for working to improve it, instead of the haphazard approach employed at most companies.

prezjordan• 12 years ago

Admittedly I was extremely turned off by all the preparation materials. My instinct was that if I had to prepare and study for an interview, then I wouldn't be too qualified for it in my current state.

enneff2• 12 years ago

If that's how you felt then you're probably right.

raldi7• 12 years ago

When hiring, I'm a lot less interested in whether a candidate still remembers, e.g., the details of a red-black tree, and a lot more interested in whether they can talk about them intelligently a few days after they've had a chance to re-skim the Wikipedia page.

hibikir• 12 years ago

The best interviews for Software Engineers I've been a part of, on both sides of the table, were those in which a certain level of preparation was expected.

If I ask you random language questions, that you've not prepared for, I am not selecting you for being a good developer, but by being good at remembering language minutiae, which you could always google anyway. Now, if instead I hand you 2000 lines of code 2 days in advance, and tell you that we'll be working on them during the interview, I get a much better approximation of what you can and can't do in a real life scenario. Can you become very familiar with a tiny codebase quickly? Can you really analyze it critically, and tell me where it sucks? When asked to fix bugs, or make code changes, can you make the changes actually fit into the existing structure, or are you going to want to rewrite the world?

If you'd get a very different result in an interview if I gave you the questions I was going to ask an hour in advance, then I am not testing work skills at all. You don't hire a juggler by seeing if he is good at playing basketball, you watch him juggle.

wyclif2• 12 years ago

Care to share the book recommendations? I'm curious because I've been contacted by Google recruiters before, but have never done well enough to be hired. I'd like to get to the point where I was SRE material, even if I were to be hired by another company and not Google. I've never seen any reading material suggestions or videos.

aiiane10• 12 years ago

3 weeks, eh? Shucks, I won't be teaching your orientation class. Should have started this coming week instead. :P

Theodores• 12 years ago

Well if there are any Google people reading this: ask HR to give the guy a job! He went through all that and he came out positive about it all.

skj6• 12 years ago

So, it seems likely that culture fit was not a problem.

In general if you get one interview that results in a weak no-hire, you need a couple strong-hires to balance it out. If you get one with a confident no-hire, that's it.

yeukhon• 12 years ago

Are you interviewing for the system engineer or software engineer on the SRE team? The 2nd phone interview covering three types of tasks seems unexpected to me. I am also preparing for SRE interview and I now feel chicken out... I'm interviewing for SE on SRE. I have been expecting just algorithm/codeval type of questions until onsite.

B-Con5• 12 years ago

IIRC, SW and SE interviews in SRE are about 80% the same, the primary difference is that one of the 5 on-site interviews is on a different topic and the phone screens may emphasize slightly different topics. But the tracks for the two positions are very similar.

darksim905• 12 years ago

Does anyone know if this process holds true for the Sysadmin SRE side?

k3oni2• 12 years ago

There is no such thing as a "Sysadmin SRE" , it is just SRE. Went through it and was the same, granted i didn't make it to the on-site interviews as i screwed up on the last call after a really bad day at work.

gergles3• 12 years ago

This is not true - there definitely are 2 SRE tracks[1] and one is more code focused and one is more operational focused. They do not tell you this while you are being screened but it is absolutely the case.

1: http://rachelbythebay.com/w/2011/11/22/caste/

aiiane10• 12 years ago

The process itself is essentially identically whether you're on the SRE-SE or SRE-SWE track. The specific topic areas focused on will vary slightly, but the process is the same.

(SE = Systems Engineering, SWE = Software Engineering. People hired from both tracks wind up doing the same work; it's mostly just about what background you're coming from.)

ryguytilidie• 12 years ago

I used to recruit for Google SRE, I can confirm that this is pretty much exactly the process and that bombing a single interview will often be a death sentence.

laurenstill3• 12 years ago

Thanks. I've turned down a couple interviews knowing I'd bomb at least one, didn't want that on my perm Google Record. It's nice to know it's not just me, and also there's redemption.

skj6• 12 years ago

This is a bad strategy. One thing that actually helps you tremendously is evidence of growth. The easiest way for the hiring committee to see this is for you to show improvement from a previous interview.

I guess it'd be possible to game this signal, but that's a lot of effort and still just one signal.

My real point is that it absolutely does not count against you.

raldi7• 12 years ago

By "death sentence", are you suggesting that the candidate will never again be considered for a role at Google, or are you just saying that they're not going to space today?

laureny• 12 years ago

Being rejected at an interview at Google means you have a one year cool down period. You can try again then (or they might call you back themselves).

BlackJack• 12 years ago

This Jobvine infographic describes the hiring process pretty well imo: http://www.jobvine.co.za/what-does-it-take-to-get-a-job-at-g...

aiiane10• 12 years ago

The "crazy questions" there are completely wrong - Google interviews do not contain questions like these (and any interviewer that tried to use them would be run out on a rail).

nawitus2• 12 years ago

Google used have those, but recently stopped asking them. The infographic is old.

tmhedberg3• 12 years ago

I'm a Google SRE that had a very similar experience. My best advice would be not to sweat the rejection.

I was also contacted by a recruiter based on an open source project I had contributed to. I went through the same series of phone interviews, culminating in an on-site in NYC. I left there feeling largely positive about my chances, but a few days later, I was politely rejected. I was not that broken up about it, as I already had a job that I liked, so I just counted it as good interview practice and moved on.

A year later, almost to the day, the same recruiter called me up out of the blue and asked if I'd be willing to try again. I agreed, and after an abbreviated version of the phone interview process, went to Mountain View for another on-site. Soon after, I was hired!

It's actually very common for Google to reject candidates the first time around, as the interview process is deliberately tuned to produce a lot more false negatives than false positives. We have that luxury thanks to the volume of applicants we receive (there are still a surprising number of Nooglers starting each week despite the selectivity). The hiring committees recognize this tendency to reject qualified candidates and won't count you out after one try. If you got to the on-site stage, then rest assured that your interviewers took you seriously as a candidate. If you've decided that you would really like to work at Google, you will still have a good shot if you try again in a year or so. And if not, then hopefully it was at least a fun challenge and a free trip to London.

yukichan3• 12 years ago

But why go through that? Google is just another place to work. If I got rejected from a job at HoneyWell or G&E I wouldn't be thinking to myself well in one more year I can try again. I'd be thinking about the next place to apply and never go back.

The guy was obviously qualified for the job and they still rejected him because he got nervous. He had done well except for one and they said no. I guess if you have so many people interviewing where you have some that did better it makes sense, but it's silly and personally I don't plan at working at companies that interview this way.

Google isn't special. They're just a well known consumer brand, and all that marketing is what's got into people's brains. Coke is just sugar water, it doesn't make you cool. It just puts fructose corn syrup into your stomach. You wanna be the person that has to obey Larry Page's whims and integrate Google+ into more places users don't want it?

toomuchtodo3• 12 years ago

Google isn't special. They're just a well known consumer brand, and all that marketing is what's got into people's brains. Coke is just sugar water, it doesn't make you cool. It just puts fructose corn syrup into your stomach.

It takes you a while to get to this mindset as a technology worker. Everyone's motivation is different though.

I'm 31, have 13 years of experience, and keep getting rejected by SpaceX for a Linux Admin job at launch operations Cape Canaveral. Yes, I'm overqualified. Yes, I'd be taking an enormous paycut. But I want to help send rockets to Mars damn it.

It's not always about the money, nor about the work. Sometimes, you're simply irrational about it.

greenrice• 12 years ago

You're kidding yourself if you think Google is "just a well known consumer brand"...

asdfologist• 12 years ago

Right, being a company filled with some of the best and brightest engineers in the industry working on technical challenges of massive scale makes Google just like any other company.

tmhedberg3• 12 years ago

If you don't think it's worth your time to interview twice at the same company, then simply don't. That's a completely reasonable point of view, and I didn't mean to imply otherwise.

For what it's worth, Google has a public reputation as a great place to work and as a company that hires "high quality" engineers. Both of these things mean that some people are willing to put more effort into getting a position there. My post wasn't meant to say that he must keep trying at all costs, but just to let him know not to be discouraged if he happened to really want that job. In many places, you're dead in the water if you don't make it the first time, but Google is not like that.

Personally, I didn't see my interview process as "something I had to go through", i.e. a laborious means to an end. I enjoyed the challenge and the opportunity to get a glimpse of a company like Google from the inside. Even when I was turned down the first time, I came away feeling glad that I had done it. It's not like I had anything to lose from trying.

You wanna be the person that has to obey Larry Page's whims and integrate Google+ into more places users don't want it?

Not in the least, nor do I feel that I am doing that. I don't work on Google+ or anything related to it. However it may look from the outside, Google is not Google+. It's a big, multifaceted organization with opportunities to work on all sorts of interesting things. Much of our work is driven directly by the engineers themselves and not by management whims. And there's plenty of mobility to change roles if you decide you don't like what you're doing.

SRE specifically has proved to be a truly interesting and unique position. There are engineering challenges that we face which quite simply don't exist anywhere else. Beyond the much-touted perks, that's what makes Google special, in my opinion, and well worth the comparatively small effort I put into getting there.

yuhong• 12 years ago

On Google+: http://www.businessinsider.com/sex-and-politics-at-google-it...

mcintyre1994• 12 years ago

It seems from the parent like a recruiter just following up when they could. That seems smart - there was a level of interest so why not see if it's still there? A lot could happen in a year to make someone more (or equally, less) likely to decide to give it a go again.

dc2447• 12 years ago

I have been approached a few times to interview for "Director, Site Reliability Engineering", which as far as I could tell was leading up the SRE team in Europe.

I had a few conversations but did not take it further. My reasoning was not going further was not the nature of the interview process, although I found it a little strange that they expected a director level position to know how many bits were in a mac address or nature of google as a business but the description of what the engineering team did for the majority of the time. Rather than building 'stuff' the team seemed to be involved in very low level debugging on the Google infra and apps.

I know somebody has to do that stuff but it's not something I was very sold on when it was described to me.

Incidentally the role is still being advertised so maybe finding engineering directors that know how many bits in a mac address or what the default signal sent with a kill command :)

http://www.linkedin.com/jobs2/view/10632280

iDemonix3• 12 years ago

I'm asking this question of all SRE's in this thread: How did you develop the necessary skills required for an SRE role? It's my dream job but I feel I'll never be intelligent enough. My current job in an ISP NOC doesn't give me tons of room to develop the skills required for an SRE role, I script/code and create small projects outside of work, but the entry level for SRE just looks so high compared to what I feel I could ever achieve.

aiiane10• 12 years ago

Reading is a good place to start. Find places where people describe their system architecture, and work your way through it, trying to understand why they make the choices they do. If you come across concepts you're not familiar with, look them up on Wikipedia or Google and keep digging until you do understand them. Ask questions if you need to - StackOverflow and ServerFault are good for this.

nerfhammer• 12 years ago

The fact that the same person going through the same process twice gets a completely different result strongly suggests that luck and randomness is a large factor, despite the apparent thoroughness and time investment.

I've advised people on a job search that the way it works with tech hiring is they ask you a few questions and either you'll happen to be able to get the answer quickly or you won't, and if you go on enough interviews eventually you'll get one where you get all the answers quickly and you will look really smart as a result. Consequently, you can't pin your hopes on or predict whether you would be able get hired by any particular company.

aiiane10• 12 years ago

Randomness is a factor whenever you have social interaction; as it turns out, people are subjective to the core. Trying to gauge someone's ability to deliver results is educated guesswork at best. You can only be so objective about it; there's going to be some measure of variance.

Google errs on the side of rejection because one bad hire has a much bigger impact than one good hire.

gaius3• 12 years ago

Yes, I've said it before, Google is what you get if you hire essentially at random and then tell all those people that the process that hired them is infallible.

bostik• 12 years ago

I was contacted right after the Elopcalypse, when all Nokia engineers and subcontractors were fair game and easy pickings.

Had one recruiter phone screen with some easy questions, a technical phone interview and then was informed they decided to skip the second phone interview. Got called on-site in Dublin directly instead.

My experience from there pretty much resembles what the author describes. First interview was about my knowledge of C, memory mappings and the related security implications. Let's just say that it helped I had experience from both little-endian and big-endian architectures. The second one was the fairly well known python interview. The problem is devious but really interesting. I botched that one, because I failed to follow my instinct and sketch a visualisation out first. That caused my attempts to derail pretty badly and by the time I realised I would need to rethink and simplify much of the logic, we were out of time.

Third one was a fascinating trouble-shooting problem. It was not just about the technical problem or the symptoms, but also about how to deal with people who may have these kinds of problems due to being frightfully clever at getting themselves set up with the problems in the first place. I think I nailed that one. We had a nice chat about the history of the problem with the interviewer because we had some time to spare.

Fourth one was a system design problem, and it was a true pleasure. The interviewer wasn't as much asking me questions, as he was more laying out the problem and the proposed architecture. We went through the requirements, limitations and he even showed me one neat trick which I hadn't ever thought before. (The actual architecture in question was effectively a DHT with an interesting twist. I thought it was brilliant.)

The fifth one was basically about how well I understood the system internals of unix and linux. However, the approach chosen was not the off-the-shelf one I've seen elsewhere - the dive into internals started with task_struct. Yes, the one which nobody is supposed to understand completely. (At least according to Robert Love in Linux Kernel Development.) I got it right, evevntually.

I was, of course, rejected due to "not good enough at coding". I don't hold that against the interviewers or the process. I know I'm a slow starter with any code - and I seem to suck at whiteboard coding. Whiteboard is great for doing visualisations and writing down short snippets where they are relevant, but for actual programming it just doesn't work for me. My most important design tools to this day are a large paper pad and a good pencil. After that it comes down to debug logs...

What may sound strange, is that the on-site day remains one of the most enjoyable experiences of my life. The interviews were designed to keep my brain spinning at overdrive, and the interviewers themselves were good enough not to actively mislead (they did keep me asking questions and stating the reasons for my approaches, which was crucial). Loved it.

I've told all this to my hacker-type friends and still remain convinced that any engineer worth his title should experience the Google on-site day. It's a challenging, but also extremely satisfying experience. Some of them have tried, and at least one has been actively courted. I'm convinced he would ace the interviews without a hitch. The requirement to move is the one thing keeping him away.

23david• 12 years ago

Google's SRE team will continue to get people coming straight out of college, but for people with startup industry experience I don't see how it's an awesome opportunity. Most of the interesting SRE work and innovation that I see today is happening at startups.

If Larry or Sergei call up personally and give you a blank check to start your own team/project, (and let you open-source your work...) maybe that'd be a different story.

But otherwise, seems like working elsewhere will be the better option if you care about experience/skills over climbing the corporate ladder and having an easy paycheck.

philsnow3• 12 years ago

Most of the interesting SRE work and innovation that I see today is happening at startups.

Can you give some examples of some pieces of interesting SRE work that is happening at startups ? I'm curious.

I think the SRE role is still misunderstood a bit outside Google. Few startups that I'm aware of (there are a few, mind) care enough about "Reliability" to make a dedicated reliability engineer among the first 50 employees.

omegote• 12 years ago

Lots of comments already here, but anyway I'll try.

I interviewed for a SRE position in Dublin half a year ago, I made it to the on-site interviews but after that I didn't get hired. I agree with pretty much everything the article says, specially the concerns about the interview about Large Scale Design. That one is pretty weird, it's the only one I couldn't really prepare. The only thing I don't agree with are the ending conclusions.

First, I wasn't told anything about where I had failed. It'd have been great to know it, so I could prepare better for the (unlikely) next time.

Second, maybe I'm a pessimist, but I don't feel like _I can compete on this very top level of computer engineering_. I feel more like I was quite lucky to get where I got. Maybe (in fact, I hope) it has something to do with the Dunning-Kruger effect...

BorisMelnik2• 12 years ago

Excellent write-up. I love your attitude about the whole thing.

One part I found interesting and note I have not interviewed for anything in probable 10 years is the fact that they would not attempt to place you into another position being that you passed the majority of the questions with flying colors. Actually my only criticism is you used a ton of ;)'s :-0

So you are an excellent coder, scripter, and network admin. You also know all the OSI layer stuff but aren't good at "large scale" sysadmin functions? If this were my company I would try to have you placed in some other department and postiion.

Either way, thank you so much for sharing your experience!

aiiane10• 12 years ago

Sometimes that does happen - interviewers can always provide feedback that they think someone should be jumped over to a different track and re-interviewed.

DigitalSea• 12 years ago

This is ridiculous. I understand Google is a big company and thus needs to implement a concise recruitment process because understandable they get a lot of applicants, but the amount of tests and ways of which you were asked to solve problems is ridiculous.

When is the last time you heard of a system administrator writing down problems on a whiteboard as opposed to asking a colleague or Googling the answer? I think the real test after the initial phone call interviews and Google docs one is to sit you down in-front of a computer and make you solve real problems, not theoretical & made-up problems in which they ask you to solve them in impracticable and unrealistic ways.

This is a flaw in the corporate hiring process almost everywhere in the software world. It's not the 70's any more, people rarely solve problems on whiteboards and paper. They solve them on the computer, sometimes through knowledge and their skill-set and other times through luck and Googling.

Don't feel too bad for being rejected, it just means something else could come along that's better.

uhno• 12 years ago

It's not the 70's any more, people rarely solve problems on whiteboards and paper. They solve them on the computer, sometimes through knowledge and their skill-set and other times through luck and Googling.

This isn't universally true. Anecdotally, I often find I'm much better able to think through tough problems if I step away from the keyboard and spend some time sketching out ideas on paper or on a whiteboard. I also keep the on-paper results in a notebook, which is occasionally useful to refer back to later in a project.

Sometimes just introducing some distance between you and the problem is enough to give you a key insight. That said, the interview environment is still nothing like this. There, you're under great pressure on the whiteboard, something which is probably not true in your day-to-day.

_delirium• 12 years ago

I'd like to think this is true, because I enjoy working in that manner, but I've generally found going totally offline and solving problems gives me satisfyingly worked-out-by-myself solutions to problems that... I could've solved more quickly if I'd done more reading for half that time instead.

If something really, truly has never been done before, nor even anything similar enough to be useful to me, then yes, this is the right way to work. But it's more common that someone has actually worked on something at least related (even if not quite the same), and that I could solve the problem more quickly if I looked for what they said about it first. That might take a bit of searching and reading to discover, sometimes even a few hours of it. But usually not as much time overall as re-solving it myself does... especially taking into account re-discovering all the edge cases.

My hypothesis is that many people don't realize this because they never follow up later to check if their solution was really novel, or was just lurking behind a keyword they didn't think to try. If you do that a bit and adapt your habits to miss things less often in the future, you can get better at finding and adapting existing solutions, rather than re-inventing things from scratch. But that's sometimes a bit deflating, because then you realize you weren't inventing so many new things before, either...

mark_l_watson2• 12 years ago

If I really need to think about a tough problem, I tend to get a pad of paper and go sit outside on my deck and think about it away from a computer. Sort of an "astrophysicists are not in the telescope business" thing :-)

jodrellblank• 12 years ago

When is the last time you heard of a system administrator writing down problems on a whiteboard as opposed to asking a colleague or Googling the answer?

"You're a site reliability engineer at Google. Google is offline and you have to fix it, what do you do?"

"I'd Google the answer"

"Try again"

"I'd ... Bing and decide?"

both laugh

fourspace• 12 years ago

Former Google SRE here. Trust me, many of the problems we faced were solved on whiteboards, during chats at lunch, or via email. Most of issues we faced were of a class that simply isn't covered on basic sites like Stack Overflow.

The interview described is pretty much exactly as I remember them. SRE is actually quite difficult to get into, precisely because you need to have fairly deep knowledge on a wide range of topics. The "ways in which you were asked to solve problems" are actually the best way to determine if an application actually knows about what they will need to know.

zobzu4• 12 years ago

Heck i'd be worried if your real issues were the stuff solved on stackoverflow :P not that stackoverflow is bad, but the stuff there is generally a basic/quick reference.

raldi7• 12 years ago

I'm a Google SRE, and the first place I go when faced with a complicated problem is the whiteboard (if collaborating with others) or the legal pad (if I'm trying to figure it out myself).

If collaborating with others in a remote office, I begrudgingly make a Google doc and treat it like a whiteboard.

Most of what we do involves enormously complicated systems with lots of nodes and RPC flows between those nodes, different pathways depending on the flavor of the RPC, systems spread out in different metros around the world... it's very difficult to visualize all that complexity in your head if you've never seen it on a cocktail napkin.

For example, just last week I sketched out a diagram inspired by the famous visualization of Napoleon's campaign to Moscow (http://www.aviz.fr/wiki/uploads/Research/minard.jpg), primarily to help me wrap my head around a complicated RPC flow (at the first node, 32% of the RPCs are classified as XYZ. Each of those spawns 3 new RPCs that go here and 1 that goes over there...) When I got stuck, I called over a colleague, and he was able to immediately see, just by looking, where I was going wrong, and with a few strokes of the marker, set me straight.

I later turned it into a spreadsheet so I could use it to explain the model to others. Also, it was nice to be able to use worksheet functions to do the math. But I never would have been able to get that far that fast without starting at the whiteboard.

curun1r2• 12 years ago

Having heard numerous accounts of the Google hiring process, both successful and unsuccessful, I'm convinced that it serves a dual role. In addition to brining in qualified employees, I believe it's intent is to reject qualified candidates. Those qualified candidates will end up being engineering leaders in companies that Google interacts with and the more that they can maintain the impression that Google engineers are the best of the best, the easier those interactions will be and the better received Google products will be. It also means that when Google calls and invites someone to interview, they almost never get turned down.

hueving2• 12 years ago

they can maintain the impression that Google engineers are the best of the best

This is not true for everyone. I see it as more of a failure on Google's behalf to create a good selection process. The flawed assumption you're making is that, since they have a noticeable false positive rate (i.e. good people getting rejected), they don't have false negatives (i.e. unqualified candidates getting offers). There is no guaranteed correlation between false negatives and false positives.

To carry this a little further, I would argue that it's very likely that some bad engineers get into Google because, by definition, their selection process is not correctly picking good engineers - just a rough approximation of what they think makes a good engineer.

turnip1979• 12 years ago

Not true. I'm not a big shot but I've turned them down. Had two rejections from them in the past .. don't want to waste my time.

rurban• 12 years ago

If this is the actual goal they are not even bad at hiring, which I can testify personally, they are also bad to be hired at. They cannot offer interesting jobs, their hiring practice and personnel sucks, their pay is not really competitive. Who wants to work for Google if you are >30 and experienced enough? You get free meals elsewhere also.

vidarh3• 12 years ago

I've turned down Google recruiters multiple times. The one time I started going down the interview process at Google for a position in London, the experience was so bad that the recruiter got the technical interview set aside (the guy interviewing me would have reported to me had I taken the position, and frankly based on our interaction, had I been interviewing him I wouldn't have hired him - that's not a good first impression), and invited me to do a second round interview.

I turned her down because I'd gotten a really bad impression of the whole process. No other company I've interviewed at have managed to be nearly as Kafkaesque in the hiring process, and several of the Google recruiters I've spoken to over the years have vented their frustrations about the process at me when I told them this, while non-Google recruiters have gleefully told me they hear this a lot and consequently see less and less competition from Google for candidates.

I'd consider a request from a Google recruiter for an interview again, but the threshold for me to bother starting down that route again has gotten higher each time - I don't feel Google is worth the hassle unless they were to approach me with something exceptional.

skj6• 12 years ago

You seem to think a Google SRE is something like a sysadmin. This is not true.

d0m• 12 years ago

Well, you laugh but yesterday I googled something and it was down. First time it happened to me.. I tried twitter, facebook, both up. Tried again google and it was down. So well I tried bing and got my answer ;-)

toomuchtodo3• 12 years ago

From all the posts here, it doesn't like a sysadmin. It sounds like an underpaid C++ DevOps engineer role.

incision• 12 years ago

"When is the last time you heard of a system administrator writing down problems on a whiteboard as opposed to asking a colleague or Googling the answer?"

Once you move beyond roles which are effectively operations, maintenance or been-done implementation to anything that can be called engineering you'll start to face problems for which there are no canned answers.

"It's not the 70's any more, people rarely solve problems on whiteboards and paper."

This has nothing to do with era, it's a matter of scope and scale. When you have problem that is big / complex enough that it can't be reasonably solved by a single person a whiteboard is invaluable in laying things out and thinking them through.

"They solve them on the computer, sometimes through knowledge and their skill-set and other times through luck and Googling."

Where do you think all that helpful information on Google, or anywhere else actually originates? When the folks at Google were in the process of engineering GFS do you think they just Googled "chunk replica placement" and hoped to luck into a StackOverflow post on HDFS from 10 years in the future?

nknighthb• 12 years ago

people rarely solve problems on whiteboards

I don't know what you do or where you work, but it's no job I've ever had, nor any office I've ever been in. It's not unusual to end up spending most of a day in rooms with whiteboards and a few colleagues working through problems.

aiiane10• 12 years ago

Another Google SRE chiming in here... there's a reason why I swear half of the flat vertical surfaces in our office are whiteboards. Including some of the walls.

I'd estimate that no more than 50% of job is spent coding, if that. Solving computer/software/system/architecture problems, yes. Coding, no.

fortuitous• 12 years ago

And you have every right to be grumpy, you're supposed to fix problems engineers create, this is why we don't get mad at you. I feel like there should be no difference between sre/swe. swe's should be responsible for the problems they create. That's how it worked at my previous jobs.

gaius3• 12 years ago

It's a covert form of ageism. The more questions you ask that are like undergraduate puzzles and less like real world situations, the more likely it is that your "objective" recruitment process will filter for recent graduates.

meisterbrendan• 12 years ago

Is this true? It seems, from the comments from the SREs in this thread, that these puzzles are highly relevant to this functional role.

mark_l_watson2• 12 years ago

I don't think there is any real doubt that Google's application process favors bright young people right out of (usually very good) schools, rather than people with 20+ years of experience. And, that is fine because they are a business and have found that this process works well for them.

patmcguire• 12 years ago

I've been on a few interviews with the "not-a-test" lunch, and it always made me grateful to be in this industry. I talk to people applying for law jobs, where it seems that lunches and dinners during the process are very much a test. Associates fill out scorecards for candidates later, on how good a conversationalist the candidate was! On a one to five scoring system, across multiple categories!

From a certain point of view it makes sense - how are they going to schmooze with clients if they can't schmooze with you, but... shudder

mhurron• 12 years ago

Those lunches are a test. They're for 'culture fit' questions.

tmhedberg3• 12 years ago

The Google lunches are emphatically not a test. It's very rare for the lunch "interviewer" to provide any feedback at all. That usually only happens if the candidate starts spouting racist comments or punches someone in the lunch line, etc. The real purpose is to answer any questions the candidate has about culture, perks, etc. and to give them a chance to relax, catch their breath, and feel at ease in the middle of a stressful day of interviews. If the candidate was referred by a Googler, then often that person will be the one to take them to lunch, which would obviously create a conflict of interest if real feedback was expected.

skj6• 12 years ago

There is no feedback expected from those lunches.

But you're right, they are tests... for the candidate to test Google.

raldi7• 12 years ago

As a Google interviewer, I can tell you with certainty: The lunches are absolutely not a test.

However, "culture fit" can play a role in a different way: the actual interviewers (the ones who aren't taking you to lunch) will be given the opportunity to chime in on whether they think a candidate seems Googley, and this is often factored into the hire / no-hire decision.

pmb• 12 years ago

I have been a lunch "interviewer" at Google. I was never asked to provide any feedback.

zobzu4• 12 years ago

yeah they are. people will tell you that this doesn't qualify as a test.. its wrong. its not a technical test.

but if they don't feel comfortable with you at lunch, or you come out as a jerk/asshole for anything, you're not taken. it makes sense.

its not a hard test but its a test.

raverbashing2• 12 years ago

So, yes, Google hasn't changed a bit. Curious about where the interview took place (I don't think there's a hotel across the street from Google in Dublin)

I'll just point them to this article whenever this kind of questioning happens again: (https://news.ycombinator.com/item?id=7373310 - questioning is the first comment)

michaelmior• 12 years ago

Around a week later the recruiter called and once again reported "very good" feedback, so I could go on and have an on-site interview in London.
The hotel was right across the street of the Google building, which I really liked a lot, since it meant I didn't have to find my way through London in the morning before the interview.

Seems like it was in London.

raverbashing2• 12 years ago

Oh Well, I missed it. I blame it on it being in the very end of paragraphs.

theboss• 12 years ago

I have an interview with Google this week. I've expressed my interest in security to the recruiter 3-4 times and I have been told I will be interviewing as a software engineer (aka....a data structures and algorithms interviewing...)....

Doesn't make sense to me, the things they do. I've had bad experiences with several of the big tech guys now...

Too bad you were denied but I'm sure it will all work out for you. In my opinion computer science is really cool because if you want to be really good you don't have to work at a big tech company. You can learn in your cube, learn at home, learn wherever. It is so easy to network and be with other smart people that you don't need them and they need you.

justinhj• 12 years ago

I don't think one should be surprised when they focus on the things you claim to be good at. They probably have a large pool of interviewers, so they can choose one with a deep knowledge in your specialist areas to chat with you.

octonion• 12 years ago

How come only programmers and data scientists are treated to such a ridiculous process? Why aren't managers and marketers given pop quizzes and domain-relevant puzzles to solve?

skj6• 12 years ago

Lower-level management is drawn from the same pool as the engineers.

For upper-level, Google does not have the luxury of getting one thousand credible and serious applicants every week, so the process necessarily differs.

judk2• 12 years ago

That's begging the question. The eng interview process is an artifact of how they choose candidates. So is the VP interview process.

raldi7• 12 years ago

What are you referring to by "pop quizzes"?

FLUX-YOU• 12 years ago

Puzzles simply aren't in fashion for those industries.

vidarh3• 12 years ago

Because nobody else would put up with it in sufficient numbers, and then spend ages discussing it in places equivalent to HN afterwards.

It would make it impossible for them to hire. A large segment of potential tech hires salivates at the prospect of being hired by Google as some sort of intelligence test or proof of their self worth, and the convoluted hiring process serves to reinforce that impression. While it puts some people off, it draws others in.

For higher level management positions, marketing and other types of roles, Google does not have nearly the same draw. And for higher level positions the pool of potential candidates is much smaller too.

marinhero• 12 years ago

Wow! thank you for sharing your experience, I've just started the interview process for Google SRE and I found this very valuable. It looks like a tough road but I'll do my best. Thank you!

thecolorblue• 12 years ago

Google's hiring process seems very intense and extensive. Are there other tech companies that go to these lengths to find the right people? I completely understand their reasoning, but I have never applied at a company that had four interviews.

philwelch2• 12 years ago

A full day interview loop is standard procedure at most large tech companies you would have heard of.

vidarh3• 12 years ago

I had 6 on site interviews when I joined Yahoo Europe back in 2003 for an engineering manager position. Developers back then would usually go through fewer, but still 3-4 wouldn't have been unusual depending on team - when I hired for my team at Yahoo, typical process would've been phone screen, then 3-4 on site interviews with a selection of people including my manager, someone from the product team, one of my developers, and me.

It's a function of size, frankly - companies above a certain size tends to tack on more interviews largely because they can, and it doesn't hurt to get feedback from one more person. They also have the luxury (and problem) of a much larger pool of interested candidates to hire from.

calinet6• 12 years ago

Google's interview process in short is laser-focused. Whether this is good or bad is up to them and their goals (and clearly something is working), but personally I find it incredibly narrow-minded and inhuman. Wouldn't want to work there.

zobzu4• 12 years ago

it was some years ago [actually, a lot of years ago now..], but when i got interviewed for an SRE google interview i certainly wasn't given any indicator of what to prepare for ;-)

I also think many of the recruitment questions were above the skills of the employees asking the questions. they just had a checklist and basic understanding.

Now i would probably do smth similar if i was google and a got a lot of applicants, but it felt weird.

I think what was the weirdest tho, was how they made me feel like the SRE job was very stressing and more of a throw away tool used so that, luckily, i could switch to another position after 13month (1 cycle).

That sealed the deal the wrong way for me. I just told the guy we should stop the process now so that nobody wastes their time and went home.

iDemonix3• 12 years ago

This has long been my dream job and I have alerts setup for when they're interviewing - but I don't feel I'll ever be smart enough or well rounded for this specific job. I left university a couple years ago and have been working in my first job (an ISP NOC) for about a year, but I just can't see myself ever getting to the skill level required for an SRE. That and I have crippling anxiety when it comes to interviews.

Ah well...

aiiane10• 12 years ago

Something that not many people realize: interviewing is a skill that can be practiced and directly improved. The more interviews you do, the easier they get. You can also do dry runs - even just grabbing yourself a small 20" whiteboard and writing code and diagrams on it helps.

iDemonix3• 12 years ago

That's good advice, I always tend to focus on the skills and see an interview as something that is done ad-lib. I forget how much of it is prep and the actual practicing of interviews.

graycat• 12 years ago

The job apparently was not 'reliability engineering'.

In an auto analogy, Google was trying to hire narrowly experienced, self-taught auto mechanics and not mechanical engineers.

mml• 12 years ago

I had 3 interviews with google about 9 years ago. totally failed the on-site in mountainview. sorting a balanced tree on a whiteboard is tricky (quiet you!).

In hindsight, I'm not sure why I was excited to work at a post-ipo cube farm, even if it says "google" on the side.

edit: I believe it was also for SRE, which (at the time) means you do two interviews: the sysadmin interview, and the developer interview.

zerr• 12 years ago

Can anybody confirm that it is still the case? That this kind of questions are still given to non-fresh-graduate (i.e. experienced) candidates? (sorting a balanced tree... ridiculous).

aosmith2• 12 years ago

Hate to say it but this is why I said "thanks but no thanks" to google, and amazon. You can learn a lot more about a person and their skills from a few conversations than from a grilling against a white board.

voidptr• 12 years ago

Hi, I am the author of that blog post at http://blog.lambda-startup.com .

I just want to say that I appreciate the overwhelming feedback and all the positive and encouraging statements to not take that rejection too seriously.

Furthermore, I'm glad that my little article causes these mostly constructive discussions on here, and that sharing my experiences seems to be appreciated. Thanks.

cunac• 12 years ago

out of curiosity; do you get paid for one day you miss work or it is implied you should take a day off ?

logicallee• 12 years ago

The artcle mentions one interview out of 3+ phone interviews 5+ personal ones, that didn't go so well.

It leaves me with the following impression:

What a ridiculous waste of this guy's time. Why on Earth do recruitment processes need to be so capricious? Of course the guy's going to get nervous when he fouls up, given the HUGE amount of his time that has already been wasted, and how much you are putting on the line for every single moment to be perfect. maybe if the stakes weren't ridiculous and so capricious, it wouldn't take you two recruiters, three phone interviews, and a day of wasting everyone's time, only to reject a perfect employee.

Makes me glad not to work for a huge organization. What a travesty. I have no idea why people feel it's okay to waste other people's time like that.

gergles3• 12 years ago

Multi-stage interview processes like this are calibrated that you simply must pass every step of them to do the job. There's no fluff, there's no duplication; each module tests a specific component that the company feels is necessary. If you can't deliver on one component, then you aren't qualified. Harsh, but if you wanted to include an allowance for failing one module, you'd have to add even more modules and duplicate content in them so that you could still feel confident they could do the job. A false positive in the interview process is immensely painful for everyone involved and every effort should be made to prevent that.

yhager• 12 years ago

Exactly my thoughts. I guess they do it because they can, and the candidates are willing to go through this.

dclowd9901• 12 years ago

I recently interviewed for a position at twitter as a front end engineer (API stuff, HTML, CSS, etc). I failed out as well, but it was on, I think, much more nuanced terms.

The interviewer had me write a client jsonp API interface, that is a controller that would handle calls to and from a jsonp API. Embarrassingly, I knew little about jsonp except for that it has a callback parameter that correlates with a global function you have defined in the javascript of your page.

So as I was literally learning how JSONP worked during this interview, I also had to write an interface that would gracefully fail and queue multiple calls. I did pretty well, at that, having completed the essence of the exercise. Toward the end, I got mentally hung up on a scope issue and my brain was pretty much fried at that point so I ended up crashing out (it was the end of the interview time, anyway, so it's not like there was then 10 minutes of awkward silence).

Anyway, this sma hiccup, which in no way indicated I didn't know my craft, and opposing my seemed to indicate I was an extremely quick study, still managed to keep me from moving on in the process.

I'm not miffed or anything. In fact, out of that experience, I wrote a really cool lazy loading geolocation API that uses promises in conjunction with JSONP to deliver a really smooth API consumer experience, something I wouldn't have thought to do at my previous level of understanding.

dorfsmay• 12 years ago

FWIW, the linkedin interview process is eerily identical, same multi phone calls filtering, travel arrangements, meeting with 5 or 6 SREs for interviews, range of topics, trainee interviewers, friendly lunch, even the rejection process.

pkinsky• 12 years ago

How does google feel about non [Java, Python, C] languages? Could I use Scala to solve interview questions?

rjdagost• 12 years ago

I've been contacted several times by Google recruiters and the interactions have always left a bad taste in my mouth (and I've never even agreed to go forward with the interview process). They won't tell me what projects / products I'd be working on at Google if hired, which right off the bat is a non-starter for me. They want me to spend weeks studying puzzle coding problems for the interviews, which is also a non-starter. It seems like the entire process is centered around me proving to them that I am willing to go along with whatever they say. It's not a two way street at all.

As an experienced engineer who is happy in his current job, what is my motivation to jump ship? Google recruiters act like the chance of working at Google is by itself sufficient incentive. Google's entire process seems to geared towards selecting people who really want to work for Google and who will jump through whatever hoops Google sets up. Long-term, it is probably dangerous for Google to be so chock full of corporate confirmation bias.

pbnjay• 12 years ago

I've had this exact same experience 3 times in the past year. I echo your thoughts entirely. I've told them multiple times what kind of position at google would ACTUALLY interest me and yet I still get matched to nothing even close. Each time I've turned them down I get the same feeling of "Ok! So -- wait you said no? What? Are you sure? ... but we're google!?"

I'm also an experienced engineer, happy in my current job.

Glad to see someone else with the same thoughts!

YesThatTom2• 12 years ago

Former Google SRE here.

From the outside it looks like Google's strategy is to hire people that have the skills that people can only get from already working at Google. i.e. They have forgotten that they themselves did not have those skills when they were hired.

Sadly blog posts like this make me think they are suffering an unintended side effect from doing this: http://googleresearch.blogspot.com/2006/03/hiring-lake-wobeg...

lostcolony• 12 years ago

I keep coming back to this, wondering whether it's effective or not. It feels like an SAT, or really any other form of academic standardized testing, except not so objective.

"Here's a list of things to know, go study it, then come in and solve problems with it in an artificial environment so that we may grade you". So you cram in preparation, and then if you don't pass the test, you plan to retake it in a year.

I'd think the best and brightest would be the ones whose working knowledge, without cramming, allows for innovative, interesting, clever solutions. Even better if you can get away from the artificial feeling of interviews, into a "here's an actual, and unsolved, problem, let's figure out how to solve it together so I can see how you tick", rather than "here's a fake question, one with a well known, posted on the internet, solution, that if you ever faced in real life you'd solve in 5 minutes of Googling and move on, and which I expect you to recognize as a (X) problem, and regurgitate the solution from the selected readings". (That said, one or two stages of the interview seemed like they ~might~ be that).

Maybe the intent isn't really to hire the best and brightest (that's hard to test for), but really the people who want to work at Google the most; are you willing to devote hours to the mere possibility?

Recruitment Process for a Google Site Reliability Engineer

Related Stories