Ask HN: Who are the engineers that don't get hired?

https://news.ycombinator.com/item?id=7939267

by jparker165 • 12 years ago

35 31 12 years ago

news.ycombinator.com

All I hear about is how no one can find (good) engineers. I had dinner with a friend last night complaining that this is killing his company's growth...but they only make offers to 1/100 resumes that come-through. I've heard this before and don't think it's uncommon.

What's really wrong with these developers: undertrained, low mental horsepower, bad "fit", imperfect industry experience? All of these are true, but 99/100!?!

With such a developer shortage, can these companies really not get productive work of a slightly broader group?

Something isn't fitting together here.

aguynamedben• 12 years ago

For me, it's difficult to hire an engineer that's not specifically interested in what my company is doing. A very common reason I've passed on engineers in the past is "they're okay... but they don't seem really excited about what we're doing here..."

S4M• 12 years ago

What is your company doing, then?

nostrademons• 12 years ago

http://www.joelonsoftware.com/items/2005/01/27.html

From what I've seen, the other 99% of resumes are people trying to break into the field but with no actual programming experience. Competent developers with a track record get snapped up really quickly, while newbies without a track record go on to apply to 100 more jobs.

JSeymourATL2• 12 years ago

can these companies really not get productive work of a slightly broader group?

Relative to the 1/100 resumes-- if your friend took the time to dig deeper, he'd likely spot 2-3 diamonds in the rough, a moneyball misfit.

If your friend is like most senior managers, he'll scoff at the idea-- 'I'm too busy to read 100 resumes' he'll likely say, or 'that's HRs job!'

Finding & attracting talent is a mission critical competency that senior leadership must drive. Too often it's delegated to mid-tier management that don't share the same sense of urgency.

It takes an exceptionally rare manager who can assess and train-up mediocre talent into A-Player status. Fewer still, who have the guts to make a trial hire. A bad hire can be costly on a variety of levels.

But most guys would rather play it safe-- blame HR, lack of bandwidth, or the market. Meanwhile, the guys who figure it out manage to scale.

toomuchtodo6• 12 years ago

It could be a resources problem. The company I'm about to leave wanted me to find 24x7 on-call DevOps folks for $70K/piece (Chicago). I told them $100K/year was the floor for quality people and they balked.

Finding talent isn't the only challenge. Incentivizing them to join and stay is equally challenging.

JSeymourATL2• 12 years ago

Limited budget demands that you, get creative and dig-deeper still. Could a local QA guy get the job done in a pinch? Will your Ops support someone working remotely, even off-shore? Economical solutions require some flexibility.

illini1234• 12 years ago

Given the Chicago reference (fellow Chicagoan here), do you view this as a Midwest problem? I feel like a lot of the companies here tend to lowball on salaries / benefits. I did a study on this for the VC / startup communities in the Midwest, and we think we have enough data to back it up.

x0x02• 12 years ago

This is interesting.

I screen a lot of ML researchers and engineers.

The process typically starts with a 30-90 minute phone screen (it ends faster if I don't like you). I start by discussing the company and the position, in as much depth as the candidate wants. Then we discuss one project of the candidate's in depth, both for some technical expertise but also to suss out what the candidate, as opposed to a team, actually did. Good candidates should be able to discuss what the project did, why, implementation, challenges, problems overcome, tools used, etc. Then we do a quick survey of general ML technologies; we're hiring senior people so they should have a pretty good breadth of experiences. So they should know their way around regression, trees, forests, boosting, svm, linear algebra techniques, plus optimization tricks for all of the above. It's ok if a candidate hasn't used any one specific thing, but they should have a rough knowledge of which tool to use when, plusses and minuses of tools, and the typical ways one improves results (hold-out testing, k-folds, l1 and l2 penalties, etc etc.) If you've actually used most of these tools you should easily pass the phone screen.

Then we bring people in or fly them out. Here, we go more into depth in several tools. You should know the math and the practical use of several ml techniques really well. We'll also make sure you can write enough code that you don't have to be handheld. That is, you need to be able to do some data processing and string together data pipelines to get your work done.

common resume failings include (NB: this is for an ML engineer / data scientist):

* (very common) no academic prep and no demonstrable experience. ie a potential junior person applying for a senior position. If he or she has no degree but spent 4 years at netflix or somewhere good, we'll definitely at minimum talk on the phone. Demonstrable experience definitely means education barely matters.

* you live abroad, thus requiring sponsorship and a move, and aren't a really strong candidate

* (surprisingly common) you spammed all our different jobs, making us believe you just want a job, any job, and paid no attention to what we're looking for

Common failings from a phone screen or interview include:

* (weird, but happens) don't make the agreed upon phone screen time without an email before (or even an email after, if an emergency occurred)

* (really weird, but happens) you have no idea what the company does. I'm looking for you to have spent at least 5 minutes reading our site and have a 1 sentence summary of what the company does. If you haven't done even that much research, why on earth is the candidate applying to us? A perfectly fine example would be, for example, if I worked at google: You build a search engine and you want ML people to improve search results. At, say, square, a great response would be "you're handling monetary transactions and therefore have fraud problems from card holders and probably want to build risk management tools to protect yourself from bad merchants." I find it really weird many people don't even know that much. This isn't an instant fail, but it's a negative sign.

* he or she wasted his or her phd. They may (or may not) have a very deep understanding of one tiny piece, but lack a decent understanding of the breadth of the field. We need breadth. And yes, they may be able to learn, but the question is then what the hell did they spend a 6 year phd learning? Also, we're hiring senior people. And again, anyone with a good ms in ml should do fine (not just a theory; we've hired them from stanford and ga tech).

* They don't know deeply at least one or two techniques and tools. Whether its hadoop, vowpal wabbit, R, sklearn, weka, whatever, you should have become an expert in at least one tool and know it inside and out.

* (very common) an inability to program enough. Obviously most ml people aren't engineers, but they have to be able to do data processing, data cleaning, run ml tools, build their own ml tools if necessary, etc. Senior people in yahoo or google's research labs may not need to program, but for us you do.

* (somewhat common) bad communication skills. We speak english; you need to be able to speak reasonably clear english so that we can discuss technical topics clearly. How people can finish a long stretch in an american university without this escapes me. And to be clear, about 1/2 the employees here are foreign born, so we're pretty good with different accents and the typical indian and chinese esl idiosyncrasies.

The last thing I can say about job seekers is we're talking because I have a problem. That is, I really want you to succeed if there's a good chance this will work.

eshvk• 12 years ago

This is very detailed. Pretty similar to what I have seen in industry. A few comments from a guy who has been involved in several ML teams for the past few years.

1. Hire depending on your team. There are candidates who are amazing data scientists but do retarded shit like select * from table order by rand() on a ten billion row table and bring down the cluster. Sometimes, it is OK because your team has incredibly strong engineers who can work in close combinations with these people. Sometimes you can't really afford that. At my current workplace, my manager did a really good job of hiring people who are fullstack (data + backend + viz + ml) while having strong preferences in one direction or the other. So you might have an incredibly strong ML engineer who can roll his own ML code out on Spark, while also being open to writing back-end infrastructure code once in a while. Or you might have this guy who thinks in data pipelines all day long, but will spend a bit of time thinking about mathematical model. The teams work well.

2. Hiring for ML teams is really really hard. When I was in San Francisco at my previous job, we used to screen quite a few candidates everyweek. We would see failures similar to what you posted. Either engineers who knew fuck all about ML or people who were good at the theory but didn't know how to code. Somehow, at my current job in NYC, we lucked out because we have a bunch of really awesome people who moved out of finance but want to continue to live in NYC.

They may (or may not) have a very deep understanding of one tiny piece, but lack a decent understanding of the breadth of the field.

This is close to what I have seen from both sides of the table. But, one thing that annoys me is the number of teams where people have pet algorithms that they are obsessed about. They keep asking questions about those pet algorithms and ding a candidate for not knowing that specific algorithm in depth. One should be able to be flexible while doing this.

They don't know deeply at least one or two techniques and tools.

I think we see people from so many different walks of life that for junior people the goal is to make sure that they know the shit that they have in their resume and drill them if they don't. As in, we don't hire for specific techniques or tools because the team has its own collection of tools/libraries (Spark/Hadoop/Python/Word2Vec/CF etc) which you will be able to pick up fast if you are good enough.

aidenn02• 12 years ago

> They don't know deeply at least one or two techniques and tools.
I think we see people from so many different walks of life that for junior people the goal is to make sure that they know the shit that they have in their resume and drill them if they don't. As in, we don't hire for specific techniques or tools because the team has its own collection of tools/libraries (Spark/Hadoop/Python/Word2Vec/CF etc) which you will be able to pick up fast if you are good enough.

I think you misread what eshvk was saying; he isn't saying that they don't know one or two specific techniques or tools, but rather they don't have any tools/techniques at all that they know deeply. If you read what followed the part you quoted, it should become clear.

xiaoma• 12 years ago

It sounds like the main problem is your focus on hiring senior people. Maybe you should go with Reid Hoffman's advice and try a junior person making a mid-career change. There will be more risk, but you stand to gain top-notch workers who the rest of the market ignore due to conservatism and credentialism.

x0x02• 12 years ago

Oh, we do hire junior people as well; it's just that you need at least a core of senior folks.

blooberr• 12 years ago

Sometimes its not the developer.

You can also blame it on an unrefined interview process. I've probably turned down the right candidates accidentally while trying to work out an interviewing system. With a startup, its hard to get enough feedback because you can't hire enough people to verify if your assumptions were correct.

Another issue to consider is unrealistic expectations for a candidate (has to know X, Y, Z frameworks + machine learning + mobile) If this candidate exists, you probably can't afford them.

For the rest of us, I strongly believe you can train most developers to do the job. It's how much time you're willing to invest in them. I've taken inexperienced developers that are very interested in the company vision and given them a chance to grow in the roles. After a year, they're the "good" developers I've always wanted.

gregcohn• 12 years ago

It's very common that companies have adverse candidate flow -- in other words, the ratio of 99:1 is not necessarily representative of the market; it's simply representative of the median candidate a particular company is attracting into its hiring pipeline.

Invest more in getting the right kinds of candidates.

fluxon• 12 years ago

jparker165, ask your friend if "Inside the Mirrortocracy" https://news.ycombinator.com/item?id=7930430 applies to his company.

alok-g• 12 years ago

Being on both sides, I feel interview processes are a part of the problem too. Every place I have interviewed assumes that the interviewers are good and by default, the candidate is bad. While it is probably statistically true, it makes things worse if a good candidate gets rejected because of an issue with the interviewer's own abilities.

deciplex• 12 years ago

Or, that competent engineers are naturally good at interviewing candidates in the first place.

janbernhart• 12 years ago

I've recruited developers for dozens of companies so hope I can give valuable insights. First of all, there's nothing 'wrong' with the 99 developers that didn't get hired. And in my experience, they will find a job somewhere else. A job that suits them (even) better, hopefully

Thing is, companies and jobs differ, and so do people. I've seen top performers from company A fail in the interviews at company B, and vice versa. Leaving both companies under the impression that the other company employs 'bad' developers.

Whether or not your skills+potential can be used optimally by a company depends on so much details, like cultural fit, to which extend you agree on their paradigms (methodology, problem solving ideas, prioritization, etc).

Long story short; companies are usually looking for the most optimal choice for their job openings, and so are applicants. This results a way more complex matching strategy that just 'could do the job'.

Perhaps John Nash can help us with optimizing these strategies. Until than, I think 1:50 to 1:99 ratio's are here to stay.

aidenn02• 12 years ago

Some points:

* As far as graduating seniors, when you post a job offer, you get maybe 10% of the resumes from people with either a CS/CE major or minor. That's not to say they can't code (in my experience a resume from a non-CS major with good experience is a much better bet than a resume from a CS major with little or poor experience). But we literally get people whose experience totals up to "took a 'How to write HTML' class once freshman year as an elective" applying to jobs involving low-level C

* Actual on-site interviews are very expensive; unless you are really hurting for people, you need a cheaper (and potentially less precise) way of weeding things out.

* With the exception of graduating seniors, there really are relatively few good candidates actively seeking work; most of the good ones have jobs (and you can hire them, but that usually isn't counted in the X% of resumes get offers, which usually refers to responses to a job posting; hiring people away from their current jobs usually starts in a different manner).

* On the other hand huge numbers of people who are desperate for work send in resumes. The less harmful of these are the ones who obviously are unqualified and clearly just spammed every job offer listed regardless of the requirements. Since we aren't talking about graduating seniors (see above) the majority of the not-obviously-unqualified often even have relevant work experience, but that's just because it takes time for people to fire you.

I am talking about people with 2-5 years of experience working in language X who are barely able to (or sometimes even completely unable to) write FizzBuzz in language X.

Another example was someone with over 5 years of OS kernel development, who in the interview was unable to describe what exactly they did in those 5 years and they didn't seem to know a much about any of the various OS topics we tried asking them about: scheduling, interrupts, DMA, filesystems, memory paging, IPC.

These people are particularly harmful, since there are very few tells at all on their resumes (and no reliable ones), you would have to interview all of them, which gets expensive. The majority can't get past a phone-interview, but when you are talking about

This leads to a lot of companies not even bothering to interview people (again ignoring those graduating from school) without referrals from current employees, or some other way of weeding out the massive numbers of "good resume, but bad candidate" submission that also will, as a side effect, weed out some of the "good resume, good candidate". It also means that if you are looking for work, and can't find it, then either you are unqualified, or need to get referrals from friends who have jobs at companies that are hiring.

Ask HN: Who are the engineers that don't get hired?

Related Stories