jia_min

That's true of any hiring system though: how do you gather data on the job performance of the people who don't pass a resume screen? A coffee date? A phone screen? Missing counterfactuals everywhere.

To run a full analysis, you need to hire people randomly - both people who fail and pass your hiring process - and assess their subsequent performance. This never happens in real life.

However, you can still run an informative statistical analysis based on the variability in interview scores and performance scores. For example, the people who scored 5/5 on the interview should perform better than the ones who scored 4/5.

The company line is that they use structured interviews. What actually happens in practice is often a different story.

Here's the NYT article where Laszlo mentions the "random mess" they found when they used the interviewer as the predictor and then subsequently mentions how they use structured interviews now instead: http://www.nytimes.com/2013/06/20/business/in-head-hunting-b...

The interview correlations I cited are from Schmidt and Hunter, 1998: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

My point is, it doesn't matter who does the interviewing - looking for people who are "good interviewers" is a unicorn chase because the vast majority of people don't have a magical ability to guess who's going to be a good performer pre-hire. Unfortunately, Amazon is making the same mistake with their "bar raisers": http://firstround.com/article/Mechanize-Your-Hiring-Process-...

Geez, you engineers don't seem to like to learn from each other.

You might be interested in this utility app that calculates the probability of a successful hire created by an I/O Psychologist: https://morgan.shinyapps.io/utility_calculator/

It takes the base rate of successful performance into account.

You, I like. That's exactly what the 85 years of research on hiring tells us are the strongest predictors of job performance and employee turnover.

Source: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

From the NYT's article quoting Laszlo Bock: "Years ago, we did a study to determine whether anyone at Google is particularly good at hiring."

Google made the classic mistake of using the interviewer as the predictor of interest. Meanwhile, decades of research has shown it's not who does the asking but how and what you ask. For example, structured interviews are about 50% more predictive of job performance (r = .51) compared to unstructured interviews (i.e., the way almost everyone interviews; r = .38).

So why did Google run this experiment when they had an army of Industrial/Organizational Psychologists who already knew the research? They wanted to collect their own data and test the theories on their own employees, which I encourage and applaud.