Job hunting is a matter of Big Data, not how you perform at an interview

http://www.theguardian.com/technology/2014/may/10/job-hunting-big-data-interview-algorithms-employees

by Roonerelli • 12 years ago

122 80 12 years ago

waps2• 12 years ago

Another way for big companies to more accurately exploit their workforce under market-based conditions. If this software goes through, the only possible way to get a raise is if the program believes not getting a raise would actually cause you to quit. There's only one reliable way to make the program think that : make sure you actually would quit if you don't get it.

In other words: expect a massive increase in job hopping as people find the only possible way to negotiate with this program : quit.

By morning, he says: "If a customer has thousands of people in similar job types, our system can predict accurately on a given day which individuals are most likely to quit." In response, Evolv then offers employers "what-if types of analysis" by which if they change certain incentives – a bonus, training scheme, change in environment – they can see exactly what effect it is likely to have on a particular person's behaviour. In this way Evolv advertises average reduced employee attrition rates among its clients, who include one fifth of Fortune 100 companies, of up to 15%.

This sounds horrible. It would force employees into quit-to-improve-working conditions dynamics. Constantly interview, at a non-ridiculous rate. If you get offered better conditions, either Evolv will offer you the same at your current position, or you should quit.

Of course that's already mostly true : my advice working for a fortune 100 company that isn't Google or Facebook : prepare to quit after 1 year or less. Regardless of whether you want to stay or not, have a serious discussion with your boss about quitting after 6 months at most.

I wonder if it would defeat the negotiation tactic used by "Evolv" here. If you can call it a tactic, that is.

anjc2• 12 years ago

Regardless of whether you want to stay or not, have a serious discussion with your boss about quitting after 6 months at most.

To get a raise you mean?

waps2• 12 years ago

In my experience, mostly to guarantee you get a raise at the end of the year. At the six month mark I'd settle for a concrete promise doubling inflation (but not easily). Make sure to both ask what you're doing right and wrong and make sure to discuss what the company is doing right and wrong.

icegreentea2• 12 years ago

Once you have a sufficient percentage of a job market gated by these types of algorithms, the individual worker literally cannot win. Adversarial approaches to 'beating' the algorithm can be detected and compensated for within the algorithm. Ironically, it'll just be right back to forming unions, except you'll be negotiating against algorithms.

Seriously, almost any type of 'optimization' on the half of companies/employers right now is bad news for employees and job seekers. We call it optimization because its nice and clean. But the effect of every optimization is to squeeze out every last bit of productivity out of a given work force as possible.

Honestly, I hope this type of thing is never developed and deployed to the degree to which some of the proponents in the article wish it to. They can paint starry eyed pictures of a future where everyone gets to work the job that fits them the best, but all I can see is a future where everyone who has a job is scared shitless of losing it, since it'll blackmark them forever.

dkarapetyan4• 12 years ago

This type of optimization has been happening since the beginning of the assembly line or even before it. Plus, employee/employer relationships are always adversarial. I want as much as I can get out of the company and the company wants to get as much out of me as possible.

As for it going into production I don't see why it is a bad thing. You could turn the whole thing around and help job seekers find companies where they know they would be happy and productive. After all it is a similar kind of optimization but from a different perspective.

jib2• 12 years ago

Thats a highly cynical view. There's plenty of optimization problems that aren't about squeezing out productivity at the cost of anyone, but rather about improving productivity and employee satisfaction at the same time.

For instance - lets say that I realise I have a problem with turnover that I want to fix, because my costs of replacing staff are too high. To address that I want to spend 1M on retention activities in a year. Should I spend that on additional vacation days, or should I spend that on extra events for the staff, or on extra training opportunities? If I spend that money, will the impact on my turnover costs be positive enough to warrant the spend?

That's the kind of discussions that always pop up. Being able to quantify the impact would make it easier to do the right thing that both benefits employees and the employer.

rmc3• 12 years ago

Ironically, it'll just be right back to forming unions, except you'll be negotiating against algorithms.

An algorithm can't 'negotiate' with a strike, or other more extreme forms of trade union actions (like burning down the compeditors, shunning scabs, etc.)

hyperpape2• 12 years ago

Think of the algorithm as playing a game against a human, where the human players' goal is to maximize compensation/opportunities/generally feeling good or at least ok about work. The algorithm's goal is to maximize the business's surplus (productivity minus cost). This game is competitive (but not zero sum).

You are claiming that the human cannot beat the algorithm, because the algorithm can always adapt. But humans are still better than computers at some games (Arimaa, Havannah, Hex, Go) and worse at others (Chess, Checkers). So your claim needs a lot more justification.

The comparison is worrisome: depending on how good these algorithms get and how zero-sum the competition is, we could be screwed. But we don't know.

6d0debc0714• 12 years ago

I wonder about that. There'll be a motivation for companies to defect.

If I can get awesome people to work at my company just by offering them a reasonable amount more than at Evilcorp, I'm just gonna do that. If it forces Evilcorp to raise their wages, then that's good for me - EC is spending more to get what they'd get anyway - and if they come work for me then that's good too.

Wage fixing, is a problem, don't get me wrong. However, it's a problem regardless of how efficient you are in hiring.

dkarapetyan4• 12 years ago

I think threatening to quit or quitting with a better offer has always been a guaranteed way to get a raise. These new startups are not going to change that dynamic.

Funny you should mention the 6 month rule. I've kinda stumbled on it through trial and error but it seems like a pretty good checkpoint for salary discussions.

zo1• 12 years ago

Care to elaborate a little more about this 6 month rule, please?

jib2• 12 years ago

Or you can see it as finding out how to take the issue of money and other hygiene factors off the table.

Money isn't a great motivator. Lack of decently accurate compensation is a very good de-motivator. Same goes for a bunch of other things - I dont want to have to think about them, but if they aren't thought about it will annoy and demotivate me. Find a way to script that and it will increase my happiness by decreasing irritants.

anjc2• 12 years ago

It's been known for a long time that standard interviews are a poor indicator of future job performance, so probably any analyses of data regarding behavioural modes, a person's intrinsic motivators, cultural fit etc, will do a slightly better job. Presumably all of this will still be subordinate to an on-the-job trial anyway, though.

It'd worry me that, although this guy says you need to increase behavioural diversity but minimise value diversity, that you're effectively just minimising the pool of potential employees, rather than figuring out ways that a larger pool of people could fit. Or in other words, whether it's Myers-Briggs, Belbin's roles, IQ tests etc, it seems that evaluation tools are trying to quantify the diversity of people, and pick off, with increasing accuracy, the exact archetype that aligns with organisational goals. But if you assume that people are diverse, and that employment roles are diverse, and that there's a large pool of both, would it not be a better idea to focus on quantifying the differences in attributes required for your employment roles, so as to maximise your pool of potential applicants?

I'm willing to bet that someone who's slobbish and lazy and unpleasant could play a valuable role if you could quantify the requirements and goals of positions in your company, for example. And does value alignment matter for all roles in all companies, or is it just a phenomenon arising in the last decade, being an intuitive way to maximise employee investment and increase profit? Why would financial staff need to want to 'change the world' or some bullshit to work at crappy Startup X?

e12e• 12 years ago

I'm willing to bet that someone who's slobbish and lazy and unpleasant could play a valuable role if you could quantify the requirements and goals of positions in your company, for example.

Yeah, but the positions of CEO and HR are usually filled at an early stage...

rmc3• 12 years ago

if you could quantify the requirements and goals of positions in your company

That hits on one big problem I see with all this: How do you know what the company actually wants? All/Loads of companies claim to want the same thing, and have the same vision of "changing the world". But some companies are run by abusive wankers who want cheap developers who'll be happy working 24/7 for low money. You can tell by the company's actions, not their words. If the only pay peanuts, then they only want cheap labour.

Will this big data be turned around to the company, so employees can see what sort of place the company is?

mkr-hn• 12 years ago

Will this big data be turned around to the company, so employees can see what sort of place the company is?

This sounds like what Glassdoor does. Even small local companies get reviews on it.

kremlin• 12 years ago

I was also hoping to see the analysis apply both ways to some degree. Rather than simply having a paying company ask you to find the best fit, is it not better to have something akin to a dating service, where you're not just finding the best fit for one party; you're trying to find the best fit for all parties. An employee who's not a great fit for Company A may be a perfect fit for Company B -- this, ideally, is the sort of value I see from these ideas, rather than just ending with 'He's not a great fit for company A'.

Malarkey732• 12 years ago

I'm not a regular interviewer but I've been on a panel for a few recently. For one job in particular the standout candidate on paper gave the worst interview I have ever seen.

He was arrogant, confrontational, couldn't answer simple questions and was half an hour late - and not apologetic at all. He was just weirdly and inexplicably unpleasant to everyone.

Perhaps he would have been great in some role or would have aced a psychological test .. but I'm glad we did an interview so we could reject him out of hand.

I'm sure there is a place for widespread empirical evaluations .. but ultimately if you are employing someone to work with you there has to be space to say: "Err, No! - I don't care what the computer says".

silverbax88• 12 years ago

Or he took one look at the place, decided he (or she) did not want the job and tanked the interview.

baudehlo• 12 years ago

Disclaimer: I work at a company that provides this type of service.

We are huge believers that this sort of testing is purely a pre-interview step. You get everyone to take it and you save hours and hours of digging through resumes. That's the big money saver.

6d0debc0714• 12 years ago

Why do you think an empirical evaluation wouldn't have picked up on that?

It strikes me you could easily record the interview and run it past a machine looking for the tempo of the conversation, the tone of voice and so on - and see whether that correlated with employment a year later.

Retric• 12 years ago

The best people are rarely going to sit though an on-the-job trial or the classic tech 6+ hours of interviews over 3+ days. More importantly the single most important part of the hiring process takes place after the fact. If you don't take care of your people and deal with people that are a poor fit minor changes in how you higher quickly becomes meaningless. On the other hand if you can manage people appropriately then you can almost randomly higher people and get by simply because your filtering based on a long term on-the-job trial after the fact.

munrocape• 12 years ago

Now any analysis applied to a data set is now "Big Data." While easy to just resolve it to being how the media treats most science stories, I had hard time understanding the demarcation. I now go by DevOps Borat's definition - "any thing which crash Excel" [1]

[1] https://twitter.com/DEVOPS_BORAT/status/288698056470315008

gaius• 12 years ago

It's worse than that. People will run Hadoop on 20 nodes to do stuff Excel could handle easily, so they can join the "big data" fun. It's just CV-padding really, ironic given the discussion.

001sky• 12 years ago

https://twitter.com/Lumonade/status/289171814444331008

"@DEVOPS_BORAT got heavy duty @dev ops gig available in San Mateo,ca and Seattle area! $130-$140+k doe email me Ryan.Lum@greythorn.com"

does this mean it works?

rmc3• 12 years ago

Unfortunately, employers will treat that as big data experience, and hire that way. Employees are merely reacting to what the market tells them.

collyw• 12 years ago

I has someone contact me about a "big data" start-up recently, and did I know anyone that would be interested.

I replied saying "I may be" and what size was their "big data", as usually I don't see the point in taking it out of a traditional relational database in most cases. I don't think they like my attitude, as I never got a reply.

tom_jones• 12 years ago

I'm suspicious of this technology. While I concede that it probably has some statistical and even empirical merit, those in HR will always do everything they can to make sure that "troublemakers and journeymen" get kept out. But unfortunately, most people who have had the same job for a long time are either just trying to pay their mortgage and smiling through gritted teeth long enough to see their kids alright or have no imagination/ambition and those who switch jobs regularly are often simply tired of taking shit from people. I think you'll find that "troublemakers" usually have tried to alert their line managers to the real flaws in the company's systems and been told to shut up. Look at the businesses this data is being taken from, Law Companies, Corporate Banks etc. It is obviously yet another toy to allow rich kids to recognise their own. What has this got to do with those who do the vast majority of the actual work?

ddebernardy• 12 years ago

Best quote from the entire article imho:

The data suggested that the success of teams had much less to do with experience, education, gender balance, or even personality types; it was closely correlated with a single factor: "Does everybody talk to each other?"
Ideally this talk was in animated short bursts indicating listening, involvement and trust – long speeches generally correlated with unsuccessful outcomes. For creative groups such as drug discovery teams or for traders at financial institutions, say, the other overwhelming factor determining success was: do they also talk to a lot of people outside their group? "What we call 'engagement' and 'exploration' appeared to be about 40% of the explanation of the difference between a low-performing group and a high-performing group across all the studies," Pentland says.
It was important that a good deal of engagement happened outside formal meetings. From this data, Pentland extrapolates a series of observations on everything from patterns of home-working (not generally a good idea) to office design (open and collegiate) to leadership. "If you create a highly energetic environment where people want to talk to each other right across the organisation then you have pretty much done your job right there."

So true.

asdfologist• 12 years ago

The article didn't make clear though, whether having more engaging conversations actually causes team success. If a team is successful, that may boosts morale and lead to increased levels of conversation.

Fr0styMatt• 12 years ago

As someone in the job market right now, I'm refreshed by the article's admission of how much chance plays a role in whether you land a job.

I'm currently working on my CV and there's about as much conflicting advice as to what the 'ideal CV' is as there is about what the ideal diet is. It's somewhat frustrating but at the same time it just shows the amount of chance and variation involved in the whole process.

At least in the industry I'm going for (games / Unity3D for what it's worth), actions & side-projects seem to speak louder than words.

morgante• 12 years ago

I stopped at "collecting the data of all our private moves on the internet and applying their logarithms accordingly." Apparently programming is all about writing logarithms.

walshemj• 12 years ago

And your not worried about collecting private data about job applicants?

seabee• 12 years ago

Seeing shady behaviour isn't a reason to stop reading. Seeing bullshit is.

michaelochurch• 12 years ago

Evolv data undermines certain truisms, among them the idea of the serial job-hopper. "The number of jobs you have previously had," Simkoff says, "and even whether you're employed or not at the time of application, has zero statistical correlation with how successful you will be or how long you will stay."

The job-hopper stigma isn't about imputed low skill or merit. It's about social status. The person who is presently unemployed has (temporarily) low social status. The person with 5 jobs in 6 years, it is perceived, failed to achieve high social status at any of them.

The problem with humans is that most don't make decisions based on value-add potential, but on social status. They see Harvard on a resume and want to hire that person, to be socially "closer" to Harvard. It's not about whether Harvard graduates are better hires or not; that question is irrelevant.

Job-hopping might seem like it could be a high-status behavior, in that the best people get bored quickly and always have other opportunities, so they don't put up with abuse. After all, the serially fired job hoppers are maybe 1/10 of that set. It's not so, because the people who make hiring and promotion decisions are in corporate in-crowds, and part of being an in-crowd is the necessary assumption that everyone wants to be in an in-crowd. The job hopper may be individually excellent, and it may be that he'd be a 5+ year employee if given high-quality work and colleagues, but all his paper says is that he never stayed long enough to join a corporate in-crowd, and that even if he was invited into one, he made the "wrong" decision to leave it.

dkarapetyan4• 12 years ago

I have heard people say that serial job hoppers have commitment issues which is a little weird because why would anyone commit to a shitty job. I tackle the issue directly by addressing it honestly when people ask why I'm looking for a new job only after 6 months. I guess so far I've given good enough answers.

ytNumbers• 12 years ago

Today's Dilbert is forecasting where this new approach to staff efficiency is heading.

http://www.dilbert.com/dyn/str_strip/000000000/00000000/0000...

coldcode• 12 years ago

While I like the idea, I don't care for the side effects. If everyone uses a big data based algorithm, and you are an outlier, you will never get a job.

cruise02• 12 years ago

That would only be the case if everyone used the same algorithm with the same inputs.

ThePhysicist• 12 years ago

Honestly, I think a good tech journalist should be able to critically review what his interview partner claims and not take it for face value, which the author of this article seems to do most of the time. Today, all start-ups that do any kind of data processing advertise themselves as "big data" companies that use "advanced machine learning", but from my own experience most of them rely on pretty trivial algorithms behind the scenes.

Also, some of the numbers in the article really make you scratch your head: Achieving more than 95 % accuracy when ranking a large number of student teams in an eight-months long business plan competition, based solely on the results of a simple online questionnaire taken at the beginning of the competition? This just seems too good to be true considering the data sources they have at hand, even assuming that they use the most advanced machine learning in the world.

Of course, if you test your algorithm many times at different competitions you will achieve a perfect or near-perfect prediction accuracy for some of them (by pure chance), which however doesn't mean that you can achieve this kind of accuracy consistently (which is where the business value lies).

jqm5• 12 years ago

"Achieving more than 95 % accuracy when ranking a large number of student teams in an eight-months long business plan competition, based solely on the results of a simple online questionnaire taken at the beginning of the competition? This just seems too good to be true."

My first thought exactly.

blahedo• 12 years ago

Technically, the two sentences in that quote may not be connected. The first sentence is about the longest eval they did (8 months); the second sentence is about their performance on competitions (95%), but I read it that the 95% was over all the competitions they did, and that they could have been (and presumably were) worse on the 8-month one.

jqm5• 12 years ago

Probably this science is in it's infancy, and as data grows and algorithms are refined, it will go places we can scarcely imagine at the moment.

I can hardly wait for the day when a baby is born and "garbageman" or "engineer" is stamped on it's head and it becomes futile to argue with empirical truth about "best fit".

Actually, kidding aside I think companies and employees may both benefit from this research if it is applied properly. But if a better form of these tools were available to our current system it would likely produce some very bad effects. I think maybe we have some important decisions about humans and their role in society coming up soon. Because technology never seems to go back in the bottle.

klunger2• 12 years ago

A few months ago, I met the founder of a company in the recruiting business. They aggregate online profiles of people, both applicants and other people happily employed at their current jobs. Then, based on the combination of their LinkedIn, StackOverflow, Facebook, FourSquare etc, their algorithm ranks folks according to the desired characteristics for a given position.

It sounds interesting in theory. That is, until I got to asking about how they quantified softer qualities that employers look for, like an applicant's social skills or potential for a client facing role. Apparently, to determine this, they look at the number of "check ins" people do at locations that are not their home city while employed. Their algorithm assumes that the person is traveling for business and is therefore trusted to meet customers.

There are so many assumptions in this one example that it makes me question the integrity of the whole system. An algorithm is only as good as the person designing it. Maybe Evolv really is better than these guys at finding quantitative markers for softer skills, but I remain skeptical.

Balgair2• 12 years ago

And what if you have 2/4 of those? Or have their privacy to max status? How do they account for those? Like you said, the algorithm is only as good as the writer.

klunger2• 12 years ago

Actually, I asked about that, since my privacy settings is at the max on Facebook and maybe I used FourSquare once? I also never anchor Facebook posts to a location.

His response was basically that they are "trusted partner" (quotes because I can't remember the exact term, but that sounds right) of Facebook and so they get all the data somehow. Maybe they pay for it? I am not sure how it would work because it couldn't be anonymized for their service.

Anyway, he basically said that, when you combine the Facebook location based posts and FourSquare checkins (and probably location anchored Tweets too as well as others I am missing), there are so many millions of these happening everyday that some of those people are bound to be qualified for the position a company is hiring for.

...Which brings us back to problematic assumptions. Garbage in, garbage out.

chroem• 12 years ago

This is exactly the sort of thing that I have been absolutely terrified about for a while now. When you're being hired, companies often tell you that they're going to run a professional background check, and I am concerned this may mean they buy data from Google, Facebook, cell carriers, etc.

Unsavory political views? Get blacklisted.

Don't get along with a relative? Get blacklisted.

Indulge a porn habit more than the HR manager likes? Get blacklisted.

I cannot stress enough how dangerous this is.

jakejake• 12 years ago

You could also wind up at a job where your personality and views are appreciated and you don't have to hide your true feelings from anybody. I realize the point is about invasion of privacy, but there are places that are looking for non-cookie-cutter people too. Like my own company - we don't really hire anybody unless they seem to be a weirdo of some sort.

stephengillie• 12 years ago

Why are you so worried about the opinions of others? Do you have more passion than other people who do what you do? Then that will be good enough.

wwwwwwwwww• 12 years ago

not everyone wants to end up like brendan eich

jqm5• 12 years ago

Play on HN more than you should? Get black.... hey, wait a minute!

codystebbins• 12 years ago

I think this is cynical and forgets the reality of competition. In this 'big data hiring' future companies still need to compete for talent. Companies which hire on the basis of arbitrary factors like political views, porn habits, family infighting risk losing talent to competitors. There are incentives to find the right factors of a good employee, just like there are today. In fact a lot of these concerns can happen with the human interview process today.

The algorithm to hire people in this future is a competitive asset of a company and they would spend lots of time & money to narrow down the actual important factors. I highly doubt companies would just all settle on the same algorithm given the incentives to compete.

Also if you are concerned about this info impacting employment, just don't post it online. Not much different than today.

quanticle• 12 years ago

In this 'big data hiring' future companies still need to compete for talent. Companies which hire on the basis of arbitrary factors like political views, porn habits, family infighting risk losing talent to competitors.

Yes, but you forget the old banking maxim: it is better to fail conventionally than it is to succeed unconventionally. I think it's very likely that hiring managers would pass up the potential increased talent of an employee if that talent came coupled with increased risk that the employee would do something to embarrass the company or do something to make the company look bad.

Also if you are concerned about this info impacting employment, just don't post it online.

So what do I do when the lack of online information about me in itself becomes a drawback?

kumbasha• 12 years ago

Oh, joy-a fully automated caste system.

lifeisstillgood4• 12 years ago

I love the reverse idea here - take an assessment test and then release that data out to the public - presumably as anonymised as possible.

Then folks can take the same test and see if they should run screaming from the interview

lifeofanalysis3• 12 years ago

There is obvious up trending of terms like 'big data', 'predictive analytics' and 'data mining'. I have worked in this area since 1998. So here are a few thoughts:

- Good analytics (I'll combine the three terms into 'analytics' for the sake of simplicity) requires an understanding of the tools, as well as a significant understanding of statistics so that you know which analysis to pick. But in addition, it requires a lot of creativity (see my examples below) and a significant amount of time to analyze/slice/dice data in a zillion different ways.

- This is a huge opportunity. Much, much bigger than people realize and much bigger than past trends of new technologies like client-server in early nineties or web apps of 4-5 years back. Why? Because it has the power to affect business processes very powerfully.

- Example 1: I spent 10 months working for a $5B shipping company analyzing data from their Marketing department. I combined it with several hundred global data sources. I worked on over 100 hypotheses. At the end of it, I came up three specific actions that their existing customers take about 6 months before going to a competitor. The Marketing department was thrilled. They spent $17 Million coming with a plan to tackle this. It has been a few months since then; and they have not lost a single customer. This is a powerful proprietary competitive weapon for them now.

- Example 2: I analyzed 10 years of power meter reading data for a large utility company. I combined it publicly available data sources of power consumption of major appliances and census data on family composition/wealth for various neighborhoods. I was able to reliably predict the lifestyle of every family, down to whether the person living in the house streamed a movie on Friday evenings and a whole lot more. So the company decided to use this analysis to change their Direct Mailers with very specific, personalized offerings. Their response to the first test mailer sent to 10,000 people? Twenty seven percent!!! They predict that a significant portion of their profits would come from DM's.

lifeisstillgood4• 12 years ago

I am ... stunned.

What level of sophistication did you have to reach in the shipping company case. Was it, hypothetically speaking, the client starts using more than one other alternative shipper (fairly low sophistication) or was it multi factor stuff at levels of statistical skill that you need a Phd to grasp?

And thank you - these one off comments are why HN is such a valuable place to contribute to.

lifeofanalysis3• 12 years ago

It was very sophisticated analysis. But let me emphasize two things:

- When you start, you are taking a leap into total emptiness. You explore a thousand different avenues, most of them are dead ends. Your day consists of massive amount of mental effort to stay focused, to stay sane, and to make good assumptions. Then you change tack/analyses continuously. You keep adding/deleting datasets. This is not theoretical statistics but sometimes you use arcane things; so you definitely need to be very strong at Stats. My personal dream is that one day, when I have time and money, I will use these methods to come up with a Meta- Statistics approach to empirical analysis -- i.e., to use Analytics to predict, based on the problem definition and available data sets, which methods to use.

- You have to have patient clients. Jumping 10 months into a project with absolutely NO guarantee of success is a huge leap of faith, financially speaking; but the rewards can be huge. One day you are still nowhere, and literally the next day, everything clicks, you check your conclusions once-twice-thrice, make a presentation to the client, and, boom!, your project is over.

ewinters123• 12 years ago

"Past performance is not a reliable indicator of future results" this expression should be used more often when data has a big human element attached to it.

Job hunting is a matter of Big Data, not how you perform at an interview

Related Stories