What Thomas Edison expected job candidates to know

http://query.nytimes.com/mem/archive-free/pdf?res=F40F10FB355B1B7A93C3A8178ED85F458285F9

by davidvaughan • 13 years ago

136 141 13 years ago

query.nytimes.com

tjr• 13 years ago

I wonder if Edison viewed a lot of these bits of information as truly pertinent to his work (e.g., many of the questions seem to be about building materials and where to get them), and then just tossed in a few items of random trivia for good measure.

eshvk2• 13 years ago

This reminds me so much of so many software engineering interviews.

gambiting7• 13 years ago

All of those are what I would call common knowledge that an average human being should know. Not that they are relevant in any way to the job Edison was offering - but they would serve as a GREAT way to indicate whatever a person answering these questions is well educated and well oriented in the world surrounding them.

AaronBBrown• 13 years ago

Really? You think the voltage on a street car and the wood that kerosene barrels is made of is common knowledge? I would say that I knew maybe 1/3rd of the questions in that list.

gambiting7• 13 years ago

Yes. Will I boast if I say that I honestly don't know just 3 things on this list?

Back when I was at school, we had all the subjects as mandatory, chemistry, physics, biology, history - there was no choice like you get today. So yes, I happen to know the weight of the air,or the lightest/heaviest types of wood. And if you like researching things(in my case - spending hours on wikipedia) questions like the one about optical lenses are easy.

And I am pretty sure that's what Edison was looking for - people who are curious about stuff and research it themselves. Common knowledge in my opinion,because in an ideal world I would like everyone to be that curious about the world that surrounds them.

icedog• 13 years ago

How old are you, gambit?

run4yourlives3• 13 years ago

Considering in the 1920s the US was in the middle of converting horse drawn street cars into electric, and this is an application for EDISON, I certainly hope you would know that answer.

Kerosene was the most important fuel used up until the 1920s, so I'd imagine most people would have seen kerosene barrels personally. A few would have derived the type of wood via curiosity.

fnordfnordfnord5• 13 years ago

A street car? What's that? I've seen them in movies. I guess in the nineteen-twenties, knowledge about street cars might have been much more likely to have been in the minds of engineers that Edison would want to hire.

ImprovedSilence• 13 years ago

Today is not 1921, mind you.

run4yourlives3• 13 years ago

There's an assumption there that there is some reason why some of these are relevant to "common knowledge". Perhaps they had recently been in the news or something...

For example, I guessed that Spitsbergen was in Austria/Germany. It's actually a town of <3000 in Norway. Not significant, until you realize that it was the subject of the 1920 Svalbard Treaty between 41 countries. The Edison list purports to be from 1921.

So yes, you are exactly right... the man wanted people that were actively self and location aware, nothing wrong with that.

jpdoctor2• 13 years ago

I'd add: Watching the reaction of a person when they don't have one or more factoids at their fingertips is also useful for measuring the temperament of people.

The fact that he torqued a candidate so badly that it resulted in a letter to the NYT tells you that Edison didn't miss too much by passing on that hire.

doktrin2• 13 years ago

I don't see why a candidates willingness to stand up for themselves would be a negative trait, per se. If anything, it reflects poorly on the interviewer.

lbebber• 13 years ago

An average American, at most, since about half of them are USA-centric.

27182818284• 13 years ago

Common knowledge now with the Internet? Yes. At the time? No.

Those were difficult and discriminating questions at the time.

This is a time when poor girls who couldn't afford stockings wouldn't be allowed to go to school and the majority of people didn't have indoor plumbing let alone the luxury of books. Heck, 6% of possible candidates of the country were immediately eliminated because they would have not been able to read or write in any language, let alone know the arithmetic, physics, etc, required.

hhabgood• 13 years ago

They're significantly less discriminatory than you would think.

Keep in mind that he's not interviewing every single person in the country, he's interviewing college graduates.

Not only was having a college education not as prevalent as it is now, the sole fact that the people who are taking this test have graduated from college eliminates every single one of the "discriminatory" accusations you are making. If anything, the college admissions process (an obvious prerequisite of the test) is discriminatory, not the test itself.

gambiting7• 13 years ago

What is the point that you are trying to make? If somebody did not know how to read or write, they were not a suitable candidate anyway, and that's not discriminatory against illiterate people, like looking for someone with a maths degree is not discriminatory towards people without one.

RyanZAG2• 13 years ago

Common knowledge as an indicator of business fit has been proven to be useless (and should be obvious on examination). Knowing which small town is in which country is simply irrelevant to whether a candidate apply for a management position can improve morale or productivity, or whether a candidate applying for a marketing position has any idea on how to convince customers to purchase a product.

Common knowledge simply has no relation to actual skill in a set field and neither does it have any relation to intelligence. This is doubly so true today with any of these questions being answerable within seconds on a smartphone, and the thoughts instead turned to actually using the information to accomplish something useful.

Houshalter3• 13 years ago

It's possibly a good indicator of how well they absorb information and/or how much they read or pay attention to the news, or wherever they are learning these random facts from. I'm not saying that means a lot though, and I definitely would have failed this test.

alexholehouse• 13 years ago

All of those are what I would call common knowledge that an average human being should know.

I don't mean this in a rude way, but if you honestly believe an average human being will/should know all these, prepare yourself for a life of abject disappointment.

gambiting7• 13 years ago

I am disappointed every single day. Forget asking people which countries border France, I know people who don't know which countries border their own country. Hell, I even know people who don't know the day the WW2 started/ended. Sure, you can get around it life without knowing these. It will cause you no harm, not knowing how the world works around you. Sure. But I would like to live in a world where people actually put some effort into their education,and not just openly boast around about not knowing stuff. Like it is a competition to be ignorant or something, I don't know.

chc2• 13 years ago

I don't see how being able to recall on demand who Leonidas was indicates education or being well oriented with the world. I know, but discovering that someone didn't know wouldn't change my evaluation of how educated they are unless they claimed to be a historian.

tjoff• 13 years ago

"... but they would serve as a GREAT way to indicate whatever a person answering these questions is well educated and well oriented in the world surrounding them."

So then you have to decide, is having a person that is "well educated" and well oriented preferable to one that isn't?

josh-j• 13 years ago

"Should know", "well educated", "well oriented." All these qualifiers are subjective and are consequently meaningless. I could say the opposite of what you said and I'd be just as right (only taking into account what was written by us both).

You'll need to operationalize what you mean to make any sense, and then to be right you'll need to provide evidence.

driverdan• 13 years ago

I read through half of them and completely disagree. Most of the ones I saw had to do with history and geography. Those are the two subjects you'd learn from a formal education that I care the least about. They have no bearing on your ability to manage a team or run a profitable business (unless the business has something to do with those topics).

Where is my time better spent? Reading the biography about an ancient historical figure or reading a programming book relevant to my profession (and passions)? I argue the latter, especially since I don't care about said historical figure.

codex_irl• 13 years ago

Just like a software interview - demonstrate that you know a bunch of stuff that we, in our company have & never will use. For many software jobs...its like hiring a mailing man based on his understanding of mail-delivery-cart mechanics.

aroman• 13 years ago

Reminds me of Google's fabled interview process. And on that note, I find it very interesting that virtually all of these questions could be answered almost verbatim by Google or WolframAlpha.

eshvk2• 13 years ago

I don't work for Google. However, I have never heard of their interview process involving trivia factoids. There are several other companies in the valley that are guilty of this crap but Google at least focuses on algorithm questions, scaling etc.

To give you an example of what I am talking about:

1. Figure out if a binary tree is a mirror image of another binary tree.

2. Describe X where X is a language specific feature. More amusingly, in machine learning interviews, pick one out of millions of algorithms that are out there that the interviewer knows very well, demand that the candidate answer and derive every single part of that algorithm.

There is a distinct difference between demanding that people rely on memory to solve a problem vs problem solving abilities to solve a problem. While both are certainly useful for a job, I am honestly not sure how exclusively devoting yourself to the former is better than the latter.

hakaaaaak• 13 years ago

Google once required you to put together factual trivia with technical knowledge.

One question in an interview was something like: How many possible IP addresses are there per square meter of the Earth?

loboman• 13 years ago

http://www.pangeaprogress.com/1/post/2010/09/einstein-edison... 'While in Boston, Einstein was subjected to a pop quiz known as the Edison test. (...) A reporter asked him a question from the test. "What is the speed of sound?" If anyone understood the propogation of sound waves, it was Einstein. But he admitted that he did not "carry such information in my mind since it is readily available in books." Then he made a larger point designed to disparage Edison's view of education. "The value of a college education is not the learning of many facts but the training of the mind to think," he said.'

run4yourlives3• 13 years ago

Thomas Edison invented the cylindrical Phonograph, and movies were beginning to just show up in 1921. I'd imagine knowing the speed of sound - especially how it relates to syncing with video - was of major importance to him, much more than it was to Einstein.

Context is everything.

tieTYT• 13 years ago

Regardless of the context, it's still a fact that that information is readily available in books.

gallerytungsten• 13 years ago

It's somewhat of a trick question, as the speed of sound can vary greatly depending on the material it's propagated through. In air, it varies according to temperature and altitude.

sofal4• 13 years ago

Is it important in this context to know exactly what the speed of sound is in feet per second without looking it up, or is it more important to be generally aware that sound travels through air via compressed air waves and that it is quite a bit slower than light, the details of which can be looked up when necessary? Are these executives going to be tasked with making lightning-quick product decisions involving physics calculations on a daily basis?

fnordfnordfnord5• 13 years ago

I doubt Edison had a full understanding of the theory of operation for the phonograph he invented. I can't see how knowing the speed of sound in air could matter at all in relation to synchronizing audio with video, or Edison's motion picture camera. He could not broadcast sound from a speaker far enough for the sound to become unsynchronized with any video that was played.

Surio• 13 years ago

While context also has some part to play, in general I am with you in what you are trying to say.

BTW, are fictional characters counted as references? ;-)

http://en.wikipedia.org/wiki/Sherlock_Holmes#Knowledge_and_s...

From that article, In A Study in Scarlet, Holmes claims he does not know that the Earth revolves around the Sun, as such information is irrelevant to his work. Directly after having heard that fact from Watson, he says he will immediately try to forget it. He says he believes that the mind has a finite capacity for information storage, and so learning useless things would merely reduce his ability to learn useful things.

EDIT: Somewhat relevant (and OT) comic... http://theoatmeal.com/comics/tesla

brazzy• 13 years ago

obligatory debunking: http://www.forbes.com/sites/alexknapp/2012/05/18/nikola-tesl...

jholman2• 13 years ago

If you're going to use Holmes as a reference, then you agree with Edison and disagree with Einstein.

Holmes was characterized as profoundly valuing facts (as well as methods of thinking), if and only if those facts were useful to crime-solving. He was noted, for example, for his monograph on tobacco-ash residues. That is, he knew so much about the details of tobacco ash that, upon sampling some ash found at a crime scene, he could infer many useful things from it. Another example is that he is so familiar with all the various mud around London that when he sees some dried, he can guess where it came from.

Holmes is, overall, preposterous. And his theories (the theories that Sir ACD put in his mouth) are almost entirely without evidence.

That said, theories without facts to work upon are like a level with no fulcrum.

enraged_camel2• 13 years ago

Reminds me of a similar story with Henry Ford, where he said he doesn't need to know everything since with the press of a button on his desk he can summon someone who can relay the information to him.

hooande• 13 years ago

Henry Ford was notoriously ill-informed. He sued a newspaper for libel after they said he was an idiot. In the trial they asked him in what year the revolution that founded the country took place. "1812?" facepalm

edit: unfortunate typo

hsitz• 13 years ago

The Ford story has a superficial similarity to the Einstein one, but in reality they're diametrically opposed. Einstein _understood_ how the physical world worked so he didn't need to carry around unnecessary "information". Ford, at least in the story, has neither "information" nor understanding.

The distinction is similar to the distinction made in philosophy between "knowledge" and mere "true belief".

mattschoch• 13 years ago

I completely agree with this, especially for us living now, since we have access to virtually any information on the internet. However, I can see why Edison would ask seemingly pointless questions. First, while that information is readily available in books, going to a library to find the book with the information needed was much more time consuming than pulling up Google, so there is the convenience factor of memorizing random tidbits of information. Second, and perhaps more important, is the fact that Edison was an inventor, hiring people to help invent things. As an inventor, one needs to have a vast array of knowledge in many fields, and then to be able to make connections between seemingly unrelated items/topics. In our case (hacker news readers), we seek to combine separate entities in ways that no one has done before, creating a new product (read: app/website). Making connections no one else sees is at the heart of entrepreneurship, and invention as well, so having that wide array of knowledge could be see as invaluable.

Macsenour• 13 years ago

When I was in the Army the officer in charge of my section commented that another PFC was better because he read the software manuals so he could answer questions that came over the phone.

I said I memorize things that I can't look up easily.

dlss2• 13 years ago

This view is even more correct now that "what is the speed of sound" is only a google search away.

Maybe Edison only asked these questions because of how long it can take to look something up in a book.

For example, would you hire a programmer who couldn't answer:

- name one or two html elements and what they are for

- what is a for loop, and when would you use one?

These can of course also be looked up in books, but already knowing the answer to those questions (and many more advanced questions) in a lot of what you're paying for when you hire a programmer.

anonymous• 13 years ago

Well, let's compare "what is the speed of sound" and "what is a for loop". Both are domain-specific questions, one for physics, the other for programming. For both I'd accept an answer that describes the notion or idea behind the concept and not a memorised instance of it.

So, for "what is the speed of sound", I would accept an answer such as "The speed of sound is the speed at which a wave propagates through a given medium"; though I expect an actual physicist to involve molecules, springs and so on in his answer. What I would not accept is a string of digits. A string of digits shows you know how to remember a string of digits.

Similarly, for "what is a for loop", an acceptable answer is one such as "A for loop is a construct for bounded (at least in principle, but you can have unbounded for loops in some languages) iteration over a series of elements, either generated on-the-fly or from a concrete container". The analogue to a string of digits for this question would be to give the BNF definition of a for loop in C. I think you'll agree that knowing C syntax doesn't show you know how to program.

justincormack• 13 years ago

Sure I would. Some time back html did not exist, and knowing that someone decided <blink> was a good idea doesn't matter. And for loops only exist in some programming languages and are not that fundamental even if common.

I wouldn't even think to ask those questions in an interview.

cschmidt• 13 years ago

Of course Einstein is famous for not knowing his own phone number in Princeton. He didn't see the point in remembering things he could look up.

etfb• 13 years ago

I have to nitpick there. I suspect Einstein is actually famous for his Special and General Theories of Relativity, and for his amazing hair.

ajross• 13 years ago

This is undeniably true, but I think misses an important correlation: in my experience, people who are best able to find and access "minor facts" like this when needed are precisely those who are commonly considered "walking encyclopedias". I strongly suspect this was true of Einstein too, even if he couldn't pull up that one figure.

I think penalizing someone for not knowing any fact in particular is silly. But throwing a ton of questions like this at a candidate just to see how many they hit has, IMHO, more value than is commonly admitted.

jodrellblank• 13 years ago

In the original link, I didn't see any claim that Edison was looking for correct answers, or that Edison knew the answers. Only that he looked at the paper for a bit and then dismissed the candidate.

For all we know, he was looking for people who deliberately ignored some subset of questions as irrelevant, or who demonstrated some quality of judgment such as questioning the appropriateness of the quiz verbally before starting, asking to take it in a quieter environment away from the arguing people, helping mediate the argument instead, stopping and asking for their first few answers to be checked to see if there was any point in them continuing, or ... anything at all, really.

nsoun• 13 years ago

How I read this: Einstein would have approved of candidates using Google.

raintrees• 13 years ago

Link is to a pdf... (Don't we normally indicate this in the title?)

Houshalter3• 13 years ago

So what? It's not like a research paper or anything, just an article that has been scanned.

deepblueocean• 13 years ago

I wonder what Nikola Tesla asked prospective assistants?

fnordfnordfnord5• 13 years ago

[T/F] Thomas Edison is a brilliant man of impeccable morals and character.

* And I bet if you circled "T", he'd zap you with a Tesla Coil and have you thrown out.

d0m• 13 years ago

Still, before there was no google/wikipedia so common knowledge and general interest was much more important.

vitno• 13 years ago

My first response was indignation. A lot of those questions are seemingly irrelevant. However, I think we need to put them in perspective of the business and time. Once you do that, A lot of them become something I would expect people to know.

nathanb• 13 years ago

Would he appreciate an intelligent guess?

If I didn't know how much a square foot of air weighed, could I represent it with x and then give a formula?

If he asked me who discovered the south pole and I asked him to clarify whether he meant who first reached the south pole, who first postulated that the earth, being spheroid, must have a southernmost extremity, or who first realized that the earth produces a magnetic field, would he be impressed or merely annoyed?

I'd be more interested to know how he reacted to the answers than the fact that he asked the questions.

fnordfnordfnord5• 13 years ago

From what I've inferred by reading about Edison, he probably would have feared you more intelligent than himself; and wouldn't have hired you on that basis, after taking the opportunity to chastise you for your lack of knowledge.

scarmig2• 13 years ago

These... honestly aren't that bad. Not that I know all of them, but many of them, totally. And those that I don't are pretty context dependent, particularly those about sectors of the economy that have been de-emphasized or historical tidbits that have been de-emphasized.

Replace "what city produces the most laundry machines" with "which American city is known for producing cars" and "from where domestically do we get sardines" with "what part of the United States is known for producing wine"?

Note also that there are some ostensibly hard questions that seem blindingly obvious to people here now: "Where is Korea?"

absconditus• 13 years ago

Why are any of these good interview questions?

trentmb• 13 years ago

because he knew the answers to them.

scarmig2• 13 years ago

Never said they were good interview questions, but the implication elsewhere has been "these are impossible to know the answers to!"

numbsafari4• 13 years ago

Perhaps if you are going to run an international business that may be interested in expanding to the Pacific countries, having employees that can physically locate Korea might be important. Apparently he couldn't "just assume" that every college graduate had ever spent time curiously inspecting a globe and developing a general knowledge of different parts of the world.

GabrielF00• 13 years ago

They aren't good questions for, say, a software engineer, but they might be useful for executives, which is how the Times article says they were used. If you're going to be analyzing potential business opportunities you probably should have a sense of different markets, their size, their relative strengths and weaknesses, etc. You should probably have a basic idea of where we get products and materials from and a basic sense of what different types of materials do. In our post-industrial society we don't care so much about sulphur and rubber and borax, but would you hire a tech executive who didn't know where electronics are manufactured? Would you hire someone who couldn't identify the regions of the US where the tech industry has a large presence? Would you hire someone who couldn't identify "six big businessmen in the United States"? If you can't answer these questions than you don't know anything about the industry and you probably aren't curious or engaged enough to keep yourself well informed.

If we updated this list to include things that are relevant to today's world, I would expect that a well-educated, well-informed person would be able to answer most of these questions.

qxf2• 13 years ago

I can see general knowledge type of questions useful for several reasons in the time Edison lived:

a) How good is the candidate at saying 'I don't know'?

b) Can the candidate handle repeated failures i.e., a string of "I don't know"?

c) Can they BS? Instead of leaving a blank, can they BS about something tangential?

d) Building hype around Edison. Imagine if the smartest person in your town failed to get hired. In the days of telegraph, the word of mouth would be 'only the best and brightest can become inspectors at Edison's factories.'

e) Head fake. Perhaps Edison really judged the candidates in some other way but made a big show of 'ignorant college grad' to prevent people from gaming the interview.

f) Make people that pass the test feel privileged. Its a common recruiting tool.

I think Edison was a good at spotting and using talent (yeah, using) for his own ends. I give him the benefit of the doubt on the interview process.

hnriot• 13 years ago

Many of these have ambiguous answers, like the country that makes the best optical lenses, who invented photography, axe handle wood etc. There are different answers because they are either subjective (Leica, I presume he meant, but Zeiss would be a solid choice too), Photography was possible the french guy Niépce, or Fox Talbot in the UK and there were many US contenders for the title also. Axe handles are made out of many types of wood...

These questions seem rather pointless, it's a case of I know this, so you should too. If Edison had been born on the west coast he would have likely asked a different set of questions.

gravedave3• 13 years ago

He probably relied on the popular options. If all your neighbours think of Leica as the best, there's no way you'd pick Zeiss in a pre-internet world without a Wikipedia that has a disambiguation page for every other term. Sure, people on the other coast might pick Zeiss, but how many people were willing to travel across a whole continent for a job interview in those days?

mumrah• 13 years ago

+1 for use of ampersand ligature in "&c", i.e., "et cetera"

gwern• 13 years ago

> "Only some thirty of the several hundred applicants have managed to pass the test, it is true, but those who did and thus became inspectors of the factory have made good in every case."

Of course, there's no control group... One might expect his half-baked trivia test to work a little bit (being a poor IQ test), but there's no way to know.

couchnaut• 13 years ago

http://theoatmeal.com/comics/tesla

yes - and we should we give a f.ck about Edison's views? The man was no better than a lowlife

MSM• 13 years ago

And you're willing to admit that a webcomic convinced you of that?

_Dude_2• 13 years ago

That's just a neat visual summary that everyone can quickly scan. Kinda like that post about another thief of ideas - Zynga.

gravedave3• 13 years ago

Heard of Ad Hominem? It seems to affect you. He might have been a thief, but he was a quite capable business man that many could learn from.

sown• 13 years ago

To be sort of fair, Mr Edison was an engineer, and it was back in a time when there was more 'empirical' methods rather than predictive theories of why such and such happened. So, when there's an emphasis on building things, knowing all of this can be helpful. There's also a business type question in there, perhaps to keep an engineer's awareness of business needs in mind.

For example, as Engineering Guy Bill explains, http://www.youtube.com/watch?v=DIGqBb3iZPo, the method for discovering light bulb filaments was very much a search problem, so knowing as much seemingly distant facts about barrel wood, cork, optical facets might actually be important.

Questions about Cleopatra and so forth may have been to see how educated an applicant was. I wonder if those helped or hurt a candidate. I.e., did this person learn this on their own through curiosity or did they pick it up at school...

Disclaimer: I am not qualified to give opinions on any topic.

appleflaxen• 13 years ago

The prune question must be very important; it's on there twice.

numbsafari4• 13 years ago

You could say it came up regularly...

enraged_camel2• 13 years ago

I have mixed feelings over this.

On the one hand, knowledge is different than education. Knowledge is the possession of information. Whereas education is the ability to find information quickly and efficiently, and pass it through a filter of critical thinking. Questions like the ones on Edison's list measure knowledge, but not education.

On the other hand, in my personal experience people who possess seemingly "random" pieces of information such as the location of countries on a world map or the birthday of a jazz singer tend to be much more productive. Not because the random bits of knowledge they possess are related to the work they are doing, but because information like that gives them a wider perspective on everything and allows them to be better at "pattern-matching", i.e. drawing connections between seemingly unrelated fields and subjects. This is a very, very important skill for any knowledge worker.

jacquesm3• 13 years ago

There is a compound effect to knowledge though. The more of the bits you've got the more the bits you already had start to make sense. And at some point knowledge starts to beget new knowledge all by itself, ideas and hypothesis about how unknown stuff could work based on what you already know, and sometimes completely new stuff.

So even if knowledge does not automatically mean understanding or education it can be a precursor to it and it can make it easier to attain the latter.

Silly facts (baseball scores for instance) do not have that effect. So there is a definite division between the kinds of facts that you can digest and their future effects.

dkarl• 13 years ago

More than anything, this makes it sound like Edison only knew how to get useful work out of somebody whose brain worked like his.

arbuge• 13 years ago

Some of these might have been more valuable in the days when you couldn't just Google anything in an instant to find the answer.

richardjs• 13 years ago

Part of me wonders if Edison wasn't concerned so much with the actual answers to the questions as how an applicant went about answering the questions, and how the applicant reacted in the face of the test. Or perhaps he cared about some questions, and threw in a bunch of random ones to see what the applicant would do.

moron4hire• 13 years ago

Was he serious with these questions, or were they sideways questions designed to suss out how the tested person thinks?

stox• 13 years ago

And the most important question of all, "Are you a big enough sucker to believe I will give you the $50,000 I promised for building the invention I asked for?"

heironimus• 13 years ago

I think this list would certainly fail a lot of good, smart candidates, but it probably didn't pass many idiots. It's not likely that someone would know all of these facts and not be well-educated and intelligent. If he was truly getting hundreds of applicants and he didn't mind skipping some geniuses who didn't know facts, this seems like a good way to weed people out. Plus, it's pretty easy to grade.

tokenadult2• 13 years ago

Thanks for sharing the link to the interesting series of questions. As I read along, I tried to think about what the correct answer was--or how it would be defined--for the various questions. I imagine that today many of the questions about geography would be less asked, although knowing about other countries still matters for international business.

We often talk about company hiring procedures here on Hacker News. From participants in earlier discussions I have learned about many useful references on the subject, which I have gathered here in a FAQ file. The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings," Psychological Bulletin, Vol. 124, No. 2, 262-274

http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

sums up, current to 1998, a meta-analysis of much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.

http://www.siop.org/workplace/employment%20testing/testtypes...

EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.

The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.

Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case

http://scholar.google.com/scholar_case?case=8655598674229196...

interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. Thomas Edison's test might face the same challenge today. Thomas Edison or anyone else defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)

The social background to the legal environment in the United States is explained in many books about hiring procedures

http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6...

Some of the social background appears to be changing in the most recent few decades, with the prospect for further changes.

http://intl-pss.sagepub.com/content/17/10/913.full

http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...

http://books.google.com/books?hl=en&lr=&id=frfUB3GWl...

Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures" (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:

"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."

http://geb.uni-giessen.de/geb/volltexte/2012/8532/pdf/prepri...

But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."

One thing I have to say about this whole issue, after a thoughtful comment from another HN participant off-forum, is that hiring managers have to be prepared for the development of their workers. The programmer you hire today may be a manager three years from now. Being sensitive to how workers grow in the workplace is at least as important for managers as making a good hire at the beginning.

graeme2• 13 years ago

This is useful information, and a good, well-documented discussion.

However, I'm finding it kind of tiresome to see it at the top of every job discussion, particularly since it's so lengthy. I don't think this Edison post was meant as a serious suggestion for how hiring can be done.

Perhaps you could make a blog post or pdf that contains the copy-pasted text, and then produce a summarized comment that links out to this info for anyone interested?

I'm sensitive to the argument that hiring related submissions will give bad advice if this context isn't provided. But on the other hand the status quo is tiresome for those of us who have seen this comment over and over.

tokenadult2• 13 years ago

The FAQ included in my comment is slated to be a new page on my personal website. I certainly intentionally miss some opportunities to post the FAQ here on HN, as discussions of somebody's article or post about how to hire come up here more than once per week on average. Those threads often have very active discussion. Sometimes I post the FAQ, and other times I just lurk.

zukhan• 13 years ago

Agreed. I'm sorry, but it's getting annoying to see such a lengthy post on top of every job discussion.

dfabulich• 13 years ago

This post is not in a FAQ (question-and-answer) format, which is unfortunate, because I think it would benefit from it.

Frequently asked questions might include:

1) What is a reasonable work-sample test for programmers? 2) Should programmer work-sample tests be assigned as homework, or should they be administered in person? 3) Will the very best programmers refuse to take work-sample tests?

zach• 13 years ago

Basically, it seems Edison wanted his executives to have his intense intellectual curiosity (he was a notable autodidact). If he knew the kind of people he felt he could lead effectively, and this was his perhaps-crazy way to identify them, this seems like a good test for those circumstances.

One of the most damaging things is when you have what Gabe Newell calls "rent-seeking inside the corporation," which is a neat economic way of describing bureaucratic political power struggles. And this is totally normal, expected behavior unless you find someone who is pre-aligned with the mission, goals and values of the organization. This becomes totally crucial at huge corporations like Edison's.

So this, in a sense, is a cultural test more than a knowledge quiz. Thomas Edison didn't want people who could win at 1920s Jeopardy!, he wanted people who were driven by the same non-monetary pursuits he had, possibly because it was his best chance to avoid BS artists, pleasant-but-ineffective workers and political strivers. I also have to think that it was because he was a pretty narcissistic dude, but that's another story.

ynniv• 13 years ago

So what you're telling me is that people have always sucked at evaluating job candidates.

patfla• 13 years ago

I'm speaking only from my own experience but as a software engineer observing other software engineers a surprisingly large percent of the the (imo) very best have striking degrees of general knowledge. That is, that could speak very well to many topics other than just software engineering and they have eclectic and deeply developed 'outside' interests.

I believe this runs counter to the current politically correct understanding of intelligence or if you prefer intelligence(s) - although I do believe in the latter.

_Dude_2• 13 years ago

Useful if you want to hire someone you could steal ideas from... Someone like Tesla.

jleader2• 13 years ago

I found the test very interesting. You have to keep in mind that Edison was a prolific inventor, who at the time was in the business of disrupting as many industries as he could. From that perspective, he was looking for people who knew a lot about the current state of technology and business, and also able to bring to bear knowledge from apparently unrelated areas.. Many of the questions are trivia related to the technology of the time ("who invented photography?", "where is platinum found", etc.). I think the expectation was that someone who was actively interested in technology would have picked up a lot of that sort of trivia along the way. Some of the less technological questions ("what's the capital of Alabama?") are probably just trying to evaluate how aware the candidate is of the world around them.

A similar list today, say for a candidate to help run a high-tech incubator, might consist of questions like "who founded Google? which is preferable to a seller, a 2nd price auction or a standard auction? where was the web invented? what's the geopolitical and technological significance of tantalum? are lithium batteries riskier than other battery technologies, why or why not? what's a typical price for web advertising (per click, or per impression)? what's a zero-sum game? what's a derivative?" (Those are just off the top of my head, I could probably come up with a better list if I thought about it for 15 minutes). In other words, not things that you must know in order to do the job, but things that anyone capable of doing the job would likely have picked up along the way.

pessimizer• 13 years ago

This reads like a "smart test" rather than anything written by an actual engineer. Of course, I've never seen any evidence that that Edison had any engineering ability other than the ability to demand engineers invent things, and then to engineer that he receive the credit for them.

afterburner• 13 years ago

Did Edison treat his engineers well? If not, should I care what trials he put potential candidates through? If he was the overly manipulative type, perhaps he was mostly interested in humbling the candidate...

pfedor• 13 years ago

Is there any reason to believe the connection between this list and the questions Edison actually asked is any better than, say, the connection between what a Google interview actually looks like and how the newspapers like to portray it?

What Thomas Edison expected job candidates to know

Related Stories