Dear Startups: stop asking me math puzzles to figure out if I can code

http://countaleph.wordpress.com/2013/10/20/dear-startups-stop-asking-me-math-puzzles-to-figure-out-if-i-can-code/

by brryant • 13 years ago

872 341 13 years ago

countaleph.wordpress.com

rdtsc• 13 years ago

Two possible reasons:

1) I think a lot of start-ups want to hire "smart" people. Because they expect the new person to eventually wear many hats. Objective-C, Java, Android, CSS, server side concurrency, monitoring. An we've all seen Hunter and Schmidt reference that tokenadult usually posts when talk about interviewing comes around and it does seem that a general mental ability test (like an IQ test) combined with a work samples seem to predict future performance of that employee. Well except that one can't just straight up give IQ test to job applicants (there is a court case about that). So we are left with a job sample (which many forget to give, as is the point of the author). But instead many focus on the GMA and create proxies for it -- cute little puzzles about blenders, round manhole covers, and other such silly things.

2) Those interviewing don't know the technical stuff and are afraid you'd out-bullshit them. "How does an Ajax request work" well if the interviewer themselves doesn't quite know the details the might not be able to evaluate it properly. They could have it written down but well, some technical questions have many different levels of depth that a candidate might descent to. So a quick written answer to the question might seem wrong but it is really because the candidate is more advanced. So puzzles seems to be a generic and "easier" to handle.

morgante9• 13 years ago

I think (1) is probably the primary reason, along with the fun factor I mentioned above.

But honestly, why can't we just start giving IQ tests? Or at least asking for SAT scores? We shouldn't have to run through training puzzles just to prove we're smart enough to build a site.

deluxaran2• 13 years ago

What kind of IQ tests? Math thinking? Language? Other? These tests are kinda old and don't prove anything besides the fact that the person has a certain level of intelligence but that doesn't mean they can code.

bradleyjg2• 13 years ago

But honestly, why can't we just start giving IQ tests?

It's illigal. Or at least very very legally questionable.[1] Theoretically good proxies could be attacked in the same way, but there's a difference between doing something that could conceivably be the basis of a lawsuit and doing something that the Supreme Court has specifically ruled on.

[1] See Griggs v. Duke Power Co. (1971)

http://finduslaw.com/griggs-v-duke-power-co-1971-401-us-424-...

redschell• 13 years ago

Management consulting firms and Wall Street banks have employed somewhat strict GPA and SAT/GMAT cutoffs for quite some time, but even they can't resist the allure of brainteasers during the interview process.

Google, on the other hand, appears to have done away with them, and now prefers to evaluate potential hires without giving too much weight to GPA or test scores: http://qz.com/96206/google-admits-those-infamous-brainteaser...

driverdan2• 13 years ago

Standardized tests like SATs and school grades are a terrible way of ranking a potential employee and have significant biases such as income and age.

I didn't have very good grades and my SATs were ok but not great, both primarily because I didn't study or apply myself. I was completely uninterested in school. Get me to write code and you'll see what I'm capable of. I'm certainly not unique in this regard.

aestra3• 13 years ago

Because SAT scores are a very very terrible indicator of if you have 1) the ability and 2) the work ethic to build a site.

And how old are you? SAT scores? I'm only in my 30s and I don't remember my what my SAT scores were or remember if they were "good" or "bad" I only know that they changed the SATs after I took them because my youngest sibling took a different one than I did, with some crazy scoring. I wouldn't be able to find my SAT scores even if I wanted to.

dinkumthinkum• 13 years ago

In what way will an IQ or SAT test give you an indication if the applicant even knows what a compiler is? Your comment is cute but I'm not sure how useful it is ... :(

thraw3• 13 years ago

1) I think a lot of start-ups want to hire "smart" people.

What does it really mean to be smart? Lately I cannot stop thinking about it. I have always been considered a 'smart person'. I am a self-taught freelance developer now - it used to be my hobby and somehow (mostly because I needed location-independent work quickly) it became my profession. I get by because everybody thinks I am smart but I feel like an impostor because the more I think about myself the more I realize that intelligence is not some general ability to solve problems - it's more just a set of very different skills that corelate to much lesser extent than people usually think and you can be really good at something that people use to judge your abilities and at the same time really bad at something else that is actually required to get the job done.

I studied sociology and I shortly worked as a data analyst. It seems to me that this kind of work requires... ehm... a different intelligence than programming. You need to be good at connecting the dots, noticing things, seeing patterns. This is my kind of thinking and I have always been good at this - doubting everything, seeing pure assumptions where other people saw 'truths', permanently creating hypotheses and alternative theories, trying to spot logical fallacies in prevailing theories... basically trying to spot things.

Programing is very different (at least it seems to me - so different tat it's even difficult do describe it). I guess it's about creating stuff, not just observing stuff. You need to build very complex and abstract mental models, keep them in your head and be able to operate with them - and this is the part of intelligence that I seem to be lacking. It just does not feel natural. I try to solve some problem and I am thinking... if this condition and that condition but at the same time not that condition... and bang!, suddenly I am lost and I don't even remember what I am doing. I cannot keep it in my head. I totally get what OP was saying about passing anonymous functions in JavaScript - I had the same experience. The first time I encountered something so simple as JavaScript closures it took me two hours to get it. And the day after that I had to repeat the whole mental process to get there again because I somehow lost it over night. This is simply not how my brain works and I think I am really bad at this. Yet people pay me for this... which is just depressing (and you understand why I write this under throwaway).

I remember our 'statistical analysis 101' professor always telling us 'remember, you are not really testing hypotheses, you are only testing indicators!' - if HR department picks wrong indicators for the skills that they actually need both company and employee are going to be unhappy - and I think this is very common because our understanding of the indicators for different kinds of 'being smart' is still poor.

samatman• 13 years ago

You're conflating intelligence and knowledge. Intelligence is a matter of capacity and capability; knowledge is one of the things intelligent people have a great capacity for attaining and retaining. The main thing, in fact.

Knowledge, unlike intelligence, is in practice infinite. And now you can understand the inverse Dunning-Kruger effect: intelligent people have the capacity to recognize how little knowledge they have, and can ever have, about any given topic.

scott_s• 13 years ago

The first time I encountered something so simple as JavaScript closures it took me two hours to get it.

Closures are not simple. They are a powerful concept, one which introduces a whole new way of thinking about programming compared to what most people start out doing. It's normal for it to take a long time, over many sessions, for the concept to sink in and become a part of your programming vocabulary.

Freshmen year of undergrad, it took me well over a week to wrap my head around this "public" and "private" concept of classes. I had only written C++ that was mostly C with structs and C++ input/output. I had never had to reason in an object-oriented way before, and it was different. It was completely alien to me that I would intentionally "hide" some parts of my code from myself. Now, it's completely natural to me.

What the author did not see was those same developers learn about closures for the first time. When she sat down with them, they had already reasoned with the concept many times. Don't confuse initial difficulty with some inherent deficiency in your mental ability.

brspangler• 13 years ago

* I get by because everybody thinks I am smart but I feel like an impostor*

That's a real thing. Almost all of the wildly talented people in my life feel Impostor Syndrome[0] at some level. Three Panel Soul even did a spot-on comic about it [1]

[0] http://en.wikipedia.org/wiki/Impostor_syndrome

[1] http://threepanelsoul.com/2013/10/14/on-impostor-syndrome/

edited for sane line breaks

taurath2• 13 years ago

I've been thinking about the same problem for a while, and I'm also a self-taught developer that looks way towards sociology. I have a little different spin on the main hurdles of doing programming. With proper tools and the right mindset, the entire goal of the organization process is that you DON'T have to think and hold 100 things in memory at once. It works okay for some problems, but any problem with a sufficient complexity (like, say a full rich website with lots and lots of features) will be nearly impossible to work with if everything was intertwined.

I find the hardest park of programming is getting over hurdles as they come up, over and over and over again. It takes a while to get over fear of small failure. Also, if you're constantly reanalyzing and have a perfectionism streak you can be paralyzed quite frequently. Have you run into the same problems?

morgante9• 13 years ago

It is rather unfortunate how little correlation most tech interviews have with their respective jobs. It's largely a lose-lose situation for everyone. Developers who could easily build great systems but aren't experts in graph theory get passed over while brilliant mathematicians who can't necessarily code get hired. Result? Companies simultaneously having to fire employees while facing a supposed talent crunch. Given that this hurts everyone, how did we even get into this situation?

Probably because the only person who doesn't lose from this is the interviewer: they get to have fun. Honestly, when you spend all day buried in code, it's fun to play with puzzles for a change.

Perhaps it's time we started optimizing interviews for hiring success rather than interviewer happiness.

kenster076• 13 years ago

Okay, but any specific suggestions on how to do that?

morgante9• 13 years ago

Start with a simple interview which simply checks that the person (a) has a brain; (b) is somebody you're comfortable working with.

Then do a short-term contracting gig (maybe just 1 day). If it works well, hire them.

Edit: clarify length which even the best people could do.

deluxaran2• 13 years ago

My opinion on this is that most of the interview processes is pretty old(over 20-30 years) and back then a good programmer was also a pretty good mathematician, and now most of the people that do interviews just use the same old patterns because, maybe, some of them don't know any better or because that is what they found in some books they have read.

I tend to hate the interviews that ask me to solve math and logic brainteasers because I don't see the value in them regarding my knowledge of programming.

eksith2• 13 years ago

This may be another reason people are eager to start their own company in lieu of working for someone else. If the questions are rubbish and completely unrelated to the actual job, then there's a huge disconnect between the interviewer (or HR company, as a lot of places outsource that) and where the actual work is to take place. I blame both.

The irony is that, in an effort to hire the "smartest" people, they leave out the wisest. Which is arguably more useful.

jzzskijj• 13 years ago

This may be another reason people are eager to start their own company in lieu of working for someone else.

If it is so, then it is a good thing. Right?

eksith2• 13 years ago

Well, if they really enjoy the challenge (and it is challenging), then yes. But I think there's something sad about being forced into it because hiring practices are archaic elsewhere. It doesn't have to be this way.

I don't like it that so many new startups fail and I have a feeling many of these (besides lacking an idea, failure in implementation etc...) are due to a lack of other options. If you can't get hired with the skillset you have, even though it should be more than enough, it can drive you to do desperate things. Including becoming a founder.

collyw4• 13 years ago

Depends. They will likely work on the sort of problems they are good at solving. But at the expense of spending a large part of their time running a business, rather than solving the problems they are good at.

Xylakant11• 13 years ago

I actually like asking math questions on interviews. It shows how people approach a problem. Asking code questions in an arbitrary interview setting shows just about nothing - no access to a reference doc, somebody peering over your shoulder. Heck, I couldn't code my way out of a wet paperback in that setting.

Certainly, asking only math questions is stupid as well, people should know at least a little about the stuff they're supposed to work with, but teaching an actual language to a smart person eager to learn is a breeze compared to teaching problem solving to someone who memorized the reference manual.

kenster076• 13 years ago

"...a smart person eager to learn is a breeze compared to teaching problem solving to someone who memorized the reference manual."

I would guess that there is actually significant overlap between these two groups.

morgante9• 13 years ago

I would guess that there is actually significant overlap between these two groups.

I wouldn't. Smart people get bored sitting around memorizing things. They'd rather be thinking. Purely anecdotally, the smartest people I know rarely have encyclopedic knowledge of anything.

Pitarou2• 13 years ago

Not really.

Memorising the manual is the strategy of a surface learner. Surface learners can make pretty good PHP programmers, but they'll struggle with, say, passing Javascript closures.

A deep learner will look for the underlying principles and abstractions. They can quickly get an overview that, even when it's fuzzy, is still accurate enough for them to know where the gaps in their knowledge are, so they can fill those gaps quickly when they need to. They're like Mendeleev with his first periodic table. He didn't need somebody to show him a sample of gallium to know that it existed: he could inferred its presence and properties from the overall structure of the system.

Of course, a really advanced programmer will have done both. They will know and understand everything. But those guys are few and far between.

auggierose2• 13 years ago

I think you don't get the point. Learning a new language after you programmed for 5 years in a variety of paradigms like assembler, object-oriented, purely functional, will take you between a couple of hours and a week. But if you have NOT had those 5 years of varied programming experience, then the new language is not your problem. Learning the concepts is, and that will take time.

Xylakant11• 13 years ago

I think you don't get my point: I expect a domain expert to be a domain expert, but I don't expect a trainee to know anything and I'll structure my interview accordingly. I'll ask the domain expert questions about his domain. But I don't expect a domain expert to know the programming language that we work with just because he's a domain expert. And seriously, I'll prefer the eager-to-learn math graduate with little coding experience over the bored cs graduate that knows the java reference doc by heart and now thinks he knows how to build stuff. In any case coding examples in your interview will show you little beyond the fact that somebody memorized the docs.

The actual pain point I read from the article is a different one: There's a mismatch between hiring process and expectations. If I need a lead programmer for the iOS project that I'm about to start next week then I can't hire anyone that doesn't know Objective C and then I absolutely need to structure my interview accordingly. But if I have a couple of weeks more I'll happily teach him. And if I'm hiring somewhat smart I try to avoid those "we need someone urgently" situations.

kelnos5• 13 years ago

Learning a new language is fairly easy. Learning the language's standard library to the point that you don't have to look at the reference docs every 30 seconds takes a bit more time.

ubersoldat2k7• 13 years ago

So true. I've been a professional developer for 15 years and I've worked with several languages (C++/Java/PHP/Python/Perl/C#/Ruby/JS/Groovy), but man, does this whole "functional" stuff is being hard to get a hold of. Specially Scala, for me, is very hard to understand the syntax, even after a year of working with Play. The most frustrating part is understanding the flow of the data in chained calls like in map-reduce.

uniclaude2• 13 years ago

Asking math questions might exclude some good candidates. I know more than a couple very productive programmers that did not go through a formal CS education. Asking math questions to those candidates could even scare them for no reason. I hired people who had no idea about Lagrange Multipliers but were able to ship code in various languages and even learn new paradigms when necessary.

There are not only smart people and persons referencing reference manuals. Being a programmer often means solving bugs in messed-up codebases, build web apps using the technology du jour, or making data go from one place to another, and asking math questions does not help a lot to find people able to do this. This blog post resonates with some people I met.

I have been programming for a while, and went through a CS education, but my experience with hiring made me realize that being good with maths and being a productive programmer are not necessarily two things that always come together.

baddox• 13 years ago

Everything might exclude some good candidates, right? With the possible exception of a lengthy internship or "trial period."

Xylakant11• 13 years ago

Asking math questions might exclude some good candidates.

Every question can exclude a good candidate, especially if you only ask the question and tick the "correct/incorrect answer" box. However, often you can learn the most interesting trait from asking a question which the candidate can't answer right off the bat: How does the candidate deal with failure or lack of knowledge. Does she/he start guessing? Does she/he ask the right questions moving in the general direction of solving the problem?

I'm not checking for academic knowledge in interviews.

and even learn new paradigms when necessary.

This often requires knowledge about the stuff you don't know. That is a value of formal education: Not the stuff that you memorize, the bigger value I derive from my formal education is all that "I know that there's a solution to the problem but can't remember exactly" kind of knowledge. I can't remember all the sort algorithms I had to code, but I remember there's more than one and that there are tradeoffs between all of them. So if I'm constrained on memory and have a pre-sorted list I can go luck up how bubblesort is implemented exactly. That's a knowledge that self-taught programmers often lack [1].

being good with maths and being a productive programmer are not necessarily two things that always come together.

No, nobody proclaimed so. But having a trait for problem solving and logical puzzles certainly helps :)

[1] n.b: often. Some of them have read and digested tons of theory books which could count as formal education.

RogerL4• 13 years ago

Did you read the OP? The point of the article was that one can be great at that sort of thing and yet quite terrible as a production programmer. You are not measuring the skill you want with math problems, you are measuring a proxy.

Furthermore, I've known plenty of smart math people that just never seemed to be able to program (well). I think they are different skill sets, certainly with plenty of overlap, but plenty of differences as well, and those differences matter. I laughed out loud at the "variables, variables everywhere!" answer in the OP - I've had to deal with so much code written that way. Some people, very smart people, just don't 'get' design in that way. I worked with a guy that used to run around the office, asking brain teasers, sharing tidbits of knowledge, but he couldn't execute a basic project - couldn't plan what to do, couldn't do things in a rational order, couldn't experiment and gather data, couldn't incrementally design, develop, refactor, nor do a big-bang waterfall kind of design, and so on. He was not sorely missed when let go. Smart as a whip, and useless (for programming).

I've worked at several companies with staff mathematicians. Sooner or later they got their hands on a compiler. Oh my. No, let me do that. You tell me what is wrong with my Kalman filter, but I'll take care of the implementation, thank you.

Its easy to kvetch at somebody else's answer without offering an alternative (I do agree whiteboard progamming is disastrous). So, instead of asking math, why not ask them to write a simple routine, but then start asking real world problems about the code they would face - how would you make this API interface robust? What kind of documentation would you write. How would you handle errors? Is this code exception safe? Thread safe? How would you make it either/both of those. Suppose your problem size was n=100MM, how might you need to change this (say they have a data structure that loaded everything into memory)? Ask them some problems they will see in production - what is the network delay, or whatever your problem case is. You still get to see how they approach problems, but in the context of the actual decisions they will be making while programming for you.

Anyway, that is what I try to do. I am revising my thoughts even on that, because I find people flopping on the 'code the simple problem (and, it is simple)' yet doing great on all the engineering questions, and doing fine if we hire them.

Xylakant11• 13 years ago

Did you read the OP?

Did you read my post? May I quote myself:

Certainly, asking only math questions is stupid as well

So what I actually advocate is asking questions about the field that the person is going to work in. I still like to ask some math questions - a lot of the stuff that programs deal with basically fall back to math. Heck, all relational database stuff falls back to set theory, so knowledge in that area certainly helps a lot. I'm not advocating employing pure theoretical mathematicians to develop contact forms in php. However, from my experience the following two statements hold true:

* Studying CS does just about nothing good to your actual programming skills and your knowledge of real-world problems. So just don't expect recent graduates to be able to develop a program in a rational order, using a reasonable process, refactor or any of the skills you're asking for just by virtue of having "programmed". If you're looking for an experienced programmer hire for an experienced programmer, but the OPs tale clearly shows that she was no experienced programmer - why would I quiz her in the way I'd quiz a 10-years-learned ruby programmer?

* If you have time to teach and educate a programmer, prefer a smart and eager type over someone who spools down memorized knowledge. While memorized knowledge can sometimes help, the ability to learn and educate yourself is much more helpful once your memorized stuff is not saving you.

driverdan2• 13 years ago

What kind of math questions do you ask? Are then relevant to the job? Is it knowledge that the employee will need to have for their day-to-day tasks?

Asking math questions that are unrelated to the employee's tasks has strong bias for recent graduates. Ask a 40 year old a question about an equation they haven't used since they were 19 and they won't remember.

I was once asked to do binary math on a phone interview. I hadn't had to do binary math since I was in school 10 years prior. I was pretty much guaranteed to fail. This was not a good test. If I needed it I could easily relearn binary math in a few hours at most.

jph2• 13 years ago

Breadth-first search from both ends.

I believe this is deeply valuable. For some roles, I would much prefer to hire someone who can quickly see the value of breadth-first search from both ends.

If he/she doesn't happen to know the syntax of Ruby, or Java, etc. it's less important to me.

dagw6• 13 years ago

I agree as such. Given the the choice between someone who's really smart and can solve hard problems, but a mediocre programmer or someone who's a good programmer, but sucks at problem solving, I'd chose the first one. That being said, if you hire someone in the first category, don't expect them to be happy and competent at writing an iPhone CRUD app in two weeks after seeing Objective-C for the first time.

I think that was the thrust of the article. If you're hiring based solely on someones math and algorithm skills with zero concern about their coding skills, you cannot turn around and be angry when their coding skills aren't what you needed. It's not math vs programming as such, but more generally about tuning your interviews to finding the skills you actually need (as opposed the skills you think you need).

greendestiny• 13 years ago

I think you're still suffering from same mindset that led to the problem described in the article though. He wasn't a mediocre programmer - he was a bad programmer. If you make a lot of assumptions about people in an interview you will probably be disappointed.

tylerkahn3• 13 years ago

I mean this as a genuine question:

http://sixarm.com/

This is your company, correct? (I stalked your Github profile)

Seems like the types of problems you're solving are exactly those which require far more domain experience with Ruby/HTML5/Javascript/whatever than the ability to see the value of various graph-searching techniques.

Would you hire this guy even though he's said quite plainly that he lacks the experience with these technologies (and has difficulty picking up new ones owing to that lack of experience)?

jph2• 13 years ago

TLDR: Yes, for some roles.

Some of our projects have us on site, side-by-side with a client's staff programmers. These projects involve millions of dollars, hundreds of stakeholders, and years of existing code. The programmers have a wide range of skill levels.

The work in these projects involves figuring out the project's objectives, goal decomposition, some agile/lean PoCs, then developing the BDD, MVC, DCI, API, CQRS, SQA, etc. Much of this can happen in pseudocode.

We also do pair programming, code reviews, brown bag demos, cross-training, and the like. I believe all these can help with developers getting up to speed with language syntax.

That said, choose the right person for the job. YMMV.

uniclaude2• 13 years ago

This is great. Now what would you do if after three months, your new hire can still not ship code that benefits your business? Fire the person? Keep teaching?

I also believe algorithmic knowledge is important, and tend to give algorithm questions to my candidates, but it matters more for those who will write databases and game engines than for those who will write CRUD apps.

collyw4• 13 years ago

I was good at the algorithms and data structures part of my university course, but as a developer with ten years experience, I find it is very rare to actually use these now.

Far more important is how / when to index an database table, how to design the tables in the first place.

If I need an efficient algorithm, in most cases someone has written (and tested) it and put it in a library.

Swannie• 13 years ago

Sure, it's valuable. But deeply?

If the person wrote the code in such a way that no one else could understand that it was a breath-first search, is that valuable? If they didn't even leave comments when they write code? If they didn't realise that you're using some slightly arcane feature of your language to keep a library of useful tree operations, and was expected to use the pre-existing function through macros/generics? Or was expected never to use such a technique because half the team wouldn't understand the code?

Hey, I know the theory on how to bake a large amount of cakes, breads, cookies, etc. I learned it by watching it on TV, and from cookery books. But actually it turns out that I'm a pretty crap baker, because I lack experience to know when I'm over-working/under-working dough, how to adjust cooking time with an unknown oven, etc. And I make a huge mess when I'm doing this, and take a lot longer than anyone with basically any experience of doing it at all.

eaxitect• 13 years ago

Totally agree, asking math puzzles (sometimes really hard ones) to develop a copycat iphone app? Interviewing like this is really off the rails.

I really understand that a startup with scarce resource would like to do its best shot. However as discussed long ago (https://news.ycombinator.com/item?id=2385424), it is really frustrating that asking math puzzles are assumed as the best way to hire the best for the job.

michaelpinto2• 13 years ago

After reading this I have a dumb question: The person behind the post is a CS major but only played a little bit with the C programming language in college — is this pretty common these days?

tylerkahn3• 13 years ago

He said he went to a small liberal arts school.

We definitely do C programming at my program (shell, malloc implementation, thread pool, web server, MIPS assembler/interpreter etc.). I go to Virginia Tech, a public university.

austinl2• 13 years ago

Current college student, probably have written 5,000+ lines of C directly related to class assignments

coreyja• 13 years ago

I go to an Engineering school, and there are only two classes that use C and are required for my CS major. The first one is the second programming class we take and is three quarters java and one quarter C. Pretty basic stuff. The second class is Operating Systems, and is C the whole quarter. I'm pretty sure those are the only two required classes that use C. And I didn't have to take the first one cause I tested out of it with AP credit, and I didn't do C in HS. So some people will only take one class in C when they graduate.

dagw6• 13 years ago

Certainly at the school I went to you had quite a lot of choice over what courses you took. I imagine it would be quite possible to get a CS degree without ever touching C, especially if you took either a more math/theory approach or a more UX/HCI approach to your degree. Of course it was equally possible to spend half your degree writing operating systems and programming micro-controllers in C and assembly if that was your cup of tea.

Basically CS is a huge field and any assumption you make about someones skills just because they have a "CS degree" is almost certainly false to a greater or lesser degree.

microtonal• 13 years ago

Yes, I have seen many CS graduates who couldn't write C or C++ if their life depended on it. In fact, I have seen CS graduates who could explain the theory behind, say, parallel computing in detail, but could barely write scripts.

Knowing everything about painting doesn't make you a good painter :).

dagw6• 13 years ago

Knowing everything about painting doesn't make you a good painter

Then again no one automatically expects an art history major to know how to paint (even though there no doubt is quite a bit of overlap).

tom_b• 13 years ago

Heck, in grad school I wrote C to hack data in memory layouts designed to minimize page faults. Lots of bit-twiddling nuttiness.

I don't think I could write a simple echo clone in C anymore.

I miss C, but since entering the "real world" I haven't written a line of C for work purposes.

michaelpinto2• 13 years ago

You've just shocked me! I went to art school and back in the day even if you studied say graphic design or fashion they actually taught you basic foundation skills which included painting and life drawing. Of course that was in the better schools, so maybe CS faces the same issues.

RogerL4• 13 years ago

Yes, I've interviewed plenty of people with degrees from very prestigious Universities that evidenced no real programming skills, and a wonderful (truly) grasp of algorithms. Trouble is, the former is necessary on a day to day basis, and the second is always available from some contractor when you need it.

pjmlp• 13 years ago

It depends on the quality of the university, I graduated in 1999, but from quick glance at the CS department web site, you still get to use at least C, C++, Prolog, ML, Java, C#, depending on the project assignments.

walshemj4• 13 years ago

CS is about the theory and not the practice plenty of mech engineers probably woudl not be able to use a lathe that well.

jsmeaton• 13 years ago

I started uni in 2002 and only did half a semester in C, and the other half on C++. I picked up a game dev elective for a semester, and did that in C++. Other than that, it was Java and PHP mostly, with smatterings of Lisp and Prolog for our AI classes.

dpiers2• 13 years ago

Hiring engineers is hard, and companies haven't really figured it out yet. Even the best companies rely on puzzles and gimmicks that often have little to do with day-to-day programming.

At one company I interviewed with, I was asked to implement a queue using two stacks. At that time in my programming career, I had worked with C, C++, Obj-C, Lua, Python, JavaScript, SQL, and a handful of DSLs developing games, game development tools, and web applications. Want to know what I had never done? Written a queue using two stacks. My immediate response to the question was, "Why would you want to do that?"

If you really want to know if someone has the capacity to pull their weight as an engineer, ask them about what they've built. Even if they are fresh out of college, the best engineers will have projects they can talk about and explain. Ask how they approached/solved specific problems. Ask what they're most proud of building. Ask what was most frustrating.

Those are the kind of questions that will provide insight into a person's problem solving capabilities and offer a decent picture of what they're capable of doing.

austinl2• 13 years ago

I definitely agree with a model that places emphasis on past work/projects.

"To find out if they can get stuff done, I just ask what they’ve done. If someone can actually get stuff done they should have done so by now. It’s hard to be a good programmer without some previous experience and these days anyone can get some experience by starting or contributing to a free software project." http://www.aaronsw.com/weblog/hiring

Techasura• 13 years ago

Best way to go is to ask them about their projects and hear them as to how they will explain them and how enthusiastic they are about building new things and ask them to probably build one in 2 days with the help of google. Then i think the quality would just come out naturally and now you can decide whether to hire this guy or not.

angrycoder• 13 years ago

Exactly. "Tell me about your last project" can easily turn into a 2 hour conversation with any developer who has built anything of substance. It will tell you everything you need to know about them.

hackinthebochs• 13 years ago

Except the all important: can they code? Your method may simply be filtering for good conversationalists and/or good bullshitters.

cynicalkane• 13 years ago

If you're wondering, the "two stacks queue" is an easy way to write a simple immutable queue for a functional programming language. You use one functional stack for enqueueing and one for dequeueing. When the dequeue stack is exhausted the enqueue stack is "flipped" into the dequeue. Amortized runtime is O(1) per operation.

dpiers2• 13 years ago

Once they told me to use one stack for enqueueing and one for dequeueing, I immediately solved the problem. What I didn't understand is why you would a.) implement your own queue structure and b.) implement it in such a non-intuitive way.

Now that I know this would be for a functional language, it makes way more sense. At the time I couldn't get over why you wouldn't just use a doubly-linked list like the standard implementation in most OO languages.

vasilipupkin• 13 years ago

The problem is, you often want to know what they can build for you in the future. And old projects where someone worked on something very specific for 5 years and exercised a small subset of a particular language or general algorithmic tool kit may not be predictive of their ability to work on YOUR project

tbassetto2• 13 years ago

Our current hiring process at my startup:

- After a first non-technical call, we ask the candidate to create a very small project based on our SDK. We send him the documentation and a very small sample. He can almost use every tools he wants to create that small project and, of course, we do not set any deadlines. It allows us to see how the candidate architecture his applications and it gives us a project to discuss during the following call. - If all goes well, we invite the candidate on site to present our code/project and eventually brainstorm together. So that both parties can see if they can work together and the candidate has an insight about how we work, how our code looks like.

Clearly, it's far from perfect and we are often considering changing it. Imagine if every company where you are applying would ask you to create an app from scratch with their SDK? We may lose some candidates, but at least we hire only people that fit the company's culture.

nappy-doo4• 13 years ago

I have experienced "interviews" like this from the other end. I told YC funded founders I'm not interested in their job if this is how they interview. In my opinion, it shows inability to make a decision.

I am an excellent developer, with years of experience in the industry. I know lots of technologies, and already have a great job. There is no reason for me to spend personal time writing your projects, when I would be rewarded by spending personal time on my employer's projects.

"Interviews" like this will only grab candidates with nothing better to do than to fulltime interview with your company. In my opinion, the best people already have jobs, and you're excluding them from the process.

tbassetto2• 13 years ago

I forgot to precise that if the candidate has an (significant) open source project we use it instead of sending the assignment.

I hope no company will make their decision just because a candidate says that he is an excellent developer ;)

epo2• 13 years ago

"I am an excellent developer, with years of experience", but seemingly unable or unwilling to complete a small practical test. Sounds to me like they are good at weeding out the wrong people.

sebkomianos• 13 years ago

If you have a great job there is no reason for you to spend personal time participating in any kind of interviewing process.

Also, who says "I am an excellent developer"?

biot• 13 years ago

Math puzzles are great if the problem is easily understood, the solution achievable without a math degree, and you ask them to solve it by writing code.

For example: "This database contains 100,000 problems with standardized parameters. The problem definition is defined in the file spec.txt which you can grab from our code repository. Write the code to solve these problems efficiently, passing each solution to a remote service via POSTing to a REST API, the documentation for which you can find here. Bonus points for parallel execution. Feel free to use any editor/IDE and reference online documentation, Stack Overflow, etc. that you want. If anything's not clear or you need a hand with something, just ask as you would if you were an employee already. Ready to get started?"

The great thing is that once you've identified a candidate, you can do remote screen sharing and have them write code before they even have to come into the office. I've interviewed a fair number of remote people this way and it's excellent for weeding out the people who can talk the talk but can't program worth a damn. And it limits bias because you don't care about much beyond their communication ability plus their technical ability.

x0054• 13 years ago

Here is an interesting idea that I had reading this. As a startup, what if you were to create a simple computer language that looked different from most other computer languages, at least somewhat different. Alternatively, just use one of the many really obscure programming languages out there, just make sure the applicant does not know it ahead of time. Give the applicant a 10-20 page reference manual for the language and ask them to make a simple program of some sort. Have them read the manual and write the program, hopefully while not looking over their shoulder, so they can relax. In the manual you give them omit one critical function or API reference, but make sure that info is available online (make it available if you made up the language). Then see what happens.

This would test programmers ability to learn a new language.

seanmcdirmid• 13 years ago

You would be testing them on the ability to learn a new language. You could test the same abilities using just a custom library. This is a test I used to use on intern candidates. I didn't even provide documentation, just examples of API usages, and was testing their ability to learn from example.

Tichy• 13 years ago

Hm, maybe if I ever have to hire somebody I'll challenge them to a Corewar duel :-)

Before the internet my friends and I sometimes used to get to together, everybody got some time to write a Corewar warrior on paper, and then we'd watch the tournament together. It was great fun.

collyw4• 13 years ago

It takes me a month or two to be comfortable with a new language.

How long do you expect your interview to last?

_ak• 13 years ago

I guess that proposed interview is supposed to get people out of their comfort zone and test whether how quickly they can adapt and learn from a manual and simple examples and then at least be able to write simple programs.

anuraj• 13 years ago

It is a good strategy, if the company is interviewing freshers as programming is teachable and the assumption is that new inductee will take few months to become productive. If you can't wait, the best strategy is to give a live coding problem and test the person's proficiency in the required language/technology. I invariably do the latter as my requirements are always very specific. Most start ups I suppose, are themselves undecided on product/market/technology choice and thus the former strategy.

ek• 13 years ago

Spoiler alert: to solve this problem, you need to know how to enumerate the rationals.

This problem was addressed nicely in this functional pearl by Jeremy Gibbons, et al.: http://www.cs.ox.ac.uk/jeremy.gibbons/publications/rationals... . As interesting as the result is, however, it's a pretty well-made point that research-level ideas from the programming languages community are not really software engineering interview material in the vast majority of cases.

This is yet another example of "rockstar developer"-itis, wherein startups are given to believe that they need the best of the best when in fact they do not. This particular example is entirely egregious because they asked her about something that requires enumerating the rationals when what they really wanted was an iOS code monkey. Then they fired her, based on their own shoddy interview.

AUmrysh• 13 years ago

The worst part of it is that it's easy to feel like it's your own fault when something like this happens. The real problem was the hiring manager misleading the applicant about what the job would be, but the OP clearly felt responsible for it.

I had a similar experience (working for family, at that), where I joined their R&D group and it went well for a while, but eventually they decided they wanted to get into selling solar power and didn't need a software/electrical engineer, so I was lectured about this or that bullshit and eventually brought into a meeting with the IT director and offered a position on his team. I realize now that they were trying to make me feel unwelcome, but I felt entirely responsible at the time. I never felt like I had a choice in taking the IT position (or lose my employment there entirely), and I hated it. The new boss was, for the most part, a nice guy, but he turned into a major nitpicking asshole over little things.

I'm at a new place now and it's still hard to kick the habit of trying to avoid coworkers because you know they don't like you and want you gone.

Culture has a huge impact on hiring and working. I wish I had been taught about culture when I was in school, I could have avoided the problems I faced had I realized the warning signs earlier.

yogo• 13 years ago

The need the best of the best but they are identifying the best the wrong way. Maybe they need the best CRUD developer, well if all you look at are candidates with strong research-level knowledge then they will not be the best at building CRUD apps. Sure we can all say that the research-level code is harder and naturally that makes that candidate smarter, but it's like trying to find the best doctor for a nursing position.

mariozivic2• 13 years ago

IMHO, the post is more about the interviewers not understanding what is important for success in the job they are interviewing for than about anything else. If you need a person that will have to switch technologies, languages and paradigms, you have to test for that, make sure a candidate has done it before or is capable of doing it in expected time with expected depth.

If one is good and quick in problem solving or has high GMA, that does indicate that he has the capacity to handle new and difficult things in general, but says nothing about the speed with which he can handle a particular new thing. Author's example with JavaScript is very good illustration how difficult can it be to learn a new paradigm for the first/second time.

progx3• 13 years ago

Because they don't know you, you don't have a well known name, they don't know what you can and if it is true.

E.g. if somebody hire John Carmack (ID Software), nobody will let him do some math test or ask him trivial programming questions.

But you are not John Carmack ;-)

It is like in every other job: if you are not a rockstar you are nobody.

progx3• 13 years ago

I know: the truth hurts.

keithgabryelski2• 13 years ago

There is a job at Google to ask legendary-coders coding questions.

It is reasonable.

josephlord• 13 years ago

Did you even read the article?

The author was not saying she was above simple coding tests but that she was an inexperienced programmer who was GOOD at the maths puzzles and got the job then struggled and the interviewer's assessment of her software development ability would have been better with some software related questions.

progx3• 13 years ago

Experienced or not, if you have a known name, nobody ask you silly questions. Thats all what i want to say.

We life in a world of casting shows, now we (the programmers) are in that shows too. They run only in companies.

lotsofcows• 13 years ago

I agree. But for a different reason: I'm shit at maths puzzles.

I just don't have the experience or tools or interest for them.

And yet, somehow, in 20 years of business geekery I've never come across a problem I can't solve.

Maybe when writing Tetris for J2ME I would have saved myself 10 minutes googling if I'd had the experience to realise that right angle based matrix translations don't require fp maths and maybe when writing financial indicators, I'd have saved myself half a day if I hadn't had to look up integrals but this sort of stuff is definitely in the minority as far as my experience goes.

yeukhon2• 13 years ago

I don't play Sudoku, I don't solve puzzles. I don't want to solve mathematical puzzles. I just want to deal with system and deal with security and none of them requires me to understand the tricks to solve a puzzle. Sure I can learn some cool algorithms but no thanks. I don't want to solve puzzles. Exactly.

mcphilip• 13 years ago

After much experimentation giving interviews for server side positions, I've come to favor questions that involve routine real world problems that can be handled in increasingly sophisticated ways.

One example I use is getting the candidate to write crud, list, and search controller actions for a simple category data structure. Given a basic category data model (e.g. Name, Parent), the candidate starts with the crud actions.

Crud actions aren't meant to be difficult to solve and serve as a basic screener to verify the candidate has working knowledge of the basics. The only edge case I look for the candidate to ask about is if orphaning child nodes is allowed (I.e updating parent node, deleting a node with children)

List action(s) start getting more interesting since recursion comes into play. A basic implementation of an action that can load the tree given an arbitrary category as a starting point is expected. If the candidate has some prior experience, a discussion of what performance concerns they may have with loading the category tree is a follow up question. The tree loading algorithm is then expected to be revised to handle an optional max depth parameter. An edge case I look to be considered is how to signify in the action response that a category has one or more child nodes that weren't loaded due to a depth restriction.

The search action implementation has a degree of difficulty scaled to the candidates experience level. All candidates have to write an action that returns a collection of categories matching a search string. Those with previous experience are asked about a paging solution. Senior level candidates are asked to return matching categories in a format that indicates all ancestors ( for instance: "Category 1 -> Category 1.1 -> Category 1.1.1" result for search string "1.1.1")

For an added degree of difficulty, candidates can be asked to recommend data model tweaks and algorithms supporting tree versioning requirements necessary to allow for loading the category tree's state at a given point in time.

The candidate's performance to this exercise seems to give some insight into their level of experience and ability to implement algorithms from a common real world example without having to ask much trivia or logic problems.

dionyziz• 13 years ago

OP would still blow you away by using the Nested set model: https://en.wikipedia.org/wiki/Nested_set_model

Although if you've been doing this interview for a while, you're bound to have come across someone who mentioned it, so I'm sure you're familiar.

arethuza• 13 years ago

And even better - knowing that quite a few database engines have extensions to help handling hierarchical data and it probably depends on the details as to which is actually better in a particular circumstance.

axc123• 13 years ago

I am not able to understand any of the questions because the jargon used is unfamiliar although the concepts look simple. Are all these terms specific to a particular framework ? Genuine question, thanks.

gridspy2• 13 years ago

No, from what I see they are not. CRUD relates to "Create, Remove, Update, Delete" which is the most common set of actions when dealing with database records.

Then the GP goes into detail about a tree-like structure of categories and their parents. If you're not familiar with trees and graph algorithms this might be quite daunting.

Do some googling on the terms I mentioned and please ask about specific things you do not understand.

codecrusade• 13 years ago

1. Most IQ tests are Bullshit 2. We all know what happened to the company famous for " Who moved mount fuji" 3. Math Puzzles are good if they are of the IMO level- but these things need a lot of concentration and joy to solve- Not under stress interview conditions. 4. Expecting someone to show brilliance by solving a math puzzle in under ten twenty minutes is a lot like a public willy wagging competition 5.Even more disgusting is the semi dumb questions at Mckinsey inerviews like - "Estimate the number of mineral water bottles in London" 6. 7.In 'Jobs', Walter Issacson says Steve was never into much of these puzzles- I can understand the reason. 8. ' I think a lot of what people call intelligence just boils down to curiosity'-(great quote from an inspirational friend-http://www.flickr.com/photos/elizabethbw/8373942339/) 9. People who ask these kind of puzzles end up creating a lot of CPU without any GPU. Very Little beauty. Very Little love. Disclosure- Im a member of Mensa Inernational. No Offense meant.

yeukhon2• 13 years ago

I just saw this on HN... http://www.datagenetics.com/blog/october12013/index.html

This is like solving your submarine problem. Jeese.

conductr• 13 years ago

I can relate on the opposite. I am not great at those complex math problems. But, I have been coding for 15 years at >20 hours a week average. Mostly web stuff. I've built dozens of full products, that we're complex, and I generally feel like I could build anything I wanted. Every time I use a new site I can visualize how I would have built it, usually not a question of if I could; time permitting.

Yet, I have never had the balls to pursue it professionally. I build stuff and usually never launch it. I have learned several times over that marketing is not my strong suit.

That said, I'd actually like to work for a startup. Hit me up if anyone wants to talk.

CmonDev• 13 years ago

Start-ups can afford asking candidates puzzles? I thought everyone was struggling to find developers.

seabee• 13 years ago

Struggling to find good developers maybe? (Or at least ones that get through their weird puzzle interview process)

10098• 13 years ago

Dear god, what kind of startup hires a person with only basic Java and Python knowledge, then hands them K&R and expects them to churn out production-quality code?! That's unfair.

14113• 13 years ago

The kind whose interviewing tactics don't hire staff with the skillset they need.

shindevijaykr• 13 years ago

really true

shurcooL• 13 years ago

Why not just look at the person's recent commits?

PhasmaFelis• 13 years ago

Because most paid coding work is not public, and many excellent commercial programmers prefer to do other things in their off time.

Honestly, this is a weirdly naive question. Gung-ho startup culture does not represent the industry as a whole, and you don't need to eat, breathe, and sweat code every waking moment of your life to be a great coder.

Beltiras• 13 years ago

Funny. Just made a hire and this story made me think of it.

The position I was filling is a part-time position for a CS major, sort of like an internship. I devote time to develop his/her skills, s/he would get real-world experience, and a little money to help with cost of living. If everything works out, a position could open up for full employment.

I had a pretty good idea what I was looking for. Someone that had good grasp on theory but had no experience coding. Preferably enrolled in Uni. I had 5 applicants but the only candidate I interviewed is enrolled in Math-CS.

I basically tried to gauge if he had deep interests and asked him to code a bit, solve a simple control (find me the article with the highest hitcount from the day a week ago, gave him 10 minutes).

He failed the coding test but I made the hire regardless. Reason why was 2 things out of the 4 hours we spent together: When I asked him who he considered the father of CS he rattled off von Neuman, Djikstra and Knuth. Yeah, you can make that argument I suppose, but he knew who the influential people were. The other thing was: even if he failed the coding test he failed it by not reading the code examples quite right, he was using my code to try to help himself solve the problem. I'm sure he'll work out.

We as a field should employ internships a lot more than we do, get the college kids and undergrads working on real-world problems a lot more than we do.

Tyrannosaurs• 13 years ago

On one hand I completely agree, on the other my experience of most CS graduates is that you can't code, at least not in the way that anyone codes in the real world so it's not a great thing to spend too much time on. That's not the fault of most graduates, it's what and how they're taught. Most people coming out of university know a little bit about a lot of languages and theories. That's good for giving them an overview but not great when it comes to having actual usable skills on day one.

Because of this I've pretty much given up on hiring graduates based on their technical skills so instead I'm looking for someone smart, who gets that they've got a lot to learn, who is interested in technology and can get on with the other people in the team.

I don't think asking people math questions per se is a great idea, but if you've studied a maths degree it's a good way of working out if you're smart and if you were paying any attention at all during university.

(Incidentally this may be different in other countries (I'm in the UK) or in a company where you're able to attract the very best who have picked up really solid skills, but for most organisations that's not the case as most graduates spent more of their own time in the bar than coding.)

sudomal• 13 years ago

I am willing to bet that tests give an advantage to applicants with no commercial experience, as well as those that have no life outside of technology. If that's what you want in employees then sure, it's a good way to find them, otherwise just look at their code samples and give them a trial.

Programming isn't difficult and you don't need to know complex maths or be able to solve mind bending puzzles to be a great developer.

Jugurtha• 13 years ago

The "Kevin Bacon" stuff was about degrees of separation (Does the expression "Six degrees of separation" ring a bell ?).

Not long ago, Facebook made that 4.74 degrees of separation on its networks. Meaning a maximum of only 4.74 persons are necessary to connect any two random persons on the network.

https://www.facebook.com/notes/facebook-data-team/anatomy-of...

You can also find an article on Wikipedia about the "Kevin Bacon" reference.

tommorris4• 13 years ago

Here's the test I've used in the past:

Before the interview, I ask them to write some code to access an HTTP endpoint that contains exchange rate data (USD, EUR, GBP, JPY etc.) in XML and to parse and load said data into a relational database. Then to build a very simple HTML form based front-end that lets you input a currency and convert it into another currency.

I ask them to send me either a link to a repository (Git, SVN etc.) or a zipball/tarball. If the job specifies a particular language, then I obviously expect it to be in that language. If not, so long as it isn't in something crazy like Brainfuck, they have free range.

If the code works and is basically sane, that goes a long way to get them shortlisted.

During the interview, I'll pull the code they sent up on a projector and ask them to self-review it. If they can figure out things that need improving in their code, that weighs heavily in their favour. Usually this is things like comments/documentation, tests, improving the structure or reusability. If it's really good, I'll throw a hypothetical idea for refactoring at them and see how they think.

The reason this works is that, despite Hacker News/Paul Graham dogma to the contrary, "smartness" isn't the only thing that matters in programmers. It's actually fairly low down the list. When hiring programmers, I want people who are actually able to do the daily practical job of writing code, modest and self-critical enough to spot their own mistakes, and socially capable to actually communicate their decisions and mistakes to the people they work with.

I interviewed a guy who was intellectually very smart and understood a lot about CS theory. I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code.

sillysaurus2• 13 years ago

despite Hacker News/Paul Graham dogma to the contrary, "smartness" isn't the only thing that matters in programmers. It's actually fairly low down the list. [...] I interviewed a guy who was intellectually very smart and understood a lot about CS theory. I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code.

That isn't what Paul means by "smart." Worrying about trivial inefficiencies like that is a sign of an ineffective coder. That's the opposite of a great hacker.

tommorris4• 13 years ago

So, smart includes some kind of experience-derived common sense that isn't measured well by IQ tests...? Perhaps someone should tell the people who want to use IQ tests to measure the quality of programmers...

wslh• 13 years ago

"smartness" isn't the only thing that matters in programmers.

Smartness is overrated but experience is critical for speeding up work. I can implement a whole site in Django but without good experience I will be slow (even being good at Python)

Nowadays, I think we need to find new ways to evaluate candidates productivity. For example, just following a Q&A method (stackoverflow) I can be faster than other developers.

reginaldjcooper• 13 years ago

Sounds like a good interview; you're giving them a chance to have a go at something realistic using their typical toolset instead of a whiteboard.

I don't understand your point about dogma. I think HN assumes rationality as implied by smartness. Nobody wants to hire a poor coder/communicator to write code, CS degree or no. I bet most readers would find your process to be pretty good and have similar processes of their own.

mathattack2• 13 years ago

What I like about your method is it also allows you to catch folks who fudge the work product.

eru• 13 years ago

And let's people through who fudge the work product, but can explain it. Which is a good skill.

eddiedunn• 13 years ago

Hey, that seems like an awesome hiring process.

Personally, I think that if you want a brilliant mathematician/problem solver, then that's what you should put in the job listing. Then asking the kind of questions exemplified in the article is likely merited. On the other hand, if you want a good programmer you should mainly look at the quality of the code they produce.

denzil_correa• 13 years ago

I concur. Even in software engineering job positions a lot of times people advertise for X but end up asking Y and judging according to Y.

honzzz• 13 years ago

Depends on your definition of "smartness". In my book the ability to assess the importance of things and their relevance to given task and ability to prioritize accordingly is very important aspect of being smart. The guy who 'does not like comments because they slow the PHP interpreter down' does not seem to be that smart.

walshemj4• 13 years ago

And the mundane task of coding is probably not the most important part of any developers job. Understanding fully what the spec/task requires including covering the use cases the designer forgot to mention.

TazeTSchnitzel• 13 years ago

"I don't believe in comments because they slow the PHP interpreter down."

For some bizzare reason, a lot of PHP devs obsess over mini benchmarks and tiny speed issues while ignoring massive latency factors (like the DB).

I find it amusing, though, as PHP hasn't been a direct interpreter for years. It compiles to an internal opcode array and interprets that. While its opcode compiler is silly (it doesn't build an AST and directly emits opcodes, which causes all sorts of things not to work), it does have a proper lexer and I think comments are completely ignored aside from doc comments[1] which can be obtained in userland with reflection.

[1] They look like this:

  /** DocComment */
  /* Normal comment */

jaredmcateer• 13 years ago

For some bizzare reason, a lot of P̶H̶P̶ devs obsess over mini benchmarks and tiny speed issues while ignoring massive latency factors (like the DB).

This sort of behaviour isn't unique to PHP Devs, not even close.

theboss• 13 years ago

I really like this. I'm finishing graduate school and I can't tell you how disinterested I am in a company when I apply or send an email and they want to ask me math problems or puzzles....And sadly, more ask puzzles than don't

You wanted a ______ and you are interviewing for a puzzle-solving mathematician really doesn't sound right.

Good stuff.

bdg• 13 years ago

I love your interview style: actually test for things the dev will actually do.

I've applied to other jobs who's owners frequent HN, and they did things like test my ability to write sort algos or data structures... but the job called for a good knowledge of JavaScript, Backbone and general knowledge of django to build a mix between a CMS and an elaborate voting system.

They didn't hire me. I'm utterly trash at working on algos under pressure. I felt like a lost child just trying to write a simple sort during the interview. Worse, I made it harder on myself by asking to not write the code in MSWord (or some other similar thing) and instead use jsfiddle. The owner looked at the code and said 'Yeah, that looks about right to me, it should work.' I foolishly hit run.

What I learnt was two-fold:

1) this doesn't test my ability to do the thing you want to pay me to do (as proof, I'm writing more complex applications today at a job I enjoy far more than I would have at that company).

2) The employer can't differentiate between a "smart developer" or "average developer" until the script executes.

lportion• 13 years ago

I've had similar experiences.

I went for a job a few months ago where I had to write a program on paper to produce report data from raw data collections, it was reasonably complicated. This is the type of thing I breeze through when I'm sat alone at my desk, but for whatever reason I just went into panic mode and couldn't think straight at all. I apologised and told them just that. It was frustrating to say the least.

The interview went ahead and we ended up talking about various architectures, design patterns, programming languages, functional programming and even HN, but I'm sure I never got the gig because of the technical test.

For the interview at my current gig they walked out of the room for 10 minutes while I had to write sorting algorithms and I was able to do it without a problem. So, for me at least, the element of being watched while I wrote the code seemed to be part of the problem.

alinajaf• 13 years ago

I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code.

This one wasn't smarter than Einstein. It's been a while since I've been near a live PHP installation, but IIRC running with an opcode cache like APC probably means the comments will be parsed at most once (every server restart?)

at-fates-hands2• 13 years ago

This true - but without the opcode cache the slowdown would be minimal since all the comments are ignored.

Plus, the comments are there for future developers to use as a reference. Nothing worse than having something break, cracking open a file and have to spend hours having to untie someone else's code.

I'd rather have copious amounts of comments, then a minimal speed gain due to not having them at all.

brryant• 13 years ago

I've also interviewed lots of intellectual "CS theory" types. PhDs, etc. In most cases, they have not worked in real software engineering environments that necessitate comments for simple HTML forms. Just IMHO, I don't think it's a huge knock, as some of these candidates can turn into game changers for your company.

ulyssesgrant• 13 years ago

No bullshit and to the point, this is my ideal interview

wpietri4• 13 years ago

I'm with you on this. I always do pair programming interviews when I'm hiring. People may be able to bullshit me when I'm asking questions about their resume, but they cannot fool me, the compiler, and the unit tests when we're coding together.

Smartness is great, I like smart. But I agree that it's probably the fifth- or sixth-highest-weight factor in my list.

matwood• 13 years ago

We interview people this way where I currently work. I'm always surprised at the people who send stuff in that flat out doesn't work. There is no time limit, and the task isn't very hard. I actually consider it a type of fizzbuzz.

vxNsr• 13 years ago

Are you a prof at UIC (or alumni)? I saw a job posting that basically described the first part of this, which made me smile because it actually made sense, instead of abstract nonsense about goats crossing bridges...

tommorris4• 13 years ago

No, I'm a freelancer in London. The hiring process was for an organisation I volunteer for who wanted to hire a coder. I didn't want to work there, but I volunteered to help out with interviews and CV screening.

davidkassa• 13 years ago

This works really well for the candidate too. They walk into an interview with a known quantity and can "own" the conversation by discussing the reasons that they made the decisions that they did. It ends up being a very context-specific portfolio.

gregjor• 13 years ago

This is sadly common in a community keen on logic, evidence, and avoiding fallacies in thinking. Worse than puzzles are pointless faux psychological screening questions like "Tell me about something painful that has happened to you and how you dealt with it."

I would (and have) asked if the interviewer or organization has any evidence to show that interview puzzle performance (or shit like Myers-Brigg) predicts job performance. No? Not surprising. Google did look into it and found no relationship. (http://www.businessinsider.com/how-google-hires-2013-6)

Programmer interviews are so crazy and sometimes sadistic that I catalogued some of the more common interview patterns:

http://typicalprogrammer.com/thirteen-patterns-of-programmer...

theanirudh• 13 years ago

Even after reading Jeff Atwood's post[1], it still amazes me how many programmers fail the fizz buzz test. We dont even get the chance to ask tough programming qustions. Simple questions like fizz buzz, loops and recursion were good enough to filter out a lot of applicants.

[1] http://www.codinghorror.com/blog/2007/02/why-cant-programmer...

hmsimha• 13 years ago

Is this really the case? I read this a lot, but have a hard time imagining that there exists a wealth of people who have such a hard time with such a simple task, and yet I am having a hard time finding an entry-level job when they are (presumably a substantial part of) my competition.

GFischer• 13 years ago

It is. There are a lot of really bad programmers out there.

However, you're probably not looking at the places where such programmers would apply... try an entry-level job here in Uruguay (pay: about U$ 800 / month after taxes), you'll get a lot of people that fail fizz-buzz .

The original poster (Imran Ghory) was from the UK, and Reginald is from Canada. Neither are Silicon Valley.

http://imranontech.com/2007/01/24/using-fizzbuzz-to-find-dev...

http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz....

nnoitra• 13 years ago

I don't believe this.

taude• 13 years ago

While we're at it, can we stop treating and thinking of web development (which seems to be a lot of dev positions these days) like it's rocket science?

trendspotter• 13 years ago

tl;dr

Stop asking this fine young lady math puzzles to determine her programming abilities. She is good at solving your seemingly pointless math puzzle, because she was practicing problem-solving since she was ten. But she is not anywhere near as good at programming, yet - which caused her problems at the actual jobs she had to do after she was hired.

fayyazkl• 13 years ago

Finally some one pointed out the importance of the ability to actually code and produce something that works. Algorithmic problem solving ability is far less utilized in actual every day job compared to being able to code. Just imagine how much of your math skills did you actually need going well through all those bad experiences? Would you still be considered slow learner and fired if you knew how to code pretty well but just wasn’t so good at figuring out shortest path in a graph. Isn’t it possible to know the CS basics well i.e. familiar with complexity, big Os, basic data structures and sorting and being able to learn any advanced standard algo when needed by looking it up? Just wondering….

enterx• 13 years ago

You speak wise, my friend.

Isn't XY years of records in the same field of interest working for a successful companies a good sign that I can code?!

Ask me theory - pay me to code.

sokoloff• 13 years ago

Sadly, it's not. I can't tell you the number of 5-8 years' experienced candidates from respectable companies who couldn't solve very easy interview problems.

Yes, I get that a whiteboard isn't an editor, and you don't have google/SO, and some people just freeze in an interview, but I've long ago stopped assuming people can code just because they've got coding on their resume for successful companies. (For all I know at the interview moment, they didn't have a coding role, or may not have even worked there.)

joeblau• 13 years ago

I would say that you should keep at it. There are strong parallels between math and programming, but interviewers should definitely be asking you to write pseudo-code on a whiteboard and do a paired programming session. That would probably be a good way to relieve the awkwardness later when they realize that your programming skills aren't as strong as you'd like them to be. Definitely keep at it, soon you'll be able to think of a Markov chain as a for loop multiplying two arrays and not only as a matrix multiplication.

cicatriz• 13 years ago

Here's a writeup about a recent study that showed interviewers couldn't predict GPA any better when the interviewees answers were accurate versus random: http://www.danielwillingham.com/1/post/2013/10/why-job-inter...

Anyone who supports math puzzles (or whatever else) in an interview would have to argue that their perception of the candidates performance offers a clear enough data point that it doesn't dilute other information available to them. Given Google's study finding data otherwise, they certainly have the burden of proof.

tokenadult9• 13 years ago

There are many discussions here on HN about company hiring procedures. Company hiring procedures and their effectiveness is a heavily researched topic, but most hiring managers and most job applicants haven't looked up much of the research. After reading the blog post kindly submitted here and some of its comments, and then reading most of the comments here on HN that came in while I was asleep in my time zone, it looks like it's time to recycle some electrons from a FAQ I'm building about company hiring procedures.

The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,"[1] Psychological Bulletin, Vol. 124, No. 2, 262-274 sums up, current to 1998, a meta-analysis of much of the huge peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.[2]

EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, with its legal rules about hiring, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.

The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.

Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case[3] interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. A company defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)

The social background to the legal environment in the United States is explained in various books about hiring procedures,[4] and some of the social background appears to be changing in the most recent few decades, with the prospect for further changes.[5]

Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures"[6] (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:

"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."

But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."

[1]

http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

[2]

http://www.siop.org/workplace/employment%20testing/testtypes...

[3]

http://scholar.google.com/scholar_case?case=8655598674229196...

[4]

http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6TEC

[5]

http://intl-pss.sagepub.com/content/17/10/913.full

http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...

[6]

http://geb.uni-giessen.de/geb/volltexte/2012/8532/pdf/prepri...

Alex3917• 13 years ago

I really don't see how this is relevant to this post, since the research you're citing is about hiring people for routinized or semi-routinized labor, like at a car manufacturing plant.

C.f. https://news.ycombinator.com/item?id=4614430

tokenadult9• 13 years ago

Your question about relevance is fair. The author of the blog post kindly submitted here complains that she [I wrote "he" before edit prompted by kind reply below] gets math quiz questions when applying for programming jobs. That's a mistake on the part of the hiring companies in two ways.

1) The main thing to look for when hiring programmers is programming work samples. In general, for any kind of work, a work-sample test will tell you more relevant information about which person to hire than any other kind of hiring procedure. (Some other kinds of hiring procedures have incremental validity in addition to work-sample tests, but none have higher validity than work-sample tests.)

2) Insofar as a math quiz question purports to figure out which applicants are smart and which are not, it is legally risky, as are IQ tests and AS ARE EDUCATION CREDENTIAL REQUIREMENTS if the quiz question is not validated as related to bona-fide job requirements. The company that publishes the Wonderlic personnel test hires a lot of psychometricians and lawyers to be able to defend the use of that test for hiring for dozens of kinds of jobs, and I think the balance of the evidence across hundreds of studies on many different job categories in multiple countries shows that general mental ability matters for performing most kinds of jobs well. But in the United States, a smartness test that a hiring manager just makes up is legally risky. It's a good idea to hire smarter rather than dumber job applicants, but it's an especially good idea (for legal protection in the United States) to make sure that the smartness test you use has been validated for your workplace.

The research I'm referring to is more generalizable across more occupations than you suggest. In other words, the usual relevance of my FAQ about company hiring procedures in the frequent HN threads about company hiring procedures is that companies almost always go wrong by 1) NOT using work-sample tests, a very good way to find competent workers, and 2) often go wrong by posing trick question tests, presumably to find smart people, without validating those tests and ensuring that those tests don't have disparate impact on applicants. Companies can and should do better, and research tells them how. The people who often complain on HN about company hiring procedures are largely correct, but usually just share anecdotes about what they would do differently if they were hiring, or what they would like hiring managers to do the next time they apply for a job. Research on this topic is abundant, and it applies to people seeking work as programmers as well as to most other occupations.

ap222132• 13 years ago

Could someone explain what 'mean validity' means in [1]?

Also, how good of a GMA score is needed? That is, how good of a score would be needed for me to stop looking at other candidates?

tokenadult9• 13 years ago

Let me give you a short answer as a place-holder for a possible edit into a longer answer.

1) "Validity" here means hiring of better rather than worse workers, as all workers can be rank-ordered as to work performance, as compared to doing nothing to select workers.

2) Usually in hiring, it's enough to outrank someone else in a hiring criterion, for example general mental ability, without necessarily reaching any particular score level. The general idea is to rank higher than the other applicants when applying for a job, and to choose higher-ranked rather than lower-ranked applicants when hiring other people.

I'll see if I have time to add specifics later after some activities with my family.

ElDiablo666• 13 years ago

You really shouldn't recommend a "mental acuity" test since those are complete and utter nonsense. I think it's a better screening criteria in the sense that you wouldn't want anyone working for you who believes that IQ tests are worthwhile.

tokenadult9• 13 years ago

I would be happy to see any research evidence you have found to back up your opinion about mental ability (NOT "acuity") tests. This is an issue that is very heavily researched in multiple countries with varied cultural backgrounds across hundreds of kinds of occupations in almost all kinds of workplaces.

mattjaynes4• 13 years ago

I can confirm this works in my own experience of trial-and-error interviewing over 2000 applicants and hiring over 200 for my clients and myself (since ~2006).

Nearly 100% of the applicants were remote, so I think that helped me from falling into traps of poor "traditional" hiring practices.

The point of hiring these remote folks was to help accelerate whatever team I was on. It can be a great way to scale your existing team very quickly if you know how to do it.

For example, I took over an iPhone app dev team and it was taking them 4 months to produce an app. These apps were all very similar in functionality, but the developers were spending a ton of time slicing images, testing, and other tasks that remote workers could easily do. So I hired some remote staff (via oDesk) to do most of that supporting work and we got the app production time down consistently to 1 month. That was a huge ROI for the business since the total cost for all remote staff was the same as for 1 additional on-site engineer.

There's nothing magic about hiring well, but I've watched others try to hire remote staff and the vast majority of them try once, fail, and give up on it.

They will approach the hiring process in a traditional way (personally interview them to watch how they handle puzzles, etc). It's a grueling process and then they still get really poor hires and conclude that "outsourcing doesn't work".

It's most helpful to think of the process as panning for gold. (Naturally, I'm not saying that some people are more valuable than others innately, just that you're looking for those who are most valuable at performing your given tasks.)

So, to find gold, you must filter, filter, filter. That's the exact process for finding applicants that are high performers. Most of your applicants will be pretty terrible at the job you're hiring for, so the filtering process is critical for success.

  - filter out the very worst applicants with a small easy question
  - filter out the remaining applicants:
     - pick a real-life production task you've recently completed
     - ensure that the task is *exactly* what they'd be doing in the job
     - have them perform the task
     - compare their task results to your task results
  - hire more than you need of the top performers
  - filter out (gently fire) the ones that aren't as good
  - repeat as needed until you have gold

When I see others attempt this, the most common problem is that they essentially go down to the river and just grab whatever pebbles they see in their first handful and hope there's gold in it (hire without filtering). Or they go down and carefully pick the prettiest pebbles hoping they will be gold (wrong filter / puzzle interviewing). But the only way to really find gold is to seriously invest in a filtering process that will yield actual gold. That means filtering based on their ability to do the actual tasks they'll be doing on the job.

The great thing about hiring remote folks is that I care 0% how they get the task done. I don't care if they've automated it, or have their mom do it for them, or whatever. If they provide the results I need, I'm happy, period.

There are plenty of other smaller caveats and gotchas to watch out for, but I'll try to cover those in a blog post sometime.

If you're a startup and want to go faster, try this out by off-loading some of the grunt work from your staff. It can be a big competitive advantage if you can do it right.

sdoowpilihp• 13 years ago

You used the term "gently fire". I was wondering if you could elaborate on how you "gently fire".

NAFV_P• 13 years ago

"filter out (gently fire) the ones that aren't as good." If the employee finds out you "hire more than you need of the top performers", you end up with a potential court case. If I found this out about a former employer, I would have pursued them to the end of hell.

nether• 13 years ago

- filter out (gently fire) the ones that aren't as good

God I would hate this. I'd rather be told that my work sucks, and shown examples of better work, so that I could actually improve and not just wonder if I was let go for some arbitrary reason. If I disagree with you BFD, life goes on, but I'd at least like to know if it was related to my output or not.

timedoctor• 13 years ago

I've used a very similar method for myself in hiring remote workers, although my ratios are more extreme. In some cases we tested over 200 people and selected one person for the job.

Have not used math questions at this stage as it seems a bit too obtuse and not to-the-point. Better to ask an actual coding question if you are hiring a coder. However I have had success in hiring people that have been involved in maths olympiads or also in coding competitions.

pjungwir• 13 years ago

After reading the OP I immediately thought of your standard hiring advice and wondered if the folks asking math questions aren't on the right track after all. (Thank you for your persistence posting it, btw.) At least half of my daily work requires learning new things on-the-fly, and I bet in 5 years most of my technical knowledge is turned over anyway--except the math and CS fundamentals. In programming aptitude matters way more than current knowledge. I think IBM discovered the same thing in the 60s when they were hiring chess players and crossworders. So if math puzzles serve as a legal(ish) proxy for an IQ test, aren't they just what interviewers should ask? Of course I'd rather have someone who knows Ajax and 3NF if I can get it, and not lose any time to up-front training, but that may not be realistic, and those things can be learned if the person is smart. Particularly for an entry-level hire, I'd rather hire someone who can solve graph theory problems than knows the command-line options for git.

lmkg• 13 years ago

Math questions are probably an okay substitute for "general IQ" testing for the majority of population, moreso than most other question categories (math is literally just logical reasoning). But, it's not a truly general skill, it's a specialized discipline--and that means the practice of using it as a screen is going to fail hard when you try to apply it to someone with specialized training. Which is what happened in the article.

pmiller2• 13 years ago

Are you hiring now? ;) Contact me if so.

bradleyjg2• 13 years ago

It seems awfully rude to cut and paste this gigantic comment, which is now virtually pinned to the top of a discussion thread of a blog post to which it barely responds.

There's nothing in this that indicates it is at all customized for the article it is attached to. Especially given that you and the author seem to have some things in common (connection to small-town Minnesota & love of mathematics) it seems like you could have done a lot better than just re-posting virtually the same comment for the 11th time.

You have to go about half way down the page to find a comment that actually engages with the article. The way the HN system works out in practice the "winning" top level comment determines the entire structure of the ensuing discussion. I think that means that top level commentators have a particular responsibility to comment well.

tokenadult9• 13 years ago

The currency of upvotes suggests that many readers have appreciated my comment, perhaps because it provided them new information about an important set of issues. That set of issues has been discussed in many of the comments here, and was discussed in the original article.

The currency of actual cash money, which is not provided in our information-sharing environment here on Hacker News, would free up time for me to customize each comment I post more thoroughly. Under current conditions on Hacker News, I will continue to post for free, and I hope some of my comments are worth more than I bill for them.

Thank you for letting me know your opinion.

andrewflnr• 13 years ago

Where is the line between a coding test and a "work-sample" test? Does a coding test become a work-sample test just by virtue of resembling a task you would do on the job? Is it a function of scale of the task?

jtheory• 13 years ago

A coding test in which they're writing code you could actually use in your business is a work sample test.

If it's more than a 10-minute job, you should be paying for them for the work.

gdy• 13 years ago

Because of a Supreme Court decision in the United States (the decision does not apply in other countries...) That's funny that you think this clarification is needed J (No offence intended)

VLM2• 13 years ago

Something I've always wanted to ask, are contractors hired the same way? I've never contracted although my father did in his retirement years. I'm curious if modern contractors have to put up with this kind of behavior at interviews, or if its a more professional atmosphere oriented around the actual job requirements.

lucasnemeth• 13 years ago

I believe there is some kind of inferiority complex, we don't believe software engineering is actually worth it. Probably, it is the result of an academic mindset that is taught at colleges, where the applied fields are seen as less important than the "pure" ones. But good software engineering, that is, writing complex systems, with a lot of requirements, maintainable, scalable, nice APIs, etc. it's very, very hard. And we know it! If we applied our hiring methods to writers, we would be asking them to improvise a rap rhyme, when we wanted to hire a novelist.

mrcactu52• 13 years ago

It looks like Emma's math prowess is working against her. It's ironic the app developers - who need her help the most - are pushing her away.

OK, so there is a difference between computer science and programming. that's why there are two different stack-exchanges:

  cs.stackexchange.com
  stackoverflow.com

And we can make even finer distinctions if we wanted to.

it's actually really fucking INCREDIBLE that

* you can know tons of CS without being able to build a decent app * you can a decent facebook clone without having any idea how it works

I feel really bad for Emma. I was a math major, but app developers won't even look at me b/c I'm not a full-stack whatever. So now I'm a Data Scientist at an advertising firm in Puerto Rico.

fnbr2• 13 years ago

Hey, I'm a math major looking for jobs/internships in data science. Any chance I could ask you a few questions about your experience finding work?

mrcactu52• 13 years ago

it was a nightmare due to most programmers' ironic contempt for computer science, let alone mathematics. they consider it a waste of money.

I moved to another country (Puerto Rico) have 2 excellent data scientist jobs and have enjoyed every day since.

keithgabryelski2• 13 years ago

My observation is that a lot of interviews come down to "stump the chump" questions; a question that is meant to show a single issue the interviewee has under their belt and is used to gauge the entirety of the interviewer's ability. Math puzzles/logic puzzles are in the same category: they require domain knowledge that probably doesn't translate to any job I've ever worked on.

That aside, one must have a way to measure the abilities of a candidate -- and asking the same set of questions to many people allows you to compare the answers as apples to apples.

I generally don't restrict my people from asking any particular question, but I will ask them to consider what a failed answer really means for the specific job (questions are generally adjusted then).

As an aside, some questions of mine that aren't specifically about coding:

* do you code outside of work (a love of coding translates to good coders)

* send me a link to some code you've written that you are proud of (let see what you got)

* tell me about a problem you had where your solution wasn't correct (how have you dealt with failure).

meshko• 13 years ago

I don't get it. He got hired? He learned how to do his stuff? If we require people to know how to work right out of college, no fresh grad would ever got a job.

rexreed• 13 years ago

If you're running a startup, the most important thing to hire for is fit. Do they fit in your culture? Do they fit a need that will help you achieve your milestones? Do they fit in the overall growth trajectory of your company? Do they have competency in the specific area you are hiring for and/or where your startup is building overall competency? Can they manage themselves and their time well?

The likelihood of failure of a startup approaches 100%, so you should optimize for likelihood of survival, not for IQ.

If you're not a startup, then the top ranked comment applies. But it doesn't really otherwise.

Thrymr• 13 years ago

Except that what a lot of startups mean by "fit" is "they think just like me" or even, "they have a similar background to the rest of the team." Sure, that may make them easier for you to talk to or hang out with, but it reduces the diversity of the team, and increases the likelihood of groupthink. The other things you mention (after "fit in your culture") are not what most people mean by "fit". "Fit" is an intangible that can be used by screeners to keep from going out of their comfort zone.

DigitalSea• 13 years ago

I failed mathematics in school, for the life of me I can't grasp them beyond the basics, but give me laptop and a copy of Sublime and I'll code anything you want. I can code, but I would fail any mathematical test given to me. This kind of approach has always bothered me, there are a lot of good developers out there bad at maths but posses strong problem-solving and highly analytical skills.

Being a developer is 80% Google and 20% actual coding knowledge. We are hackers at the end of the day, not miniature Einstein's with encyclopaedias for brains.

gboudrias• 13 years ago

I've always felt like I was supposed to like maths, but I learned (basic) programming before I learned algebra, and I couldn't figure out how I would be using it.

After years of being a programmer, I still can't.

onezeno• 13 years ago

Actually...

“Never memorize something that you can look up.”

― Albert Einstein

rehack• 13 years ago

This is a great post. And also from the other side of the fence. As typically we see, these kind of posts, from people who did not like Math puzzles, and as a result suffered in the interview rounds.

But this one talks about getting inadvertent benefit of being good in Maths to get selected for programming, and suffering the consequences later on.

Also, it highlights the importance of what is mostly taken for granted and thought of as mundane stuff, of programming - the idiosyncrasies, jargon, and best practices of various languages and OS environs.

jroseattle• 13 years ago

I'm dealing with this now, having been interviewing for different engineering roles over the past two months. It hasn't been as bad as straight-up conceptual math problems, but there have been plenty of questions that I have questioned for validity.

Interviewer: "How can we optimize the character replacement in a string such that we use no extra memory?" Me: "We do this and that and this. But, should we consider what situations we would need this optimization?" Interviewer: "What? Why?"

I can now use this as a filter as I interview organizations. Optimizing algorithms by creating your own core data structure classes (instead of using the built-in ones) is great in certain circumstances, but an absolute waste of time in many others. And if you're not going to ask me about those times when making those improvements is important, then you're not asking questions for a programmer -- you're asking them for a theoretician who can recall syntax.

It's poor practice, and I've seen it everywhere.

mindwork• 13 years ago

I am stopping to talk with people who ask for such a bs.

Check out the last technical interview task that I got ``` Objective: Write a program that prints out a multiplication table of the first 10 prime numbers. The program must run from the command line and print to screen one table.

Notes: - DO NOT use a library method for Prime (write your own) - Use Tests. TDD/BDD - IMPRESS US. ```

I mean I can impress you but how will this correlate with production code?

fnbr2• 13 years ago

I found that Facebook was really bad for this. I'm a math undergrad, and I applied for a bunch of data analysis positions with them.

I was asked, as part of my application, to take a programming quiz. The quiz consisted of a graph theory problem. I did pretty poorly on it, given that I have no real knowledge of graph theory.

Had they asked me a question about statistics (or something similarly related to data analysis), I think I would have actually been able to answer, or at least been at a point where my programming knowledge- not my math knowledge- was what was holding me back.

jpgvm• 13 years ago

When I hire programmers I try to favour ingenuity, knowledge and as best as I can gauge it, work ethic. For instance I might ask them about a theoretical task, possibly something like a scheduler or packet filter etc and give them domain specific data about how it will be used and ask them if there are any optimization they could make if they had this data about the systems use case.

Or I might ask them to describe how an event loop works.. or what the I/O path between their program and the disk looks like in as much detail as they can.

Someone that loves the field is going to have a decent idea about these things even if they never had to build one before.

caveat: these examples are very system level but you can substitute them with appropriate web, financial etc domain specific knowledge.

Killswitch• 13 years ago

I'm absolutely horrible at math... I think I graduated high school (my only schooling) with the equivalent of just above grade school in math... I can code no problem though. I'm very good at it. Anybody asks me math puzzles I say thanks for your time, but I am done with the interview.

jsun• 13 years ago

I think a lot of companies use brain teasers or math problems to test general mental aptitude, whether it works or not is under serious debate, but in my opinion that's the right thing to test for (if it's even possible).

The reason is smart people can figure out git, or databases, or objective-c, or whatever, in a fairly short amount of time.

For example, my co-founder learned objective C off free online video tutorials and built an iOS app (talking an app with serious firepower and back-end transaction logic) from start to end by himself in less than 3 weeks.

That's why we're not as concerned about what you know right now as what it's possible for you to learn in 3 more week.

swelly127• 13 years ago

I have the exact same problem as OP. Getting tons of job offers because I've been doing competitive math and algorithms since grade school but really have hard time understand technology. I'm pretty ambitious and I want to join a small, high growth startup and have the excitement of being part of a founding team but I'm afraid of letting people down.

I could learn heroku/RoR/whatever other technology but news things are always coming out and some people keep up with it so easily. I'm not sure being a dev is right for me if I take so long to understand such basic stuff. But I love coding and algos! I write python scripts to do all my homework... and then run them in codecademy labs because doing it in unix makes me so confused.

If anyone has had the same problem please let me know how you got over this hurdle. Thanks.

background; sophomore, cs major, cornell

tylerkahn3• 13 years ago

You say that unix is confusing. Unix isn't confusing. It's just unfamiliar to you as it was to me at one point. The way you get better is that you push through the unfamiliarity and keep going until you solve the problem you're facing. Sometimes it's painfully slow to acquire the knowledge you need, but that's just part of the process.

All that knowledge accumulates over time and you build skills about how to learn new things.

That's what allows people to pick up new technologies easily. Hard work, previous experience with similar technologies, and some intelligence (which you obviously have already).

I would recommend that you install Ubuntu in a virtual machine (VirtualBox is a good choice) and see if you can get some Python programs running. That process alone will teach you a lot.

For what it's worth, RoR and the Heroku stack are not basic and you shouldn't feel bad for not picking the up quickly. There's so much context that you need to have to understand what's going on and to use those technologies effectively. I bet if you polled your classmates the majority of them haven't even heard of Heroku.

Xyik• 13 years ago

In my experience, only the really big companies focus heavily on algorithms and math puzzles. That's because they don't really need to hire anymore people, they just want to steal 'smart' people, and they don't need to iterate as quicky. Start-ups and smaller companies have in my experience, typically asked full-stack type of questions that dive into things like networking protocols, databases, scalability, and so on. And I believe thats the way it should be. Start-ups that focus heavily on math puzzles and algorithms are doing it wrong.

mcgwiz• 13 years ago

Dear poster, don't imply all startups are equal.

If a startup asks you to solve math puzzles, it's possible that the work you will be doing heavily involves the creative use of math or information analysis. (This is more broadly valuable than many people recognize.)

Also, it's also possible that that particular startup doesn't know how to effectively interview.

It doesn't sensationalisticly mean all Startups (capitalization yours) don't know how to effectively interview.

Also, rather than focus on your ability to learn, I would humbly recommend you reconsider the basic nature of employment. An interview should be considered a two-way conversation. You're not selling yourself as a slave, you're entering into a mutually-beneficial, private, voluntary arrangement. Thus, even someone who goes into an interview willing to accept anything and everything they offer could be expected to ask simply, "And what exactly will I be doing?" But better yet, grill them about every nitty-gritty detail you can think of. Although some insecure interviewers may be taken aback (I'm guilty of asserting the interviewer was wrong on more than one occasion, both times still receiving an offer), I for one am impressed when a candidate demonstrates a sharp, critical and skeptical mind in this way.

jasey• 13 years ago

The interview is a 2 way street. While the company makes you jump through hoops to see if your good enough, you also have a opportunity to determine if you want to work for them.

I like to ask "what will I be working on in the next 6 months" that way you don't rock up and than the second day they through you in the deep end of building a iPhone app.

Granted, startups only have a vague idea of what they will be programming with short periods but it helps.

Also ask "what will be my performance indicators". If they don't include "being able to very quickly learn new technologies" its hardly your fault.

czarpino• 13 years ago

While I agree that puzzles and mind games are silly ways to appraise coding skills, they do give an insight about a person's raw intelligence, or knowledge, or potential. As CS is an application of math and programming is an application of CS, being good in math does not necessarily mean proficiency in it's application; same goes for CS. IMO, a good programmer must, at least, have:

+ knowledge - generally mastery of math/CS concepts and can be thought of as the potential

+ application skills - modeling a real world problem into a theoretical, computable, and (ultimately) programmable form

+ execution skills - implementation (coding) of a solution including the ability to utilize requisite tools/technologies such programming languages, DBs, OS, and so on

That said, hiring process should cover each of these areas and programmers should work on all these as well.

dschiptsov• 13 years ago

So it boils down to "show me your code" and then "please write a few test examples". To staff up a cheap coding sweatshop this method is good-enough.

In most cases an applicant must be able to read English (to google some code to copy-paste and occasionally search through documentation) and able to install and run Eclipse.

The real problem with hiring is that a HR middleman is ignorant and can't tell a good code form a restaurant menu. So he must give a very few simple exercises from common text-books with known answers.

The even bigger problem is that almost no one needs coders, everyone wants programmers which is a complete different set of analytical and engineering skills.

Coding is just a process of translation of a ready-made by someone else, poorly understood (if at all) specifications into a spaghetti [Java] code by calling poorly understood methods of ready-made classes, coded by someone else.

Programming is a process of understanding and describing reality (in terms of design documents, protocol specifications, and then, least importantly, source code in a several languages).

The criteria of success for a coder, btw, is when it just compiles (unit-tests? what unit-tests?) by the industry-strength most advanced compiler of the most sophisticated industry standard static-typing language (static typing is a guarantee from stupid errors, everyone knows) which is even verified to run correctly on the most advanced VM which incorporates millions of man-hours of optimizations, unless.. Never mind.

Success of a programmer is when it, like nginx or Plan9 or OpenBSD, is good-enough.)

etler• 13 years ago

Some reasons why math puzzles might be popular:

* CS majors take math in college, but there's lots of other stuff going on with their CS classes so it's hard to remember math class. If you can actually remember anything from math class, you must be smart!

* Programmers like math, even though it's not their job, so interviewing is the only opportunity to have fun with it!

* We had to deal with this crap when we were interviewing so now it's time to make someone else suffer! >:)

Dear Startups: stop asking me math puzzles to figure out if I can code

Related Stories