Who Y Combinator Companies Want

http://data.triplebyte.com/who-y-combinator-companies-want-c1880a08ac88

by Harj • 11 years ago

667 521 11 years ago

data.triplebyte.com

minimaxir2• 11 years ago

For a blog article under a data subdomain, there's surprisingly few numbers and quantitative analysis, which is disappointing. And one matrix of survey data as the only visualization.

I get that startups do not want to reveal their competitive advantage, but there has to be some give-and-take. Taking an analysis on blind faith alone is frustrating.

reitanqild4• 11 years ago

I found the detail level close to perfect. It explains a lot of my feelings towards the recruitment process.

minimaxir2• 11 years ago

That's confirmation bias, which is dangerous especially since a lot of people in this thread seem to have similar sentiment.

Quantitive data would confirm if the patterns are stereotypes or not.

sremani2• 11 years ago

It is not what you know, but how you tell the story :) Human condition never changes, we are just emotional creatures and perception is reality. It is true in recruitment process, all these new fixes to recruiting are a different versions of that.

cogware• 11 years ago

"Two large YC companies (both with machine learning teams) have told us that they consider interest in ML a negative signal."

I wonder why this is? Since ML/AI are currently "hot" those programmers may be trend followers? Or maybe interest in ML is correlated with being a junior programmer (those that are more senior specialized when ML/AI were not so cool and consequently are in different domains)?

tdaltonc5• 11 years ago

probably because ML is the hot new thing. People that are interested in a hot new method (especially if there "excited to learn!") instead of the results that that method can product (a better product) are prone to running down technical rabbit holes over shipping.

exelius4• 11 years ago

Because ML/AI are feature enablers. They make a good product better, but they won't make a product successful. It's a signal that people are more interested in solving technical problems than solving business/product problems.

Alex39172• 11 years ago

Also, a large percentage of ML/AI projects are scams. And even the people who aren't scammers tend to massively underestimate the amount of work required to make something good. It doesn't surprise me at all that interest in these technologies could be a red flag on multiple levels.

Sure there are plenty of great startups built with these technologies, but both also tend to be the 'and then a miracle occurs' of the tech industry.

facepalm4• 11 years ago

So Google without good AI would be still as good because of the neat interface? Self driving cars would be just as good without good AI?

I get what you mean, and it might be true for a lot of products. But there are also products were good AI is the core.

daveguy• 11 years ago

A lot of ML work is currently ad hoc. Not what you want in most software design and development. Accomplishment in ML and interest in ML are very different. With the hype these days an interest in ML is almost like an interest in making the computer magic.

majormajor• 11 years ago

Not at a YC company, but yeah my guess would be that it's hard to get the trend followers on board with other stuff. I've anecdotally seen myself a lot of candidates (esp at the recent-university/recent-MS level) who've taken some ML courses cause it's trendy and sounds interesting but (a) don't have a serious enough interest in it or knowledge of how to apply it well enough to be a good fit on our ML teams and (b) aren't open to other roles because they sound less cool or have a perception that the day to day work will be more tedious.

ammon19• 11 years ago

Author of the post. I think that this is exactly right. I don't know what motivated the companies to put that policy in place (they just told us that they had this preference). But I can speculate. There is an epidemic of interest in ML. Four out of 5 college grads we speak to list it as an interest. I think that interest has grown to the point where it's no longer any kind of signal about technical strength, and perhaps a signal that the candidates will not be flexible about what they work on.

kevinschumacher• 11 years ago

Totally agree with this.

Many candidates I see that have a "strong interest" in machine learning have no idea wtf machine learning really entails; they are listing it as an interest because it is a buzzword and "sounds hard". Most of them have just used scikit-learn once or twice, and have no idea about statistics.

(also not at a YC company)

reitanqild4• 11 years ago

Maybe they are looking for more practical programmers at the moment?

dsugarman• 11 years ago

I wanted to ask the same question.

Harj, is it people with ML/AI interests without experience? Does this also include PHDs? On that note, are Academic Programmers Comp Sci PHDs or from other fields?

ammon19• 11 years ago

Author of the post. We did not distinguish by experience level in the question that we asked about tech / product. But experience is VERY helpful in a job search, so an experienced ML/AI person would almost certainly have more interest than a product person fresh out of school. We've not seen much weight given to CS PhDs. Someone fresh out of a CS PhD program is viewed much like someone out of a BS program. Industry experience is a big help.

zippergz• 11 years ago

My experience is that programmers I've worked with who strongly identified as being interested in ML tended to be prone to wanting to build very complex ML-based systems when much simpler (albeit less sexy) solutions would suffice. Certainly this is a generalization, but I've seen it enough times to be wary.

matwood• 11 years ago

I don't identify as an ML person and fight this type of complexity all the time. Start simple and then move complex. Starting complex is simply overkill for so many problems.

jonathankoren2• 11 years ago

They think ML people aren't interested in infrastructure. That's not entirely true. Many people with actual production ML experience understand and want to work on the data infrastructure because it's all to easy for someone that's not going to build products on top of it to fuck it up.

rifung3• 11 years ago

I can't speak on their behalf, but I can see how this would be interpreted as a negative signal. If someone is really excited about ML stuff, and you aren't going to hire him for your ML team, then I would be afraid that the person will be disappointed that the work we give him/her is less about solving complex problems and more about getting stuff done. There's also the issue that this person is probably going to jump ship the second he gets an offer to work on ML stuff.

As someone having a strong interest in math and theoretical computer science, I think their bias is fair. I think I'm a pretty good programmer, better than most I've met with similar experience, but I'll admit that I don't care as much as people who are really passionate about building stuff. They will write sloppy code sometimes, but they'll also focus on getting stuff shipped, whereas I naturally want to focus on solving interesting issues like that bug which only seems to happen 1/20 unit tests but no customer has reported.

It took me some time to learn how industry differs from university programming, and if I were recruiting, I don't know if I would want to deal with the hassle. Obviously now that I know I accept industry for what it is and make sure I do my best, even when it doesn't align with my own interests necessarily, but that takes some maturity (not that I am particularly mature), and I can see why hiring managers would rather avoid the risk when hiring and firing is very expensive and annoying.

tmarthal• 11 years ago

Most likely because if you are doing ML and do not have a PhD (or previous experience), you are just looking at calling a library function that you do not understand. The majority of 'machine learning meetups' (not in the Bay Area), are attended by programmers that are looking to figure out how to call an R package to give them recommendations or similar items in a list (clustering).

edit I just read the other replies to this post. I believe that most startups with Machine Learning teams are doing more than just calling R-libraries; most development work that I've done for myself and teams has been for tooling and operationalizing data infrastructure (i.e. data engineering, not data science). However if you need a simple recommendation for an app then calling the library methods without 100% understanding may be enough (but calling library methods without underlying understanding is a bad trait in a programmer (e.g. calling the sort() function without understanding quicksort)).

TheOtherHobbes• 11 years ago

Most likely because if you are doing ML and do not have a PhD (or previous experience), you are just looking at calling a library function that you do not understand.

I'm assuming that's your interpretation of the industry mindset, and not a view you personally hold.

IMO it's a naive assumption, and it would be trivial to test for it in an interview.

I have an interest in ML for a specific domain, no PhD, and it wouldn't cross my mind to try to use it as a set of shrink-wrapped library functions.

B1gred• 11 years ago

I am a statistician working in the data sciences. I see lots of programmers show interest but do not have the depth of knowledge in mathematics and statistics. They can apply libraries and do 80% fine but do not have the educational background to side step assumptions and pitfalls.

GFK_of_xmaspast• 11 years ago

In addition to what's been said, I think that ML/AI, especially AI, attract people interested in the Big Question and aren't exactly detail people.

tdaltonc5• 11 years ago

I like the ontology of programmers. Are there any other efforts to do this? Anything data driven or externally validated?

Given what a big role personas and demo/psycho-graphics play in marketing, I'm surprised that I haven't heard more about then in the context of hiring and managing.

ammon19• 11 years ago

Yeah (author of the blog post here), I think it's a powerful idea. I don't know of any other public attempts to do this (most ideas about hiring stay locked up inside companies). I'd love to hear any ideas you have. Email me at ammon@triplebyte.com

NhanH2• 11 years ago

Out of curiosity, on the taxonomy of programmers, is that supposed to be mostly exclusive, or can they overlap?

tdaltonc5• 11 years ago

You might not be able to share it, but I'd love to see a covariance analysis on that matrix. You have this hypothesis that companies hire based on some sort of culture. Is that culture random or are there types of culture? Product-focused v tech-focused. test-driven v code-review-drived. If so, the companies hiring practices should cluster.

This would be very useful for triplebyte to know. It would tell you very quickly who this or that company would like, without just throwing a bunch of candidates at them and seeing what sticks.

exelius4• 11 years ago

Well, consider that there are well documented academic studies (that I can't be bothered to look up right now) showing that most hiring processes are no better than throwing darts at a board with regards to retention rate, employee success, and every other measure of success of the HR process. The only thing that is remotely effective are IQ scores, and even then it's only a weak correlation.

So regardless of what the companies want, it's near impossible to accurately judge someone in an interview process. Even if you know what you want, it's very difficult to assess how someone is going to perform in a job through an interview process. The main benefit of interviews IMO is to help managers "buy-in" on hiring decisions.

Matt_Mickiewicz2• 11 years ago

Not true.

Actually work sample tests have a .29 correlation to on the job performance, followed closely by structured interview questions.

See: http://www.wired.com/2015/04/hire-like-google/

exelius4• 11 years ago

The work I'm referencing specifically called out Google as an example of an overly-complex practice that didn't result in objectively higher quality candidates. Google's overall quality of engineering talent is high because they are quick to push people out the door if they don't meet the high standards. Regardless, it's very difficult to scale Google's hiring practices, and it's overall seen as a growth limiter for them (even by people internally - it can take 3-6 months to hire one candidate).

endrebak• 11 years ago

According to the psychometrics books I have read structured interviews are as good as IQ-tests when it comes to predicting job-performance.

All types of hiring tests/interviews had some predictive power (except handwriting analysis).

ryandrake7• 11 years ago

"We’ve seen that most engineers only have the stomach for a limited number of interviews. Investing time in the wrong companies carries a high opportunity cost."

I suspect in addition to not having the "stomach" for an unlimited number of interviews, they don't have the vacation days to burn for them. Let's say you are working already and have 10 days a year vacation (pretty standard). With these ridiculous all-day interviews, you have the ability to waste that vacation on a maximum of 10 companies, and that's if you choose not to take any "real" vacation at all during the year.

In an environment like today, where every company is ostensibly hiring yet overly picky, a job candidate potentially will waste their entire PTO balance on interviewing with companies serious about interviewing but not serious about hiring.

draw_down• 11 years ago

A good example of why working at a place with such restrictive PTO is a trap. But, around here we like to have philosophical discussions about how unlimited vacation time is the real trap. I dunno, guys. It seems to work pretty well for me.

metamet• 11 years ago

Or just start off with a proper number of days. Two weeks seems so short, especially if you get sick. 15 days total seems about right.

toomuchtodo2• 11 years ago

I ask for five weeks/year, even if it means I have to forgo some pay. I have yet to be told no.

I don't think you've ever experienced the passive aggressiveness that comes with using "unlimited" vacation time.

dagw• 11 years ago

The problem is to my boss "unlimited" might mean 15 days and to me it might mean 4 month. If we come in with wildly different ideas about what is OK, we're both going to end up very unhappy.

chatmasta• 11 years ago

It's not only the applicants who have limited time. The startup founders are also time constrained. You can't found a successful startup if you spend 10+ hours a week interviewing potential employees.

I suspect much of the initial bias and "hard line" preferences are due to a startup's unwillingness to spend inordinate amounts of time interviewing candidates to find that one diamond in the rough.

Imagine 4 of every 10 "academic programmer" candidates are a good fit, and 6 of 10 "product programmer" candidates are a good fit. Why waste time interviewing 10 academic programmers, for a max hit rate of 40%, when you could focus solely on product programmers and get a max hit rate of 60%?

This data certainly suggests that different founders have their own biases and preferences for what makes a "good programmer." Most of them probably realize they are pigeonholing and could be wrong, but they're willing to risk that for more efficient interviewing and a high hit rate.

Stick with what you know if it's "good enough." Optimally, as a founder you can already identify multiple potential early hires just from your own network. If you have the right background and connections, you can hire your first couple employees without doing a single interview.

Time is money. Interviews take time and cost money.

ammon19• 11 years ago

This is definitely true. The filters that companies and recruiters use mostly trend in a meaningful direction. There are a higher percentage of good programmers among MIT grads than SUNY Potsdam grads. But this fact does nothing to help the good programmers who look bad on paper who everyone is ignoring. This is precisely the problem that we're trying to fix with Triplebyte. We want to make actual technical evaluation (not credential evaluation) cheap and accurate, so we can just interview everyone.

Retric• 11 years ago

Interviewing is ridiculously important for any small company. The problem is most people have little idea how to do anything except look for people like them.

TimPC• 11 years ago

Many successful people who've built and exited massive start-ups have suggested that it's worthwhile on the start-up side to spend 25% of your time on hiring when you're actually hiring as so you can easily get above 10 hrs a week given a full time role on this and have that be a good result. Optimizing to minimize the cost of hiring an early engineer is definitely solving the wrong problem, given the early engineers are almost as crucial to success as founders. Unfortunately, hit rate is also often the wrong variable from the company end: if you interview an academic programmer to confirm their product programming skills you generally get someone better with a larger skill set than if you focus in the category you want for the skills you want (early-stage you generally want breadth over depth for many roles). It clearly sucks for people on the receiving end of the low hit rate interview, but I'd caution start-ups against optimizing their process for the wrong things.

LordHumungous8• 11 years ago

That would be true if you could show that candidate "types" are actually meaningful distinctions and not just the founder's first impression based on limited data. I think that's a stretch.

s_kilk• 11 years ago

> have 10 days a year vacation (pretty standard)

Wait, what?

Is it really this bad in the States?

L_Rahman• 11 years ago

Yup. 10 days paid time off is the de-facto standard for just about every company based in this country.

Some people negotiate additional PTO as part of their compensation package but I don't need to tell HN that engineers are notoriously bad at doing this.

scarmig2• 11 years ago

In my experience in the Bay Area, jobs fall into one of three categories:

1) No paid vacation at all

2) 15-25 days of vacation a year

3) "Unlimited vacation," which is usually a hip facade for 1 or 2.

georgemcbay• 11 years ago

Yes, it is. Even worse than the fact that getting 2 weeks off is considered a pretty good amount of time, there's often lots of subtle pressure (that manifests itself implicitly in project planning) to not use it all at once.

This pressure isn't universal, but I'd say the majority of the companies I've worked for hardly anyone ever took more than a week off in a go, and when they did it was the topic of uncomfortable conversation with managers kind of acting like the person taking the time off was doing something wrong (even if just kind of half-joking about it).

It really is an entirely weird situation, for all the talk of meritocracy and such the day-to-day reality is you are pressured not to take real PTO (paid time off) and instead just do the Office Space thing where you're at work almost every day even if there are periods of time where you aren't really doing anything productive for a large chunk of most days.

merpnderp• 11 years ago

My company starts at 10, then after 4 years you get 15, then after 11 years you get 20. This is in addition to 12 holiday days off a year, plus unlimited accumulation of 1 sick day per month, which can be taken any time you feel "sick".

So you start with 22 working days off including holidays, plus generous sick time. Not perfect, but could be worse.

tedks• 11 years ago

And yet the USA is the richest country in the world, with a culture steadily dominating and extinguishing all other cultures. There is a McDonald's in Red Square right now.

You don't make an omelet without enslaving some chickens their entire lives, keeping them in barely one cubic foot of space from birth to death so they can push out eggs at maximal productivity, killing them as soon as they're no longer at peak performance, and of course, breaking the eggs.

In other words, you get what you pay for.

jlarocco• 11 years ago

I don't know about Silicon Valley, but in Colorado it's almost never that bad for software jobs.

In 12 years I don't recall any jobs with fewer than 15 days. My current job is "unlimited", and I took nearly 25 days one year.

Kluny• 11 years ago

It's pretty third world out there.

whatever_dude• 11 years ago

Yes it is. In most companies it's common to start with 10 days and then increase by 5 days with every few years of experience. Some do more; starting with 15 days is generous.

But also remember it's business days. So 10 days = 2 weeks.

stephencanon• 11 years ago

It varies tremendously from company to company. 10 days is not unheard of, but for example at my job, new employees get 12 vacation days plus 15 holidays. I've been there 9 years and get 21 vacation days a year plus the same 15 holidays (my long-tenured coworkers and I often struggle to use all of our vacation; we tend to take a lot of Fridays off to stay under the six-week cap). By contrast, many low-wage jobs get no paid vacation.

kabdib2• 11 years ago

It's typical. Often you get 15 days/year with several years at a company. National holidays usually don't count.

I was at a start-up once where we got a new CEO who essentially canceled a week of "whole company time off" at Christmas. If you wanted that time off, it came out of your vacation. He didn't care . . . and soon after that, I didn't care, either :-)

[Didn't help that he was really scarce in the office that week...]

AnimalMuppet• 11 years ago

My current place (not in the Bay Area) gives 18 days after 4 years (I forget what it is before that).

aerovistae• 11 years ago

Of course, if one plans things well, one has the leeway to leave one job and take a few weeks off before finding another...

BinaryIdiot2• 11 years ago

Not everyone is comfortable with the unknown of not having a job lined up. Even the most technically skilled developers may not pass all of their interviews and some companies, especially the bigger ones, can take a huge amount of time.

What you say is true to a degree; if you have plenty of money saved up then you can spend a lot of time unemployed but this ultimately looks poorly on your resume at some companies and most people live closer to paycheck to paycheck than anything else.

ryandrake7• 11 years ago

I would never leave a job before finding a new one. Too much uncertainty especially in today's rough hiring market. You'd want at least 6 months of living expenses saved up to burn while you job search, probably more, and most people don't have that.

vinceguidry3• 11 years ago

I found my current job through a recruiter, I think I will find all subsequent jobs through one as well for this reason. Applying for a position is fundamentally an act of sales, so it behooves you to either be a salesman yourself or to hire someone to do it for you. Otherwise you're throwing yourself at the mercy of people's deep-seated fears and prejudices.

pauldelany• 11 years ago

True, but also consider that you're the product in this case, and the salesman (in many cases) doesn't care who he makes the sale to. It may be better to first research and shortlist companies that you want to target, and then find a recruiter who has them as a client? Never done that myself, so don't know how practical it might be.

ammon19• 11 years ago

I think that recruiters do make sense (clearly). I hope that we can use our position as recruiters to be better than most companies at figuring out who's a good programmer, and better than most candidates at figuring out which companies are good places to work (we're candidate constrained, so we have no incentive to send people to bad companies)

facepalm4• 11 years ago

Yes, but did you hire the recruiter?

I think they are usually hired by the companies, or "loyal" to the companies, because that is were the money come from.

Would be a fun experiment to hire somebody to sell yourself. Then again, maybe that is what is usually called a "personal coach"?

sarciszewski• 11 years ago

Let's say you are working already and have 10 days a year vacation (pretty standard).

Until my current gig, I was never once allowed vacation time.

jonesb6• 11 years ago

Edit: What I really meant..

Poor choice of words on my part. Take it in this context: "Person X doesn't have a stomach for hard engineering problem Y".

"Doesn't have the stomach for" now can be interpreted as a synonym for:

* Is not capable of

* Is not smart enough for

* Is not hard working enough for (!)

In the current environment I definitely shouldn't have brought up gender.

afarrell• 11 years ago

I guess I've not seen that many westerns, but I really was not aware that the stomach was a gendered organ.

pc862• 11 years ago

This kind of nonsense comment has got to stop. It's reaching the point of ridiculousness.

Edit: Was originally some kind of nonsense about the expression "doesn't have the stomach for" being "a jab at masculinity" or something.

oldmanjay• 11 years ago

The thing about looking to be offended is that you can find it everywhere without really even trying. Being offended by that phrase definitely qualifies.

You do, of course, have the natural right be offended by anything. That doesn't mean people have the responsibility to not offend you. In fact, in America, home of HN, we explicitly have the right to offend you.

pinewurst2• 11 years ago

That's definitely my perspective. I've been doing this for a very long time and it's evolved to the point where I actually start hyperventilating when thinking about interviews, at least the technical ones. I'm looking now and I don't think I can force myself through more than a few cycles before falling back to marketing/product management to avoid it.

ryandrake7• 11 years ago

Interesting that you think of product management as a "fallback" case. I've found getting hired as a product manager infinitely more difficult than getting hired as an engineer. For every company that says they are looking for a product person, there are at least 20 that say they are looking for engineers. Of course, the main point of my original comment still applies: "looking" definitely does not equal "hiring" in today's tech job market.

vdnkh• 11 years ago

I'm down to 2 days out of my 10. I used to take a day if they had a big "tech screening" over the phone but now I'm adamant about doing them after hours.

Almost no one passes all their programming interviews

But there is always someone who will, and many companies are more than happy to pass on all candidates until the find someone who days. I was super happy that my interviewer yesterday said we were skipping the whiteboard section.

the-dude• 11 years ago

20 to 25 days in Western Europe. And working 32h gives you huge flexibility.

You can start shifting that extra day around for these occasions, leaving your vacation days for actual vacation ( which is 80% as is the pay ofcouse ).

kafkaesq• 11 years ago

"We’ve seen that most engineers only have the stomach for a limited number of interviews. Investing time in the wrong companies carries a high opportunity cost."

Yet it's amazing how many companies treat candidates' time as basically worthless, and infinitely replenishable.

kabdib2• 11 years ago

Yeah, the "two week trial period" (or one week, or week-of-evenings, or whatever) is a non-starter for the experienced programmer who doesn't really need a job. And these are the folks that you want to attract.

High bars can really turn out to be self-sabotage.

LordHumungous8• 11 years ago

I've made the mistake in the past of overcommitting myself to interviewing with multiple companies at once. The time commitment wasn't the worst part, the real factor I didn't for see is that interviewing is stressful and mentally exhausting. On top of the interview itself, there's the hours of studying you have to put in. It's extremely hard to do while doing your full time job.

Now I have a rule that I will interview with ONE company at a time. Consequently, I am very careful about picking which companies I will interview with.

vonmoltke7• 11 years ago

I interview with whomever actually responds to me. :|

noarchy6• 11 years ago

On top of the interview itself, there's the hours of studying you have to put in.

Yikes, are people really doing this, or feel like they must?

If I'm interviewing, I expect to be asked about previous projects, maybe my Github portfolio, and so on. I also expect to be asked how I might approach a given problem within my area of work. I can't imagine studying that - I'm paid to do this stuff every day.

So are we talking about stuff that isn't related to my area of work, like brain-teasers and random algorithm questions?

x0x0• 11 years ago

You forgot the part where, for every $100k you earn, a vacation day that you spend interviewing costs you $240 after taxes because you don't get that paid out when you quit. Or get to use it vacationing, which I value rather higher than $240/$100k since I get so few of those days.

So it's an expensive, stressful, crappy experience. Weird I don't want to do it speculatively, eh?

jcadam3• 11 years ago

I'll use myself as an example -- I work as a programmer in the Defense industry. I've wanted out for years. So every so often I start sending out resumes and going on interviews.

After wasting about half of my vacation time on (multiple) rounds of interviews with each prospective employer (that ultimately lead nowhere -- most often, a potential employer who inexplicably stops communicating with me), I give up.

Then, when I need to find a new job (project/contract ending, etc.), I do the safe thing and apply to other defense companies, because getting an interview almost always results in an offer (and no multiple-rounds of white-board hazing either). After a while at the new job I'll start looking outside the industry again --- and promptly get pounded back into my pigeonhole :)

ryandrake7• 11 years ago

"Whiteboard hazing"

Heh. I'm going to squirrel that one away for later. It beautifully captures what's currently thought of as a best practice in tech interviewing.

brianchu2• 11 years ago

I'm a CS student and most students I know dislike whiteboarding too (and by extension many, if not most, YC startup employees do too). But I'm pretty okay at them now, having had lots of practice. The key advantage for students I think is the ability to have sustained practice over the course of a month (e.g. > 4 onsites over the course of a month), which most students can afford since their schedules are flexible. Also, I've recently had interviews at a few YC companies you've heard of, and they had me code on my laptop. I didn't whiteboard at all except for system design questions.

Reply to below: Most companies have me do at least one non-technical system design and/or talk-about-a-project interview.

MCRed5• 11 years ago

I suggest only doing interviews in the evening via phone. (not video. I've been discriminated against by video, but never by phone.) Do this in the evening. If the job is in your town and you feel its reasonable to meet them in person, do it in the evening or on a weekend.

Universally I've found that by being a little bit picky you filter out the people who aren't serious... and a lot of people aren't "serious". They might seriously want someone but they have no capability to identify that person accurately when they have found them.

swalsh2• 11 years ago

"whiteboard hazing"

there are two types of whiteboarding questions. The kind that's a perfectly reasonable thing to ask (i've had people ask me to develop some simple system... which I think is reasonable, because i'll do that as part of the job)

but then there's the solve x algorithm questions. Which ultimately is an exercise in memorization. At this point I've just accepted that i'll have to do that.

engi_nerd7• 11 years ago

The defense industry is strange. I've had three jobs in it. In every single interview, the most technical question that anyone ever asked me was, "What do you like to do on the weekends?"

And all three jobs have been engineering positions.

oskarth2• 11 years ago

Have you considered saving up enough money to quit and give yourself 3-6 months to find a job you enjoy?

BinaryIdiot2• 11 years ago

Exactly the same situation here; been in the defense space for a while and while it's easy to get another defense job as you already have a clearance (so you're a much safer bet) venturing out into the commercial world can be daunting.

I once interviewed with Amazon and after flying out and doing the interview I'm told that they liked me but I wasn't a fit for the team and could interview with another team without a delay and they'd even help me figure out the team to interview with. But why would I waste my time again taking days off to fly out and either not pass the interview or interview with the "wrong team" again?

When I interviewed at Microsoft it was a pretty positive experience aside from one person and while I certainly wasn't upset about not getting the position I was told they had 100 people interviewing in person for a single position (supposedly almost 1,000 applications). Great odds for Microsoft but for me taking 3 days off work to do it? Great experience but I'm not sure I'd do it again so easily.

taurath5• 11 years ago

This is exactly why I decided to resign from my last job instead of looking for work while in it (in addition to the crippling stress that caused me to look in the first place). There simply isn't enough time to find the right fit if you don't really quite know what you're looking for - even being remote on a very open schedule the time it took to take recruiting calls, do resume and code test work was very high. Granted, I already had an offer outstanding when I decided to do that, so I was reasonably sure I would be able to bridge the gap to my next job using savings I had accrued. Now I've found an interesting job paying MUCH better than I was expecting which will in the near term greatly overmatch the investment of some of my savings in the time to look.

scruple• 11 years ago

companies serious about interviewing but not serious about hiring

I've been watching a friend interview with their 'dream company' over the last month. It's been quite an exhaustive process. Four rounds of interviews: a phone-screen, a take-home test that took a weekend, an all-day in-person, and a half-or-full-day remote session. They've moved beyond interview step #2 and have secured a date for interview step #3.

The way you've phrased it here is exactly what I have internalized from watching this process unfold but have been unable to articulate so succinctly. Suffice it to say, I'm not looking forward to my next job hunt.

lukasm• 11 years ago

wow, I had no idea that there is no paid vacation time in US. The only country in the world. What about unpaid time off? Now I know why I hear about burnout from US developers.

EDIT: After few minutes with Google.

USA: There is no statutory minimum. It is left to the employers to offer paid vacation days as part of the compensation and benefits package.[67] According to one survey, 98% of employers offer at least some paid leave to their employees; full-time employees earn between 6 and 20 vacation days at 86% of employers surveyed.[68] About 96% of surveyed employers give their employees paid time off during public holidays,[68] typically 6 per year.[69] Some employers offer no vacations at all.[68] The average number of paid vacation days offered by private employers is 10 days after 1 year of service, 14 days after 5 years, 17 days after 10 years, and 19 days after 20 years
European Union legislation mandates that all 28 member states must by law grant all employees a minimum of 4 weeks of paid vacation
Sweden: Employees are entitled to 25 work days of annual leave. Sweden also has 11 public holidays and a few that may or may not be half days.

quanticle3• 11 years ago

There is no mandated minimum level of paid vacation time in the US. Most companies do offer some paid vacation time, but it varies from company to company (and even employee to employee, depending on negotiation skills).

As far as unpaid time off, culturally, the amount of paid vacation time you get at a company is assumed to be the maximum amount of time off you can take in the normal course of matters without putting your job at risk. Normally people only ask for unpaid time off if there's some exigent event that they have to take care of (health issues, family issues, etc.).

zo12• 11 years ago

Well, "no government-mandated minimum of paid vacation days" is a more appropriate description of your observation, judging from your EU-findings.

geebee2• 11 years ago

It's not surprising. A lot of lawyers will tell you that passing the bar was one of the more grueling things they've had to do. Here's a blog post about a guy who did it with 100 hours of study.

http://blakemasters.com/post/37113468298/pass-the-ca-bar-exa...

The bar exam is three grueling days. 100 hours of study is two and a half weeks of full time work.

If you interview at three tech companies, I don't think it's at all over the top to say you might study for a few weeks in total, especially if you are rusty, and the interviews are often quite grueling as well and do last all day….

Ok, I'm stretching, I won't put one iteration of interviews at bar exam levels, but we are starting to get there. Now keep in mind, we do this over and over and over in our field. And while the bar is tough, at least you know roughly what will be tested - tech interviews are often a completely moving target. And while the bar does have continuing education requirements, you don't have to do the 3 day massive study thing over and over (unless, in some cases, you move to a new state).

I really do think this is a severe problem in our industry that causes higher levels of attrition than we realize. I suppose it could be one of those "tragedy of the commons" type things, where each company benefits from long and difficult interviews, but the cumulative result is either 1) people not wanting to enter the field, or 2) people giving up on interviewing for new jobs and staying with their existing employer even if they are burned out, or 3) people quitting and going into a different field or role entirely where they can escape this hazing.

I also think this is something hiring managers should realize before saying that there is a a shortage of programmers. Your own interview processes may be contributing substantially to this "shortage".

Oh, one last thing - excellent article, thank you for writing it! Absolutely fascinating, and it explains a great deal.

steven20125• 11 years ago

This is probably it. I bit the bullet and scheduled 2 weeks off, and filled up those days with onsite interviews. It sucked to have to take those many vacation days but ultimately worth it.

runamok• 11 years ago

I was thinking on this and thought I should start a company that:

a. Pre-screens candidates on off-hours since they are likely already employed with a focus on keeping it brief out of respect for their time. b. Similarly match interviews at off-hours and perhaps moderate them to also not waste anybody's time.

I am sure there are many people that would not like this but if hiring is difficult and they want to make it rigorous I wonder if this could fit a niche. I personally am more than happy to go to an interview after work or on the weekend at a coffee shop so it doesn't cut into my job hours.

jmorrow977• 11 years ago

I'm also interested in this. Please email me at jmorrow977@gmail.com to discuss in private. Others are also welcome to contact me there about this.

siculars• 11 years ago

I don't think you could pay me enough to work for a company that only offered 10 days a year vacation on principle.

acjohnson55• 11 years ago

I've found in the past that companies are pretty willing to work with me and let me split interview day between two days. It's worth asking.

But just to undermine that point, I've also learned that it's quite complex to try to juggle multiple job interview processes with a day job, especially if you want to be able to make a choice with multiple offers on the table. It ended up working out for me this past time, but next time I'm ready to make a move (hopefully not for years!), I may just resign first, so I'd be able to devote my full attention to making the right move.

nilkn• 11 years ago

I've long felt that a lot of software companies are keener on interviewing than they are on actually hiring. There's got to be a breaking point somewhere where too much time spent interviewing is actually a net loss even if you end up finding good hires, and I wouldn't be surprised if many companies are beyond that threshold.

The company I'm at doesn't employ any magic voodoo in our hiring methods, but we somehow manage to find great people without being a revolving door for interviews. Many places are proud to say that they only make offers to 1% of the people they bring on-site, but I'm proud to say that our ratio is probably more like 25% or even 50%.

icedchai• 11 years ago

You actually take vacation days for interviews? I either call in sick or come in late. I usually provide a plausible excuse, like having to take my cat to the vet. Or maybe my foot is bothering me, and I had to wait to get an appointment.

Just the other week, I ran out for a meeting with a potential new employer in the middle of the day. All I said was I "had to run an errand." If I was less checked out, I would've made up a real excuse and told them I needed to get a blood test.

Most people are totally non-confrontational. You're not going to get called out on your little white lies. Obviously you can't use the same excuse 10 weeks in a row, so mix it up a bit.

vonmoltke7• 11 years ago

You obviously have the luxury of being in an area with lots of potential employers. Some of us are not that lucky.

facepalm4• 11 years ago

Imagine going on a date and after the fourth time being like "I still don't know if I like you, need more tests". I tend to get impatient with companies who ask for yet another round of interviews because I think either they like me or they don't.

Maybe the issue is too risky to employ somebody? In my country a lot of laws protect employees, so it might be difficult to get rid of bad hires (although in the first 6 months I think it is easy, as they count as probation period).

ryandrake7• 11 years ago

The USA does not have this excuse. It is easier to fire someone here than anywhere else in the developed world.

RyanZAG• 11 years ago

The pattern matching issue with Java or C# is interesting, since all these companies generally do want mobile. So I guess the humorous point of advice for that: replace all occurrences of Java on your CV with Android, and C# with WinPhone, and you'll probably bump your chances of getting passed the pattern matching.

ammon19• 11 years ago

Great advice! Mind if I add this to the blog post?

api4• 11 years ago

"Second, companies dislike programmers with enterprise backgrounds. Our data shows that companies are less likely to hire programmers coming from Java or C# backgrounds."

This I totally understand. Enterprise software is systematically horrible in almost every way: terrible UI/UX, insane degrees of over engineering, high footprint, high cost, and usually at least two to three generations behind on every technological trend. Of those traits I can't stress over-engineering enough... it's epidemic everywhere but most of all in "enterprise" where you'll see insane things like simple web application servers that use gigabytes of RAM and XML schemas that lead to ten-page-long messages to do trivial things. (The use of XML at all is usually a smell unless the domain is very HTML-like such as word processing... and even there extending Markdown would be better in every way.)

It doesn't have to be that way. Those facts reflect the management pathologies that exist in large companies and governments where you have a lot of people managing software projects who don't know anything about software... lots of "pointy haired bosses." You also have a lot of dumb requirements in those areas that twist things out of alignment with what users actually want. Startups very often have the luxury of ignoring stupid requirements unless they have to interoperate with legacy stuff.

It's so bad that I've actually heard people advise startups to pass on some enterprise revenue if they can afford it (pass on REVENUE!) if it might lead them down an "enterprisey" path, since doing so would in the end result in a systematically inferior product. If the systematic diseases of enterprise are that extreme (and I think they mostly are), then I understand the desire that some startups might have to also systematically avoid developers with enterprise backgrounds.

That being said and while I do understand, I think this underestimates the versatility of a good developer. A good developer might have gone into an enterprise job because they need the money but they might be otherwise great at what they do. You can find some great developers by offering to rescue them from enterprise hellholes. Sometimes that alone is all they need to inspire a ton of motivation and loyalty. I mean, you just dragged them out of hades. They're gonna love you.

frostmatthew• 11 years ago

usually at least two to three generations behind on every technological trend

It's easy to refactor (or completely rewrite) for whatever the hot new fad is when you're dealing with small codebases (<100k loc) that haven't been around very long and most likely won't continue to be in use for long. That's not productive when you're working on a product that has millions of lines of code and has been in use for years (and most likely will continue to be so).

It has nothing to do with "pointy haired bosses" - believe it or not a lot of us [enterprise developers] care more about actually getting stuff done then bragging to our friends how we're playing with whatever is hip this week.

api4• 11 years ago

I think that's a valid point but it only explains a small fraction of the ugliness of enterprise software. It explains why it tends to trend a bit behind, but it does not explain why you need 2gb of RAM to run a web application server or why you need ten pages of XML vomit to add a record to a database. Those kinds of things smack of over-engineering (usually via premature generalization and "architecture astronautism") and cargo cultism among other things. When I look at those systems and code bases I always just sit there and ask "what problem does this solve? what problem does this solve? what problem does this solve?"

I also didn't mean to imply that there are no good developers in "enterprise." But I do believe that "enterprise" is to good developers what, say, a corrupt economy is to good entrepreneurs. You can operate there but it's painful and everything about it gets in the way of doing the right thing.

Also also (heh) I didn't mean to imply that all enterprises are "enterprise." That's why I put it in quotes. Some large organizations manage to avoid these pathologies. I've even seen really excellent systems emerge from government labs if the people running them are clueful. "Enterprise" in quotes refers to the pathologies that afflict big over-priced over-engineered slow clunky UX abominations. Everyone's seen them and everyone's been forced to use them... forced because nobody would ever voluntarily do so.

rowborg2• 11 years ago

I posted this as a comment on the original post, but will make the point here...

Large organizations are often more dysfunctional than small ones, yes, especially when it comes to building software. But that doesn't mean they aren't filled with very smart people. The fact that enterprise programmers can't get hired at startups is not, in my opinion, a reflection of the dysfunctional nature of software development at large companies, but of a mismatch in expectations about how to communicate your ability to do your job.

As an example: one pattern I’ve seen that consistently holds enterprise programmers back in startup interviews (especially in phone screens and technical screens) is the inability to effectively articulate the projects they’ve worked on. Enterprise programmers have often worked on optimizing one small part of a very large and complex system that no one person may understand completely, and in my experience, they often cannot describe that system clearly or holistically. Startups usually need people who can build a system end-to-end, and when an enterprise programmer doesn’t seem to understand their own projects, it reflects very poorly. I see this pattern way too often.

My suggestion (and obviously this is a gross generalization) is for enterprise programmers who want to work at startups to practice being able to clearly and consisely explain what they worked on and how it fit into the overall product. Startups will care more about this than showing, for example, a deep understanding of how Java's various memory pools work (even if they work in Java).

moron4hire2• 11 years ago

There is also a bias against enterprise developers in that most often their work is on proprietary systems, leaving no code artifacts to view on Github, which a lot of companies have started using in their recruiting process.

pc862• 11 years ago

It's so bad that I've actually heard people advise startups to pass on some enterprise revenue if they can afford it (pass on REVENUE!) if it might lead them down an "enterprisey" path, since doing so would in the end result in a systematically inferior product.

Yes, because it's much better to have an architecturally pure product nobody uses than something generating millions in recurring revenue.

api4• 11 years ago

It depends on the situation. It's bad to forego millions in recurring revenue, but it's not bad to forego smaller gains if you have a vibrant business in other areas and if those smaller gains require that you go down a path that will ultimately make your product a loser in the marketplace.

This is why Apple shuns "enterprise." They value user experience. Exposing UX to enterprise development priorities and methodologies is like exposing a kitten to mustard gas. Even a little bit does tremendous harm and it's definitely cruel.

Last I checked Apple was among the most valuable companies in the world. Their stuff is also heavily used in enterprise in spite of their dismissal of it. Sometimes a good product can make inroads into enterprise if it's good enough to overcome certain biases and entrenched pathologies. Some organizations have departments running their own "guerrilla IT" to support things that aren't awful vs. the awful stuff shoved down their throats by corporate IT.

The root problem with enterprise is that products are often built to artificial specifications that someone made up, not to actual specifications learned through real use and feedback or that are shaped by underlying reality. This results in a lot of superfluous complexity that does not reflect the actual complexity of the problem domain. Another major problem is that purchasing tends to be based on feature check boxes instead of actual evaluation of the product by people who will actually be using it. You'd be surprised how often a large org will make a major purchase without even consulting with those who will actually be using the product. I've also seen cases where people go so far as to pretend to use the "boat anchor" product while actually doing things using a different system purchased or built with off-normal-channels funds.

UK-AL2• 11 years ago

So you would not hire someone of a Microsoft or Google background?

api4• 11 years ago

That's not what I said. I certainly would. I was responding to the bias against enterprise -- my point was that I understand the bias but do not agree that it should exclude developers with an enterprise background.

ctstover• 11 years ago

Why would this get down-voted? It's spot on. I know it hurts when it hits close to home folks, I know.

Jemaclus• 11 years ago

Some companies reject people based on green card status, because they don't have the resources to sponsor them. Some companies have specific salary ranges, too. Are those factors considered during your analysis? You seem to be focusing primarily on what companies want, but sometimes what they want and what they can accept are two different things. Thoughts?

ammon19• 11 years ago

We asked about visa sponsorship. We did not ask about salary. The general pattern with visa sponsorship was that small companies don't want do it, out of concern about the success rate (only 30% of H1B applications last year won the lottery). Some large companies will sponsor visas, but they set a higher bar (they have to be really excited to take on the extra work). The exception is larger companies with teams outside of the US (several YC companies have offices in Canada) to take advantage of different immigration restrictions.

biztos• 11 years ago

Doesn't this imply that a branch development office, in a country with friendlier immigration policies, should be a high priority for almost every US-based startup?

I assume most startups want to grow their engineering headcount (there are exceptions, of course). The pickier/more-focused you are about what kinds of engineers you want to hire, the more you have to gain from casting a wider net geographically.

But if you find the perfect teammate and she can't come to your main country due to visa restrictions, or maybe even won't come because the immigration story isn't worth the trouble, then what are your options?

I'm not talking about money here, though of course that will be a factor for some people. It just strikes me that the "YC companies with offices in Canada" have a serious recruiting advantage over those without, once they need to seriously grow their teams.

buro9• 11 years ago

    The “Product Programmer” and “Technical Programmer” profiles are identical,
    except one is motivated by product design, and the other by solving hard
    programming problems.

This is great.

I've struggled with trying to define the difference between the great systems engineers I meet and the engineers who make great products (sometimes at the expense of all elegance under the hood).

This sums it up nicely... it's where the focus is. I'm surrounded by engineers focusing on systems programming, but I don't think of myself as similar to them, I've always been end user obsessed and customer focused. It's nice to see this difference acknowledged.

lackbeard3• 11 years ago

This is interesting data.

I liked what Joel Spolsky said a long time ago. Basically that companies should want two types of engineers. 1. You want a few who are experts in the company's tech stack. 2. The bar for everyone else is just that they're smart and they get things done.

I guess the core problem is that we don't have a good objective measure of the latter. (Maybe an IQ test for smarts, if that wasn't political incorrect, but I don't know how you'd show definitively that you "get things done" in an interview.)

tstock55• 11 years ago

I accepted an offer at IBM a few months ago and had to take an IQ tests of sorts. It was mostly focused around pattern recognition with some algebra problems thrown in. At best it seemed only tangentially related to programming ability.. I'm not sure that attempting to measure whether a candidate is "smart" or not is a good hiring strategy.

A good analogy for this type of interviewing is like trying to judge how good someone is at basketball by measuring how tall they are. Is it related? Sure. But it certainly doesn't give you the whole picture and might not be a useful information at all. Watching them play a few games would be a better strategy.

Fortunately there a quite a few startups working on this problem. Lytmus is one in particular that I think looks very promising.

marme• 11 years ago

IQ tests are not only politically incorrect they are illegal in the US. It is illegal discrimination to make hiring decisions based on IQ scores. You can only use a test that shows those with higher scores perform better at the job. A general IQ test can not be shown to do that. Basically you can not ask questions that test skills that would not be used in the job the candidate is applying for.

You must be able to defend the metric you use for hiring and that it is not discriminatory. If you give an IQ test that is unrelated to the job and say no one below 100 IQ will be hired you must prove that someone with 101 IQ can do the job but someone with 99 IQ can not properly do the job

johncolanduoni2• 11 years ago

It is illegal discrimination to make hiring decisions based on IQ scores. You can only use a test that shows those with higher scores perform better at the job.

There is more scientific evidence that IQ predicts job performance than for most of the more qualitative criterions that employers use in practice. The source of the illegality of using IQ tests is not legislative (intelligence is not a protected class[1]). It comes from the precedent of Griggs v. Duke Power Co., in which the IQ test was being used as a barrier for jobs where IQ was not a good predictor (not programming), and prior to thorough research on the subject. There have been hundreds of studies showing that IQ tests predict job performance (though not super strongly) at the beginning of careers, even though this fades once they have more experience. This[3] meta-analysis approaches these studies very critically, and concludes that many inflate the effect, and that it doesn't validate the IQ test (i.e. there may be non-cognitive reasons for it) but that the correlation still exists. (For what it's worth, I suspect they're right about the non-cognitive part since the correlation has become much stronger over time, particularly around the 1970s. However, when your goal is to test for something and not to explain it, correlation is good enough.)

Basically you can not ask questions that test skills that would not be used in the job the candidate is applying for. ... You must be able to defend the metric you use for hiring and that it is not discriminatory.

No, the burden set in Griggs and later elaborated in a modification to the Civil Rights Act is not to prove that a test is not discriminatory at all. Instead for tests that are discriminatory in practice (though not obviously in design, like IQ), you must show that the test is truly indicative of job performance. If this were not the case, you couldn't ask if people went to college, due to the racial disparity in college graduates. Also you can ask things you can't prove are related to job performance if it isn't discriminatory (not that this is really useful).

If you give an IQ test that is unrelated to the job and say no one below 100 IQ will be hired you must prove that someone with 101 IQ can do the job but someone with 99 IQ can not properly do the job

Of course you don't have to show that anyone who fails your test will 100% be worse on a case by case basis than someone who passes. If that were the case you couldn't justify pretty much any test other than "applicant is not dead or in a coma." You just need to show that it is clearly less likely. I would find it really surprising if there wasn't some kind of IQ threshold effect for programming (i.e. the closer you are to the middle of the spectrum, the better a predictor of programming performance IQ is). That would make it easy to justify using an IQ test as a predictor, especially in combination with the more general studies.

I don't think IQ would be terribly useful to weed out programmers because I suspect there are more related ways to judge applicants that would already weed out those in the region where IQ is useful, but it is not as simple as "IQ testing for a job is illegal."

[1] https://en.wikipedia.org/wiki/Protected_class [2]: https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co. [3]: http://www.tandfonline.com/doi/full/10.1080/10888691.2014.98...

lsc3• 11 years ago

IQ tests are not only politically incorrect they are illegal in the US. It is illegal discrimination to make hiring decisions based on IQ scores.

Oh come now. Every company in the valley emphasizes "cultural fit" and that's about as subtle a dog-whistle as an '88' neck tattoo.

It's pretty obvious that nobody here cares about even the most clear-cut discrimination against protected classes (and it's not at all clear to me that less intelligent people are a protected class.)

rguzman• 11 years ago

The best way to show that you get things done is to have a record of things you've done to point to.

swalsh2• 11 years ago

Former C# programmer here, in my most recent job hunt I rewrote my resume. I talked more about the value I provided for my company/customer (what the code did) rather than how I wrote the code. I'd say my results improved significantly in terms of getting my foot in the door.

TranquilMarmot6• 11 years ago

I do mainly C# at my current job, so it's pretty prominent on my resume; perhaps I'll try a rewrite and focus more on what I've done rather than how I did it. Sounds like a good idea.

jonathankoren2• 11 years ago

"The types of programmers that each company looks for often have little to do with what the company needs or does. Rather, they reflect company culture and the backgrounds of the founders. "

THIS. In other words, the good ol' boy system still exists, only under the moniker of "meritocracy."

clbrook• 11 years ago

This article might help explain that: https://news.ycombinator.com/item?id=10659600

mercurial• 11 years ago

The company told us they valued process more than raw ability, and he’d not written tests during the interview.

Who the hell writes tests during an interview?

sharemywin• 11 years ago

people who want jobs at test driven development shops.

atom-morgan• 11 years ago

Or why not ask him, "Would you mind writing a few tests as well?"

bumbledraven• 11 years ago

Uh, yeah. That company sucks.

[edit: well, their interviewing process sucks.]

jenshoop4• 11 years ago

This was a great point: "There’s more demand for product-focused programmers than there is for programmers focused on hard technical problems." A very talented programmer from Dropbox once told me that if I wanted to attract top engineering talent, I needed to be able to show the engineer "a problem that no one else has solved yet." This totally changed the way I wrote my job descriptions and conducted interviews. Led to great outcomes, too.

random_coder• 11 years ago

Care to explain a little? Did your friend mean that talented engineers like working on novel problems and companies facing hard problems are thus more attractive?

lackbeard3• 11 years ago

I read jenshoop to mean: when recruiting, describe in some technical depth, the difficult, unique problems your company is solving to build its product(s).

karmacondon3• 11 years ago

When it comes to hiring, I think that the team is more important than the individual. As the beginning of the piece stated, it's not so much about individual characteristics as it is culture/process fit. I think the main takeaway for applicants is not to take interview rejection personally. Even if you do everything "right", you still might not be what they're looking for.

Ultimately, I don't think that individual programming ability matters that much. It's extremely rare for the technical talents of one person to turn a company around. Some people can inspire others and turn dysfunctional groups into great teams, and there's never enough of that. But said great teams are ephemeral, like sports dynasties. All a company can do is try to avoid toxic employees and hope that the magic happens.

vskarine• 11 years ago

I totally agree with you. A good, recently published book, to read on this topic is 'Team Genius'.

staunch2• 11 years ago

"I remember the first time I interviewed for a front-end programming position and got asked how to do something in JavaScript on a white board. The specifics are vague, but it’s crystal clear how stupid it made me feel and how little it had to do with the actual job."

...

"The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written, talking through bigger picture issues, and, if all that is swell, trying them out for size."

https://signalvnoise.com/posts/3071-why-we-dont-hire-program...

js82• 11 years ago

This reminds of psychological study about superstition. They studied fishing habits of South American Indian tribes; the tribes that fished in lakes, had consistent and reasonable rules about when/where to go fishing. However, the tribes that fished in the ocean, where you cannot predict the catch, had rather random and superstitious rules about when/where to go. This is due to our brains seeing patterns in randomness.

Also, it's perhaps understandable why companies don't want to reveal their preference. If you do it, you open yourself to being gamed. I think it's also better to have unreasonable requirements (looking to give genius programmers some boring product work) than be sorry. Again this happens in absence of actual test (other than wait and see) of who is a good candidate. Females do the same thing when mating.

ssalazar• 11 years ago

This reminds of psychological study about superstition. They studied fishing habits of South American Indian tribes; the tribes that fished in lakes, had consistent and reasonable rules about when/where to go fishing. However, the tribes that fished in the ocean, where you cannot predict the catch, had rather random and superstitious rules about when/where to go. This is due to our brains seeing patterns in randomness.

Thats really interesting. Would love to read more about it if you have a source for that off hand.

js82• 11 years ago

It comes from this lecture https://www.youtube.com/watch?v=P_dLJ1FxGuo&list=PLRKa53-za1..., around 10 minute mark.

lazyant• 11 years ago

So, random.

Also the company interviewing the first candidate, couldn't they just told him that they wanted testing?

trishume• 11 years ago

Indeed, it's an issue when interviewers believe that the best candidates will feel a compulsion to test their code without being prompted. I worked at a company that was big into TDD without ever having done it before, it was no problem and I wrote lots of tests, and so did everyone else on my team who didn't have a TDD background before.

On the job people will tell you if they expect tests, so why interview people in a different situation?

superuser2• 11 years ago

This is oddly reassuring. Imposter syndrome is practically guaranteed when there is a 99% chance that you only got (or could only get) a good job by a failure of the interview process, since so many companies proclaim to only hire the top 1%.

Goes to show that there is a far-greater-than-1% chance that you're legitimately in somebody's top 1%.

throwaway45678910• 11 years ago

I'm an "enterprise" programmer because I write in Java... I'm also older than the average age of a start-up employee (late twenties). I've never written a line of Ruby, and I've never written object oriented JavaScript or used Node.js.

However, I'm a really good programmer. I just happen to write the majority of my code in Java. If tomorrow we decided to use a new language, I could pick it up in a few days... I have a feeling the "enterprise" thing is just veiled age-ism. Anyone can feel free to correct me if I'm wrong, but at this point, it's literally impossible to learn every new technology. And anyone whose worth their salt should be able to learn a new environment and programming language without too much trouble.

I have a feeling that the enterprise thing is just a way for these employers to screen out the older, more experienced programmers.

This will probably get this throwaway banned, but Michael O'Church (who I am not) writes a lot about this on his blog. I tend to agree with him.

JustSomeNobody• 11 years ago

I think age-ism is part of it.

However, being an Enterprise dev myself (kind of), I feel that Startups don't think Enterprise devs "move" fast enough. They are too busy abstracting things and trying to get to the holy grail of the "perfect" architecture. Startups don't necessarily care about that right away, but enterprise devs have been steeped in it.

eldavido3• 11 years ago

Yeah well, that depends on whether you're used to working on a mature project with lots of structure and process (and usually, customers) or something newer where you're more in POC mode.

I'd favor someone who could do either but I guess if you have to pick one or the other I'd take "hack and ship" over "let's make it perfect", especially considering the second probably has lots of great options at BigCo.

sgarman• 11 years ago

I think a lot of companies want to hire people who also have programming as a hobby. It seems if this was true it would be very unlikely that someone would be a seasoned great programmer who only ever programmed in one language. I'd be curious why this candidate never wanted to try other styles/approaches.

throwaway45678910• 11 years ago

I never understood why that was a prerequisite. I program for 60 hours a week. So what if I don't want to do it on the weekends too.

taurath5• 11 years ago

Requiring programming in off-hours is pretty nuts, but in general if you're in a huge-stability big-corp programming job you're not going to be changing tools often and that will have consequences when you want to be hired at a startup.

ammon19• 11 years ago

Author here. Ageism is real, and we'll be analyzing it in a future post. But in this case I don't think it's ageism. I think it's big-company-ism. Startups define themselves in opposition to big companies, and take signs of big company culture as strong negatives (which may or may not be justified statistically, I honestly don't know). But there are great programmers at big companies in any case, and startups should figure out how to hire them.

silverpikezero2• 11 years ago

Startups should realize that this is a thoroughly idiotic notion, and should abandon it ASAP. The one thing that large software companies do well is SHIP PRODUCTS. A good software engineer is someone who can ship a product, even if its ill-defined or not feature-complete. This is a virtue. Small companies, especially startups, have a horrible track record for shipping software that in any way resembles its product plan.

At my startup, candidates from large companies are immediately prioritized in the hiring queue. This strategy has always paid off; there is a high correlation between large company experience and good software engineering practices.

cableshaft2• 11 years ago

How does that jive with the statistic in the article that engineers that have worked at companies like Google, Microsoft, Apple, Facebook, and Amazon pass interviews 30% more of the time. These are all very big companies.

If startups are so opposed to big companies, shouldn't having these on your resume be seen as being negative?

finance-geek2• 11 years ago

Funny thing is, despite Ageism, supposedly there is a massive "shortage" of developers.

ScottBurson4• 11 years ago

The first time I read your post I missed the parenthetical in the first paragraph, and was inclined to agree with you. Then I looked again and saw "(late twenties)", and had to laugh.

On the one hand, you're right, you can't learn everything. On the other, to be able to realistically claim that you can pick up new languages quickly, you should already know three or four different kinds of languages. If all you know is Java, you probably are farther away than you think from picking up a very different language like Ruby. I would encourage you to do some study on the side. Your late twenties is far too early to let yourself get pigeonholed.

throwaway45678910• 11 years ago

I am pigeonholed.

Sure I know other languages: Shell Scripting, Perl, HTML/CSS/JS, Groovy, SQL, C#, etc, but 80 percent of the code I write is in Java. I'm at a Java shop and have golden handcuffs. It's Java or bust for me.

qntty• 11 years ago

I'm also older than the average age of a start-up employee (late twenties)

I took this to mean that the average startup employee is in their late twenties, and he is somewhat older than that

nvader• 11 years ago

The parent could be stating that late twenties is the average age of startup founders, not that that they are in their late twenties themselves.

I do agree with you that someone who claims to pick up new languages in a week should already have picked up a few languages. The grandparent says they only do the _majority_ of their programming in Java, so perhaps they have that covered. I'd love to know what the other languages they've chosen are, and their dissimilarity to Java.

I don't know how to evaluate their claim that they're a really_good programmer, given that it's a throwaway account. But then, given the data of this article, I guess no one does.

alexwebb2• 11 years ago

Hiring manager here, and after many interviews with enterprise programmers, I do find myself putting more and more of them into the "no" pile right off the bat.

The problem is that those candidates, in my experience, tend to throw up way more red flags than others - their code is extremely verbose, they rattle off factory patterns like they're reading a textbook without any inkling what sort of problems they're actually solving, they parrot popular opinions without being able to articulate the reasoning behind them, they're not comfortable with developing a new feature without extensive documentation, and they miss the forest for the trees when it comes to things like testing and code organization.

Enterprise programmers usually leave me with the impression that they're heavily indoctrinated into one specific mode of thought, and that they would struggle to break out of it.

throwaway45678910• 11 years ago

It's funny you say that. I can't name one factory pattern off the top of my head... Never needed to learn one. I should clarify -- I'm in my late twenties, and I've spent my entire career (10 years) writing Java code. I don't understand how that negatively hiring managers against me.

sbov• 11 years ago

Most of this is fixable by driving YAGNI into their heads.

Beyond that, I would claim that most people startups look for (recent graduates) have no ability to understand the reason behind popular opinions. Because they don't have the experience to do so. They might be able to parrot it though.

Interestingly, I'm used to working in a pretty fluid environment without a lot of specs. But we recently started developing something with very tight rules and an extremely picky end user. It's much less efficient without a set of specs, which the project managers didn't have the constitution to make since we aren't used to doing so. As a result I basically have to constantly go back to them on things, which isn't very productive.

hoorayimhelping• 11 years ago

Enterprise programmers usually leave me with the impression that they're heavily indoctrinated into one specific mode of thought, and that they would struggle to break out of it.

That's been my impression as well, both coming from the enterprise and interviewing people from that domain. The most deleterious thing I've noticed is a lot of them, like GP, are convinced they're really good programmers and the reason they can't break out of the Enterprise is because startups are biased against some arbitrary and unimportant part of their identity (like their age), not their actual abilities and attitude. I say this as a guy who's likely half a decade older than a guy in his late 20s who thinks he's too old to work at a startup.

Joof2• 11 years ago

Valve calls this "beaten wife syndrome" and apparently it often goes away after a while (if not, they fire the new hire).

taurath5• 11 years ago

The difference in skillset and mindset in a large organization vs a startup is pretty huge. A large organization values stability and conformity (your software must work within usually a much larger picture), whereas in a startup the organization values speed and flexibility (large business changes or new data may come in at any point). The challenge when coming in from a big company is convincing people you can "hack" it to get things done quickly while still bringing the monolith of process and organizational knowledge from the big company.

theworstshill• 11 years ago

Anybody who expects you to know every new technology is a fool. The most important thing is being able to teach yourself the new technology and figure out its kinks.

darkerside2• 11 years ago

More likely it's just an inability to judge the programming skills of someone writing in a language you are unfamiliar with. My guess is that most YC founders don't write in C# or Java. Whether it behooves them to screen those applicants out is another question.

squozzer• 11 years ago

I work in an enterprise environment though my skill set is mostly on the admin side. Ageism may play a part in the bias against enterprise workers, but other factors exist as well - whether justified or not.

1) Over-specialization - the vogue for startups is to hire full-stack engineers, which to me reads as a desire to have one person perform the work of three. The constraints of a startup environment may justify such desires.

2) Risk-aversion - many of us in enterprise environments groan about all of the meetings, deferments of decisions to superiors / domain experts, and other risk-avoidance strategies, but in the end we like our butts covered just like everyone else.

3) Tolerance / preference for a certain amount of friction - some might call this a lack of urgency or an excess of complacency. Consider the problem of selecting a platform for your new project. In most enterprise environments, the decision has already been made for you - what is on the approved product / vendor list? Another is how to get a person who reports to a different chain of command to do their job.

All of these factors and others seem to exist at odds with the startup culture, which is a general term for the composite self-image of a startup's founders. Generally, they see themselves as pirates who plunder market share from the lumbering galleons of large enterprises, at least until they sell.

perlgeek• 11 years ago

age-ism for somebody in the late twenties? That sounds, ... dunno, exaggerated or even ridiculous.

I'd expected age-ism to start at the 50s or maybe 40s, but then I'm not in Silicon Valley either :-)

jakub_g6• 11 years ago

Can not find it now but apparently the average age of a developer in Facebook is way below 30 (27 IIRC).

I guess it could be true that in some startups if you're 29 and everyone is 22 you're not perceived a "cultural fit" (my evidence on it is purely from HN threads though)

kod• 11 years ago

If tomorrow we decided to use a new language, I could pick it up in a few days...

If all you've written is OO code, this is probably false bravado.

Chances are you're not going to be productive in Prolog or APL in a few days without some prior exposure to those concepts.

eldavido3• 11 years ago

I've written a lot of code in both Ruby and C#. Like a parent, I love both of them for completely different reasons.

From a pure language standpoint, Java and C# are _really_ well-designed tools. I call them "Cathedral Languages" after esr's "Cathedral and the Bazaar" -- both were designed by highly-qualified teams of experts who had the privilege of spending years designing them, after already spending collectively many lifetimes doing language design. For examples of what I'm talking about, look at C#'s implementation of generics. Look at the quality of .NET or Java's multi-generational mark-and-sweep garage collector implementations. Ruby has a lot of warts. It's slow. MRI/REE are children's toys compared to the CLR or JVM. Ruby's implementation of coroutines (blocks vs. procs vs. lambdas) is a huge mess, its concurrency story is terrible, it has nothing like .NET's async/await...I could go on, but I feel the point is made.

However...

It's easy, when using .NET and Java, since they're such "batteries-included" languages, to get into the mindset that "I'll never need another tool", which is _manifestly_ untrue. I am a committed polyglot programmer and I am appalled how frequently I encounter "cultural ignorance" within the big enterprise language communities. So I think the issue is more cultural (O'Church writes about this), one of trying to avoid "Java Shop Politics", with the associated ManagerFactorySingletonImpls, needing to write about 100 lines to get a single thing done, etc. The problem is the cultural baggage that many enterprise developers bring with them, not the language itself.

In my current project, I'm doing small-program development in .NET for a Windows LOB project. It's pretty refreshing. I get all the benefits of UNIX-style systems design, i.e.. small composable program elements, services, Git (TFS? are you kidding me?), proper CI/CD, and command-line automation, alongside the maturity of not having to deal with breaking changes on point releases of OSS libs, a thoroughly-documented, consistent BCL (.NET Framework 4.5), high-quality runtime, and generally less "hipster" culture. I'd highly recommend it.

schandur• 11 years ago

I've never written a line of Ruby

I say this exact same thing on my Linkedin profile: "I have never written a line of Ruby... but I bet you can make me if it comes to that."

coffeemug• 11 years ago

However, I'm a really good programmer.

How do you know?

throwaway45678910• 11 years ago

Plead see previous comment about my compensation. I don't think a company would pay someone that much unless they were really good.

balls1872• 11 years ago

I'll just point out (as a much older engineer than you), that this article is HIGHLY skewed to YC Companies, that have self-selected as not solving hard technical problems.

As a founder, being in YC is probably great.

As an engineering employee, it doesn't seem all that compelling to work for a YC company compared to a non-yc company. None of the benefits of being a YC founder seemingly apply to an employee of a YC company.

This feels like serious echo-chamber nonsense that has little applicability outside the world of startups (and more likely YC startups).

Glassdoor just listed the top companies to work for. https://www.glassdoor.com/Best-Places-to-Work-LST_KQ0,19.htm

Airbnb (YC company). First job i found on their careers page (note, required a lot of clicking...):

https://www.airbnb.com/careers/departments/position/90884

    Back-end, Front-end, Full Stack and Machine Learning engineers.
    Strong proficiency in any of: C/C++, Java, Python, Ruby-on-Rails
    Exposure to architectural patterns of a large, high-scale web application
    Rigor in A/B testing, test coverage, and other web best practices

Hubspot. Jobs page: http://product.hubspot.com/apply?gh_jid=86940

Back End: We write lots of micro-services, primarily with Java 8. Our APIs are RESTful and use the minimal Dropwizard framework, and we take advantage of Kafka, Spark, Hadoop for processing volumes of data.

And Zillow (not I skipped Facebook,Google, and Linkedin, and went to Zillow as it's the first Seattle based company listed).

http://www.zillow.com/jobs/openings?j=oXn70fwb

Strong experience with Java, Objective-C or C++

xiaoma2• 11 years ago

So... I worked with over 50 engineers prepping for their first SF/Silicon Valley tech interviews. And I worked at Groupon, interviewed at a bunch of YC companies (including one where I took a job).

In my experience pretty much nobody is taking Ben Horowitz's advice from The Hard Thing About Hard Things—hire based on strength instead of a perceived lack of weakness. Instead everyone seems to be looking for candidates who fill out all the boxes and have zero thumbs down in interviews. That coupled with extreme risk aversion means long interview cycles, and huge amounts of wasted time for some of the most talented employees and founders on the planet.

ammon19• 11 years ago

Well, they sort of are, just not explicitly. The standard process is to have 2 to 5 interviewers, each asking their own question, and pass everyone who get at least one strong yes, and no strong noes. This ends up with a random factor, but ultimately favoring people at least one real strength. The interviewer who sees this strength will fight for them. We've seen people with high skill variance (great at one thing, bad at others) pass interviews at a higher rater than people who are just OK at everything.

vonmoltke7• 11 years ago

no strong noes

This is the problem. It allows one interviewer with a pet peeve to torpedo an otherwise excellent hire. Where I work, whoever ends up on the wrong side of the majority needs to make a case good enough to convince the majority to switch. Being strongly in the minority is not good enough.

The culture of accepting high false negative rates leads to the "no weaknesses" hiring the GP was complaining about.

hibikir2• 11 years ago

Most of those interview questions, don't really have much to do with the way someone actually works. I just went through one of those SV gauntlets at a YC startup, and got a yes, but at no point I thought any of the interviews were any good: Not enough time to actually learn about the interviewer, or solve a problem that a recent graduate couldn't solve. I know some awesome people that wouldn't have passed my interview, and I know people I'd never want to work with that would have passed.

The whole thing made me see why the situation in SV is so screwed up: If we can't really ask for things that show real experience levels, and it's all about short term first impressions, set by interviewers that aren't even necessarily any good at interviewing, how can we improve?

I've always much preferred interviews that asked for the same amount of time, but where people could show a picture of their real output, instead of a little puzzle.

Imagine you are hiring a chef to man your restaurant, and what you ask him to do is get through Chopped: Give him secret ingredients, and ask to make a dish using all of them in 15-20 minutes. You'll be testing something: Some people will do better than others. But are the things you are testing really going to matter, in practice, when what a chef really has to do quickly is to execute a well known, practiced menu, along with kitchen management skills?

We don't test for the things we do at jobs, and therefore, we can't hire. Not a surprise.

sremani2• 11 years ago

>Ruby or Javascript. (The C# pass rate is actually much lower than the Java pass rate, but the C# numbers are not yet significant by themselves.)<< This makes me sad. I do not know who the caricature here is, the Enterprise programmer or the YC companies.

rjbwork• 11 years ago

YC Companies most likely. I've been doing C# for 3 years, none of which was in an "enterprise" company. Both were/are small software companies in a SaaS environment on the cloud (AWS/Azure).

I would laugh if a company rejected my proven ability to build giant distributed systems in the cloud because they were built in C#.

bjacks• 11 years ago

I'd really like to know what it is about a resume or in an interview that makes someone seem like more of a 'product programmer'.

Is it the particular things they have worked on at other jobs, i.e. their experience, or is it the way they talk about what they want to build?

I'd be really interested to hear how I can make myself sound like more of a product programmer.

zachrose• 11 years ago

Use white space and minimal typeface variation in all your written materials.

Describe past work in terms of what was made, not tech used to make it. Relegate tech buzzwards to a minimal "skills" section of your resume.

Use the words "make," "things," and "world" liberally. Say you got into programming to make the things that you wanted to see in the world.

Have examples of things, especially small things, that a non-technical person could use or see.

Have a BFA in some quasi-technological process like stone lithography or ceramics. Describe approaching software with a similar sense of craft.

vonmoltke7• 11 years ago

Don't forget to make sure your minimal skills section includes at least one language or framework that requires a Mac to use.

taurath5• 11 years ago

As someone who describes themselves as such and has had good success - demonstrate that you care more deeply about the success of the product than "technology" itself. Code is a tool to solve those problems - focus on what has worked and not worked in coding, rather than what is cool and exciting. Its more of a mindset thing than anything - spend enough time actually being interested in business problems rather than CS problems.

d0m• 11 years ago

Talk more about the product you built (What worked, what customers said, how was it improved overtime, what did you learn from customers using the product and talking to them), rather than going in detail about the technical requirements of the product (Used technology, improved loading time by n%, rewrote using a different framework).

And the reason why it's important is because startups fail because of bad value propositions, not lack of technical expertise.

natrius• 11 years ago

If you want a product focused career path, start getting "product" in your job titles. Every employer knows I'm a product engineer because "Product Engineer" is on my resume.

Like others have said, the rest is mostly about the way you talk about what drives you. Solving hard problems is nice, but demonstrating that you can think well about building something people want is what you should aim for.

sloanesturz• 11 years ago

I would love to see this kind of analysis (i.e. Junior Programmer vs. Child Prodigy vs. Rusty + Experienced) as it applies to hiring women. Are there biases against women with different experience and different "culture fit?" Would be a neat way to apply your data and your company contacts.

ammon19• 11 years ago

We will be doing this analysis

ditonal• 11 years ago

Keep all these spurious rejections and arbitrary hiring processes in mind next time pg or BigTech's lobbyists bemoan the desperate shortage of great programmers.

Fiahil2• 11 years ago

Almost no one passes all their programming interviews. This is true because of randomness in many interview processes (even great people are bad at some things, and an interviewer focusing on this can yield a blocking no).

Reading this is somewhat reassuring. I've been over some interviews lately and I had some trouble getting past the tech phone screen.

One particular rejection was very frustrating, because I've spend quite some time preparing for the interview. Failed the first one. Got a second chance. And I knew it was over about 5 minutes after it began. "Can you write an algorithm that would sort efficiently this k-sorted array with a complexity strictly inferior to O(n log n)?". Yeah. No. Neither my job nor my hobbies includes sorting almost sorted arrays of integers, and O(n) complexity calculations.

It really seems to me that, being hired by a tech company is just completely random. Tech interviews in general are completely random. "Just a numbers game"

taurath5• 11 years ago

Not completely random, but by my estimation being self-taught fully 40-60% of engineering organizations (trending higher as they get bigger) care much more deeply about academic CS than they do about whether you're able to ship a product. Those companies are looking for people who fit a mold (a generalized interface) so they can scale their teams out. Startups and smaller companies tend to value people who are flexible and can learn new things very quickly and are less likely to test you on alg analysis and are much more likely to throw you an unfamiliar problem and see how you solve it.

Fiahil2• 11 years ago

I agree, I've been far more successful with startups than with bigger organizations. The irony is, I have a Master's degree in software engineering, but I never gave (until now) too much attention to these common O(n) sorting problems. In practice, I would end up using whatever standard library's default sort function and it would be perfectly okay for most use cases.

I wish tech screens didn't rely so much on these exercises; I have limited brain space and I don't like to archive more important stuff.

CardenB• 11 years ago

It's not "completely random". It's a process biased toward false negatives with some variance.

I've managed to land some really stellar opportunities, and I've bombed several opportunities that are better, worse, and on par with my experience. Please get lucky and unlucky. The truth is, it's hard to tell which category you belong to.

It's best to just try your best and try not to get discouraged. Someone will notice if you're talented, eventually.

jpgvm• 11 years ago

At the end of the day that sort of interview process is a super huge red flag anyway.

You don't want to work with people that value that sort of knowledge over actual ability to ship good code.

Unless of course you are one of those people that cares about toy problems ofcourse, then you probably do want to work with other people like you.

imh• 11 years ago

If you're really concerned and want to change that, I had great success with one of those books of practice interview questions. It sounds cheesy, but (obvious in retrospect) simply practicing the reasoning patterns tested in that kind of interview helped me do well in them.

dustingetz• 11 years ago

How does compensation play into these results? The 9 categories will have very different comp profiles

UK-AL2• 11 years ago

Enterprise programmer bias? Could be higher salary expectations?

etr71115• 11 years ago

"To that end, we’ve spent the last two months doing detailed interviews with CTOs and lead recruiters at the top 25 Y Combinator companies."

How were you determining which YC companies were top?

soham2• 11 years ago

Hi Ammon/Harj, Thank you for that effort and laying it out. Very helpful.

Just so the readers don't miss the context: By definition, most companies referred here, I'm guessing, are startups. And startups will definitely want more product-focused engineers, in order to keep moving fast.

Interviewing in general, is closer to a date, than it's to a standardized test. The smaller the company is, the more pronounced that characteristic is. For even slightly larger companies, it's a different story.

When I was a Director of Engineering at Box, the engineering team was tasked with hiring 25 engineers in a single quarter. For several quarters. When hiring at that scale, it's hard to hire based on personas and elaborate preferences. At that point, process is more important. Anyone that meets a consistent process gets hired. There are always biases at resume selection, but those are only to benefit the later process, and not so much of a preference.

[About us: http://InterviewKickstart.com]

mquander5• 11 years ago

I just watched another startup in the same reference class grow at a similar speed, and I don't believe you. Suppose it takes 20 interview loops to hire one very good engineer, and each loop takes on average 10 engineer-hours. That's 200-engineer-hours per new hire, or about one engineer-month. That means if you have 100 engineers at Box (correct me if I'm way off base here, but 25 on 100 in a quarter sounds like a reasonable fast growth) you can hire 25 new engineers in a quarter while spending only 10% of your total labor recruiting.

The reason this doesn't usually work is because most engineers don't spend 10% of their time helping with recruiting. That is in my opinion an unforced error. They should.

soham2• 11 years ago

Not sure which part you're referring to, that doesn't work.

Average 10% spend by an engineer towards hiring is quite normal, from my experience. Obviously, it's average, so there are some who spend way more than 10%, and there are some who don't spend any. 4 out of 40 hours a week including phone-screens, on-sites and deliberations is easy to get to.

I'm sure you know, that a lot of hiring at that scale is systematized and (hence) heavy-lifting is done by non-technical recruiting people.

krmmalik• 11 years ago

I really liked how this analysis was done. Though i dont work in recruitment i have to work with CEOs often and notice similar approaches to hiring for other job roles as well. Way too similar in fact. I'd love to meet a recruitment company that is taking the same approach as TripleByte or perhaps TripleByte will aim to address other fields at some point. Either way its exciting.

darkerside2• 11 years ago

I couldn't help noticing that "Strong Junior Programmer" stands out from the rest. It as a category suffers from juxtaposition with "Child Prodigy Programmer". The "Strong" prefix sounds like a meaningless qualifier that only serves to blunt the label of "Junior". I have to guess that the reason companies select out of this category is that the candidates are perhaps not qualified.

marshray2• 11 years ago

Either they want the candidate to be physically strong, or more likely, they're really looking for some other qualities which they can't easily summarize.

moron4hire2• 11 years ago

This is why I don't really interview my sub-contractors. If I run into someone at a meetup that I think seems smart, I ask them if they'd be interested in picking up a small project. If they accept, I... give them a small project. No resumes. No demeaning interrogation. Straight to work on something real. And I pay them.[0]

If they do well, I give them another one. If they don't, I tell them I'm sorry but I don't have any more work for them.

Thinking more deeply on the issue, I think I want to know as little about a potential candidate as possible.[1] I don't want to bias myself against them. I've had a lot of bad experiences in my career and I'm sure I'm not perfect at keeping my prejudices to myself. I want to get people into the chair as soon as possible, get them working, and judge from the work. Once I see work getting done, it's really easy to ignore everything else.

[0] I don't think I can even make someone do an "interview assignment" for free. I'd be receiving something of value--their labor--but I'd have no accounting of it for taxes and I have no desire to try to figure out how to keep track of something like that.

[1] Yeah, I'll say this probably includes whether or not they have any experience with programming. If they can't do the job, we'll find out soon enough. If they can do the job to a satisfactory degree, why do I need know how long they've been doing it before? Give people a chance to surprise you.

20years2• 11 years ago

Very cool! Wish more companies approached it like this.

pcmaffey• 11 years ago

Are you analyzing performance / satisfaction after getting hired? Would be interesting to see distribution across types.

natmaster• 11 years ago

Why does this contradict the myth that silicon valley is ageist and only wants to hire fresh college grads?

thaumasiotes6• 11 years ago

We do our interviews without looking at resumes (in order to find great people who look bad on paper)

They don't ask for a resume, but the first thing that comes up in the interview is "where have you worked in the past, and what did you do there?"

ammon19• 11 years ago

So, we ask at the beginning to talk about a programming project that you've done it in past, but not about where you've worked. We don't ask for a work history until the end (by which point the notes and scoring that go into the final decision are over). We ask at the end so that we have the data to write posts like this.

thaumasiotes6• 11 years ago

Perhaps you've changed your policy? I'm pretty sure that's how my first interview started off. When I asked how that fit in with the advertising of "get a job based on something other than your resume", the response was "well, companies do care about which other companies you've worked at".

After the second interview, I was given to Harj and he asked me about work history again...

(Also, since I applied through the project track, I don't think there was much in the way of "tell us about a project you've done in the past".)

lewisl90292• 11 years ago

As a "Product Programmer" myself, I find it highly ironic that despite the fact that there is apparently more demand for product-focused developers than technical-focused ones, the interview process for most startups is overwhelmingly technical-focused.

If you're looking for product-focused developers, please consider tuning your interview process to evaluate whether or not candidates can build great products, rather than following the herd and grilling them on obscure algorithms and data-structure problems, which has a rather high chance of weeding out the kinds of product-focused developers that you're looking for in the first place.

tdaltonc5• 11 years ago

Maybe you only remember the super technical parts because the product-focused parts were effortless for you.

hawkice• 11 years ago

I've been in interviews where over half the questions were legit non-programming puzzles, and the rest were algorithms. Most interviews are heavy on tool-specific questions. I've had only two interviews where I've been asked to fix a bug in code and in those circumstances I wasn't able to actually run the code. Getting even remotely close to building something would be so supremely memorable that I would have a hard time believing I built an entire thing before I noticed.

axplusb• 11 years ago

I would be interested in A/B comparison of company responses according to immigration status (i.e. whether or not a candidate needs visa sponsorship). Any data available?

j2kun2• 11 years ago

I am very interested in which of these companies are the outliers. What is the one company in the table that expressed a dislike for product-focused programmers? Which are the few that like academic programmers?

ammon19• 11 years ago

I can't give company names, but we've not seen much of a pattern. It really seems to have to do with the backgrounds of the founders, and who the early successful hires were. This sets an engineering culture going forward.

steven20125• 11 years ago

What is the point of interviewing through TripleByte and going through their interview process, if I just have to interview with the YC company again? It's something I would completely avoid. I already know that almost 100% of the companies I send my resume to will respond, so what is the point of adding another set of interviews, which as the article points out, only adds a random level of success.

ammon19• 11 years ago

Several reasons: 1) people who go through us skip the screening steps and go straight to a on-site interview at the YC companies. 2) We've sent a bunch of people to most of these companies, and have a good idea what they are looking for. We help you avoid failing interviews for the reasons mentioned in this article. 3) Perhaps you have no trouble getting responses to your resume, but a lot of (strong) programmers do. We help them. 4) We help candidates negotiate offers.

steven20125• 11 years ago

I think there is probably value for inexperienced programmers, or people that don't know how to properly write their resume, but for experience programmers, it seems like an unnecessary extra step. Going through a bunch of interview questions from TripleByte just to skip a 45 minute phone screen is absolutely unnecessary and not worth it, especially since you will have to go through the entire onsite anyway.

As I mentioned, I had about a dozen interviews, with both YC and non-YC companies, and had no trouble getting contacted directly by any company.

asmdb• 11 years ago

I went through the initial TripleByte process earlier this year but was rejected. I passed the initial multiple choice programming test, but I failed miserably at the 1 hour programming exercise. I actually found the problem to be one of the more difficult algorithm type problems I've encountered in an interview situation. I am curious if the problems chosen by Triplebyte are representative of the typical types of questions for YC company interviews? I can't imagine they all have identical pre-screening processes, so how do you choose representative problems?

Mc_Big_G• 11 years ago

I've had 10+ interviews in the last month and it's really a shit process. First of all, I'm terrible at technical interviews. There's just something about being in that "we're watching your every move" environment that makes me freeze. I found myself literally reading the same for loop line over and over again, not even trying to comprehend it, but rather zoning out on it as my brain continued to focus on what I thought the interviewer was thinking. This, of course, gets worse as the seconds tick on and I've said and done nothing. I also perform terribly when I'm given a take-home "test" with a time limit. The pressure to finish on time overwhelms my thought processes. Then you have the asinine binary tree questions for a front-end web dev position or the "tell me what's wrong with this horrible, obfuscated code". If I have to work on code like that, I don't want to work there.

I had multiple interviews where I was either being video recorded (karat.io) or on a skype call with multiple developers "evaluating" me. This is a toxic environment and I knew in the first few mintutes I didn't want to work there.

I have a portfolio full of sites that I've built and I can tell you with confidence that my portfolio didn't matter even a little bit. Not one employer looked at my github account (not that it's impressive, but that says something).

The best way to get a job is through your network. A previous freelance client was hiring and pursued me aggressively. Why would they do that if I'm a shit developer as thought by my various interviewers? The experience of "interviewing" with a company that knows my work and shows a genuine enthusiasm for having me on their team is so night-and-day different from every other interview that the decision was easy. They gave me what I asked for and treat me like a valued employee.

TranquilMarmot6• 11 years ago

This sounds like exactly my experience as of late with interviews, so it's nice to know I'm not alone; unfortunately, I don't have a "network" and live far away from where any "network" could be set up so I'm a little stuck in the mud.

gedrap• 11 years ago

Well, you ARE in the network right now. On HN :) networking doesn't have to be physical.

lewisl90292• 11 years ago

I apologize for being completely off-topic, but I find it completely ridiculous that we have to scroll past half the page to see the next highest-level comment.

Are there any plans for implementing collapsible comment threads?

542458• 11 years ago

If you search, there are lots of good plugins and bookmarklets to do the job. Yeah, it would be nice to have it built-in, but people have been asking for it for years - I don't think it's likely to happen soon.

EDIT: Apparently posting really long strings without spaces breaks HN's layout now. Hunh.

Here's my bookmarklet for that:

   javascript:(function()%7Bfunction%20getScript(url,success)%7Bvar%20script=document.createElement('script');script.src=url;var%20head=document.getElementsByTagName('head')%5B0%5D,done=false;script.onload=script.onreadystatechange=function()%7Bif(!done&&(!this.readyState%7C%7Cthis.readyState=='loaded'%7C%7Cthis.readyState=='complete'))%7Bdone=true;success();script.onload=script.onreadystatechange=null;head.removeChild(script)%7D%7D;head.appendChild(script)%7DgetScript('//ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js',function()%7Bif(typeof%20jQuery!=='undefined')%7Bif(!$('body').hasClass('collapsible-comments'))%7B$('body').addClass('collapsible-comments');var%20span_html='%3Cspan%20style=%5C'cursor:pointer;margin-right:10px;%5C'%20class=%5C'expand-handle%5C'%3E%5B-%5D%3C/span%3E';if(window.location.href.indexOf('item?id=')!=-1)%7B$('center%20%3E%20table%20%3E%20tbody%20%3E%20tr:eq(2)%20%3E%20td%20%3E%20table:eq(1)%20span.comhead').prepend(span_html)%7Delse%20if(window.location.href.indexOf('threads?id=')!=-1)%7B$('center%20%3E%20table%20%3E%20tbody%20%3E%20tr%20%3E%20td%20%3E%20table%20span.comhead').prepend(span_html)%7D$('.expand-handle').live('click',function()%7Bcurrent_level_width=parseInt($(this).closest('tr').find('td:eq(0)%20%3E%20img').attr('width'),10);$(this).closest('table').closest('tr').nextAll().each(function(index,el)%7Bvar%20elWidth=parseInt($('tbody%20%3E%20tr%20%3E%20td%20%3E%20img',this).attr('width'),10);if(elWidth%3Ecurrent_level_width)%7Bif(elWidth%3C=inner_level_width)%7Binner_level_width=1000;$(this).hide()%7Dif(inner_level_width==1000&&$('.comment',this).css('display')=='none')%7Binner_level_width=elWidth%7D%7Delse%7Breturn%20false%7D%7D);inner_level_width=1000;$(this).text('%5B+%5D').addClass('expand-handle-collapsed').removeClass('expand-handle');$(this).closest('div').nextAll().hide();$(this).closest('div').parent().prev().hide();$(this).closest('div').css(%7B'margin-left':'18px','margin-bottom':'5px'%7D)%7D);$('.expand-handle-collapsed').live('click',function()%7Bcurrent_level_width=parseInt($(this).closest('tr').find('td%20%3E%20img').attr('width'),10);$(this).closest('table').closest('tr').nextAll().each(function(index,el)%7Bvar%20elWidth=parseInt($('tbody%20%3E%20tr%20%3E%20td%20%3E%20img',this).attr('width'),10);if(elWidth%3Ecurrent_level_width)%7Bif(elWidth%3C=inner_level_width)%7Binner_level_width=1000;$(this).show()%7Dif(inner_level_width==1000&&$('.comment',this).css('display')=='none')%7Binner_level_width=elWidth%7D%7Delse%7Breturn%20false%7D%7D);inner_level_width=1000;$(this).text('%5B-%5D').addClass('expand-handle').removeClass('expand-handle-collapsed');$(this).closest('div').nextAll().show();$(this).closest('div').parent().prev().show();$(this).closest('div').css(%7B'margin-left':'0','margin-bottom':'-10px'%7D)%7D)%7D%7D%7D);var%20current_level_width=0;var%20inner_level_width=1000%7D)();

jakub_g6• 11 years ago

FYI HackerWeb has collapsible threads:

https://cheeaun.github.io/hackerweb/#/item/10698009

It's designed for mobile web, but just zoom in and it should be fine

sagargv• 11 years ago

I wrote a bookmarklet for this a year ago and have since been using it every day: http://www.sagargv.com/proj/flathn/

DrNuke• 11 years ago

Pattern here is you are better if you start up instead of looking for a job. I've seen many and many mediocre engineers becoming employers just to spare themselves rejection.

iopq• 11 years ago

He solved hard algorithm problems like they were nothing

That's mostly practice. When I did webdev I was really shitty at algorithms because there were no algorithms in my daily work. When I did some Baduk AI programming I started being much better at algorithms since I was implementing some custom algorithms myself.

If you are interviewing someone for a web dev position, it's kind of ridiculous to screen people by their ability of merging sorted arrays or whatnot.

mavelikara• 11 years ago

You highlight a very important point. Doing well on algorithms in interviews only needs some dedicated practice. It is NOT a signal for higher intellect as many people misunderstand it to be.

a-dub2• 11 years ago

When I look to hire a lawyer, I'm going to evaluate potential candidates by asking them to write statutes from memory in full on a whiteboard under time pressure, because I really want to see how well they think on their feet.

As a forward thinking technology executive, I am certain this strategy is correct, because Google does it.

DrScump• 11 years ago

If you are going to hire a lawyer, you should be more interested in their knowledge of case law rather than how well they memorize statutes.

a-dub2• 11 years ago

We understand that our hiring process produces a lot of false negatives, and given that our applicant pool is already generally of high quality, we are able to fool ourselves into thinking that it has been a key to our ongoing success.

Perhaps more importantly, we're not going to change it because this is how Google does it.

fredliu• 11 years ago

I'm not a java programmer (my knee jerk reaction to quickly bang out an MVP would mostly be python), but this negative attitude towards Java seems unfair. There's a lot more to the Java universe than just enterprise Java. There are companies using Java that do cool stuff at large scale, and very reliable. Netflix is heavily Java, and nobody would (at least I wouldn't) argue that their tech is dull. Even if you are looking for pure hip factor, there's things like vert.x, and all these other JVM languages, which are not Java per se, but one of the selling points of those languages is it can utilize tons of libraries in the java ecosystem if needed. The last argument against java would be its verbosity and productivity (lack of), but I'd argue Java has one of the best IDE support among all languages, which helps alleviate the problem significantly.

staticautomatic• 11 years ago

It's what your data "show." Not what your data "shows." I am always skeptical of the analytical competence of someone who uses incorrect language to describe their own work.

Nadya2• 11 years ago

"Is `data` a singular or plural entity?" is a question you should ask yourself.

In this case their data is a collection or mass of data - and is singular. In the blog, using "the data shows that" is the proper way to write the statement.

chipsy• 11 years ago

This may be a British/American English gaffe. Collective nouns in British English are referred to using plurals, e.g. "Apple are planning a new product," vs. the American "Apple is planning a new product."

abvdasker• 11 years ago

"Reading bios of founders and applying to companies where the CTO shares your background is probably an effective job-search strategy"

My takeaway is basically that in an organization of sufficient size founders should have little or nothing to do with technical hiring.

bro-stick• 11 years ago

There is a subtle danger in only selecting candidate (people) whom think, act of look like you... the venture may end, not for a lack of talent or capability, but for a lack of ideas or questioning the normative, cultural convention.

Hire tortoises, hedgehogs and hares, introvert and extroverts, peacemakers and activists ... so long as there is respect, civility and productivity, because it's the "pulling" away from the center of gravity and institutional momentum that leads to exploring other great opportunities of venture successes that weren't originally founders' core product.

joshavant• 11 years ago

It feels like this article contains a huge amount of bias. To sum it up in a sentence... of course 'founders' all want 'product programmers'!

Ask only tenured, hands-on CTOs, and I bet that table would come out much different.

sytelus2• 11 years ago

The biggest issue in interviewing is the bad inexperienced interviewers. It's very very hard to be a good effective interviewer and in my experience even otherwise very smart programmers are not great at this. Here's few things that would help to find better match and eliminate lot of false negatives:

1. Pre-filtering should be based on candidate's public contributions at places like GitHub, StackOverflow, Wikipedia etc. Pre-conceived notions of someone being C# or academic is just simply bad.

2. Don't ask question that you didn't had to solve doing your job. It's absolutely the hardest part for interviewer to come up with great questions that distill the problem they had solved on the job in to something that can be described in 10 mins and worked on in about hour. Most interviewers are unable to do this and fall back to puzzles stolen from websites or coworkers like them. If you are not smart enough to form the great question using actual problems in your job than you have no business being an interviewer.

3. Don't do 45 minute interviews. That time is too short. Candidate has already taken a day off, there is no reason why you should limit each interview to 45 mins and make candidate rush. Countless great candidate get eliminated because they fall short of 15 mins, do longer initial chit-chat or just take one wrong turn.

4. Strongly discourage interviewers to look for exactly the right answers. Again hard to do than said. Most candidates that sail through problem have very likely practiced similar problem to death. The bad interviewer than penalize candidates who hadn't practiced that same problem VS who luckily happened to do so. Ultimately you are required to eliminate 80%-99% of candidates and this comparison is so handy that interviewers fall for it despite of knowing it.

5. Good interviewers knows the trickiest part of the problem and are willing to help out candidate without penalties. Bad interviewers considers any hints or help as sign of weakness and are mentally subtracting points. If you are an interviewer who thinks that your job is to give problem, sit back and watch the show then you are wasting everyone's time. Good interviews are lively discussion, two way conversation and in fact a collaboration.

6. Bad interviewers ask seemingly easy problem but that has chance of making a big tricky mistake or omission. Interview turns in to sport spectacle to see if gladiator candidate was able to duck the fire from the dragon that was behind him. Good interviewers make sure problems are real problem and not a competition to set up traps that everyone easily falls in to.

It's kind of bizarre that most companies claim that they have huge headcount to fill and they don't find talent while their "rejects" have already been having great jobs at great companies with nice history of career growth and compensation raises. The real problem is bad inexperienced interviewers who have been molded to ask same puzzles they had been asked and have been trained to have expectation for candidate to magically arrive at correct answer with error free compact code under 30 minutes. It's utter nonsense. The result is that most candidates now keep working on puzzles for months which has little relevance to actual problems in job and not even a remote indicator of candidate's passion or ability to take initiative or collaborate or ship something in real production consistently.

gizi• 11 years ago

I am growing less and less negative about recruiters. They can be a bit pushy, but I think that recruiters can learn how to modulate and not exaggerate. If a company wants good candidates, it will have to go to them, because the good candidates will not come to them. Good candidates are usually already too busy to bother. That leaves the company no other choice than to appoint a recruiter and try to take the initiative in contacting the good candidates anyway.

dennisgorelik• 11 years ago

There is a good reason why startups prefer Product Programmers:

If you focus on product, then you have strong feedback loop in your design and development cycle: you regularly see how users interact with your product and improve.

If your focus is on technology, then there is almost no real feedback and you are likely to optimize something not very valuable.

abrbhat• 11 years ago

I guess the reason that product programmers are most desirable is that most founders are product programmers themselves.

datashovel• 11 years ago

I think the recruiting problem in technology in general is that programmers don't meet and interact with enough different types of programmers in work settings, so they really don't know what they want or what they should be looking for.

I'd suggest everyone start doing this instead:

For anyone who meets basic criteria / filters, hire them at least part time, and put them to work. If they don't make the cut 3 months later (based on some objective criteria) let them go. Then after you've done all your recruiting like this maybe 1-2 years later evaluate what your team consists of.

It's not even a question in my mind. I'm not 99% sure. I'm 100% confident the team will not look like the recruiting team originally envisioned it would look before they started hiring.

EDIT: And I'm 100% confident those teams will consist of an overall better quality of engineer, and putting out higher quality products, than they would've ended up with if they stuck with trying to find what they believe to be the right "culture fit" or some concept of what a "prototypical engineer" is / should be.

Joof2• 11 years ago

Well if people really want product-motivated engineers, I'm going into the video games field. Also becoming a game designer.

I guess the ideal place for the technically motivated is in large companies like Google/Microsoft or in open source. Should we get a masters and find something more suited to our interests? I'm not sure what the answer actually is.

facepalm4• 11 years ago

"interest in ML as a negative signal"

Why, though? Is it because a candidate would be too interested in other things than focusing on mundane web development? Passionate about programming doesn't square well with passionate about ML, because the latter is not so much about programming?

cageface• 11 years ago

So companies want people that can build and ship products, and hire for them based on their ability to solve tricky CS problems on a whiteboard?

Something seems amiss here.

Animats• 11 years ago

This data needs to be related to the size and stage of the company. That YCombinator companies want UI people with application development experience reflects the YCombinator startup approach - it's all about the cool demo.

The requirements may be different in the later-stage companies. But most of the startups never get there. So looking at recruiting goals on a per-company basis from a VC pool will generate a bias towards the skills needed for the cool demo. What Lugg needs are people such as the article suggests. What Uber needs at its current scale is quite different. Uber will hire far more people than Lugg, but it's weighted the same in this model.

yks• 11 years ago

In this thread people stress how it is important to learn new things. And then how showing interest in a new thing (Machine Learning) is a red flag. Go figure.

jphelan• 11 years ago

I had such a disappointing experience with Triplebyte. I take their automated test and everything is hunkydory. After that they gave me a very ambiguous technical interview live-coding session without an interviewer. It seemed like an unreleased feature or a trial a/b test variant. Even now I go back and they've removed every reference to a "programming challenge"!

Anyway, the problem involved writing a tree generation/traversal along with a little equation parser and a lot of string parsing (I think, at least). I was really uncomfortable because while it said "we will run this code" I didn't know what their expectations where (In what environment are they going to run the code? Is underscore ok? They said "you can't use built in eval" so do they really want me to write my own eval? Do they care if I look on stackoverflow for string parsing stuff?). I only had an hour for a difficult problem and I spent most of the time wondering what they really wanted and stressing out about little details that wouldn't consume ten seconds of thought on the job.

I've interviewed a lot and I'm pretty confident from a lot of experience whiteboarding code or typing up an answer on the spot in interviews, so this was very frustrating and I found it to be disrespectful of my time.

pron• 11 years ago

So someone programming in Java at IBM or Oracle with a mostly academic experience would be all but completely unhireable by YC companies.

Incidentally, it was precisely that kind of programmers who built Watson at IBM, possibly the most impressive software of the past decade, which is not only both academically and technically challenging, but also brilliantly packaged and marketed and probably very lucrative to boot.

The exact same is true of the Graal team at Oracle, who have made what is probably the biggest breakthrough in compiler technology in the last decade, and might well power many important technologies in the near future (Ruby, server-side JS, R) as well as commercial Oracle products.

bluecalm• 11 years ago

The discussion about interviews comes down to a fact that if you setup your interview as a test where you position yourself as a chooser you will attract people who need you more than you need them. The best people are going to pass as they have enough options elsewhere.

For example I would be very happy to interview for Google 2 years ago and endure the process. Today I could sit with them for an hour and talk about stuff while sipping coffee. I have more opportunities than I can pursue anyway so spending hours solving mazes like some kind of a lab rat is not something I am going to spend my time on.

I am not a top expert in what I do but I can write and ship code. People like me (and especially ones better than me) will just not show up for your "process". Either you pursue them or you won't meet them. You will only get people for whom you are so attractive an option that they are willing to donate several hours or a whole day to have a low % shot. Those will not be the best programmers as the idea of paying to get evaluated is not something people with options entertain.

dlwj• 11 years ago

Employees these days have a paradoxical role to fill. On one hand, the value in an employee is the ability to solve a problem. On the other hand, they have to market themselves as passionate people. The only way to do so is to exclaim interest in interesting subjects which often are directly tied to company concerns.

Say a company needs good testers. I would find it very hard to sell myself as a passionate tester. Instead I would say something that's actually interesting to me like AI or machine learning.

In order for something to be interesting, it has to be somewhat unknown. How can you be interested in something you have completely mastered and know everything about. It just becomes process for you.

Who Y Combinator Companies Want

Related Stories