Every software company giving a take home coding test

https://twitter.com/realKetamine/status/1230127442733535232

by ritchiea • 6 years ago

42 117 6 years ago

Zekio• 6 years ago

Even better when they give you a project with tests in it, where the tests aren't even able to compile, and they then also want you to write tests which does the exact same thing as the non working ones

bberenberg• 6 years ago

What if it's paid? We pay people for interview projects where they have full access to our team and resources. This helps us gain an understanding of how they interact as well as their technical chops, produces value from most candidates, and makes sure the candidate is being compensated for using up their free time to work on this.

ghaff2• 6 years ago

You're still using their free time even if they're being compensated. In addition, there's a decent chance that, assuming you're in the same field as a current employer, they may be violating their employment contract by taking money from you. I know people will argue this isn't a big deal and "who will know?" but it seems an awkward situation to create.

fhbdukfrh2• 6 years ago

This is better for sure, but if you're trying to get someone really good to leave their current job (hint: They always have a job) it's as much the time as money so it still doesn't work well

_ea1k• 6 years ago

I once took a certification exam sort of like that. The goal was to make the tests pass, except for one random stage where the test itself was bugged. It was noted in the notes for that section, but not in an obvious way.

I was not impressed.

seanhandley• 6 years ago

A coding exercise should, ideally, be a starting point for a follow up discussion.

Then discuss things you would do next given unlimited time etc and justify those points. That's plenty insightful into what a developer prioritises and how they approach software development.

Expecting a perfect solution isn't reasonable.

ritchiea3• 6 years ago

That sounds ideal but oftentimes, intentionally or not, that isn't the expectation communicated. And in some cases I have seen take home coding exercises that are literally projects big enough for weeks of work. That particular small startup had copied their take home exam from a bigger company. They told me "work as far as I can," but it is intimidating and accomplishing a reasonable amount in say half a day still would feel like I had barely scratched the surface of the work they "suggested" I do "as much of as I feel like." It made me feel like an inadequate candidate by feeling like I can't possibly put enough of a dent in that task for it to be meaningful.

Oh and by the way the people asking me for that work were significantly more junior than me. I politely passed on the company and even beginning the take home test.

I have also had a company give me an API to build with sparse instructions. I did my best, I probably should have asked them more questions. I got no response. I followed up weeks later to see if there was more I could do with the coding exam or if I misunderstood it somehow. They didn't have anything else to say about the coding exam but they did invite me in for an in person interview. They then ghosted me and my further follow ups about the status of the hiring process.

The thing about the hiring issues in the industry is they aren't as simple as the solution you offered up. They are systemic and it doesn't feel like anyone is even trying to improve them. And that the changes that do happen are equally but differently absurd in their expectations as the mistakes of the past.

ghaff2• 6 years ago

Unless the problem is relatively trivial, it's pretty easy to see job candidates not wanting to turn in a quick hack job. But, if this test is early in the interview process--and it probably is--expecting a day of work (plus all the other prep/time) for a given interview is probably not reasonable. Which is a general problem. Assuming a couple of days to interview with a company might not be totally unreasonable in isolation. It starts to become so if someone with a job is interviewing multiple companies.

That said, there are certainly jobs where some sort of "audition" is clearly reasonable and even necessary. The arts are obvious. But I've held a number of positions where writing (and analysis, presentations, etc.) for external audiences was a big part of my job. You can be sure I want to see a writing sample and maybe even a presentation from a candidate for a similar role. If they already have some writing samples, that may be OK. But, if they don't, they're going to have to create one.

undebuggable• 6 years ago

"Afterwards we'll ghost you or send a rejection email without providing any meaningful feedback."

Dirlewanger• 6 years ago

Dearth of feedback regarding take-home stuff I've seen described as a liability issue, which I can kind of see. It really sucks, especially when you really thought you hit all the points, but I get it.

The ghosting phenomenon however is a cancer that can 100% be attributed to milliennials. It's wildly unprofessional behavior and didn't exist in any noticeable capacity before they attained positions of power.

looping__lui• 6 years ago

Well, I was ghosted plenty enough by “reputable” companies without millennials being in any position of power... Generally it’s poor HR performance or poor HR process which may (or not) tell a bit how serious the company is about HR overall...

seanmcdirmid2• 6 years ago

Ya, ghosting...what is up with that? I get a kid doing that, they might not know better, but a CEO of a startup?

subhobroto27• 6 years ago

How can candidates attack the ghosting problem?

Leave a comment on "Who's hiring" that "Hey, I got ghosted by this company?"

altmind4• 6 years ago

Ghosting candidates gives them a chance to come back with an offer in an indefinite time. 3 years after the interview: "hey, with your interest in this position, would you be initested in making a world a better place on a position of L1 support engineer, 250 miles from home, on a 6 month contract?"

mnm12• 6 years ago

A coding exorcise should only be done after the main interview and given only to finalists. Doing well should give one a huge chance of getting the job. The last few hires, we gave the test to three people and chose the best candidate after talking to all of them. Giving a test before even conducting an interview, a real one not a ten minute phone screen, is just an asshole move by shitty companies.

fhbdukfrh2• 6 years ago

Your approach doesn't save any time or effort for the hiring company though. It's the in depth interviews that take up so much time. You may not like it but this is their motivator, not being nice

logfromblammo• 6 years ago

The ideal is for the company and the candidate to have an equal investment in the process.

If I spend two hours coding some toy project, I expect the company to spend two hours talking to me. Either interviewing or constructive feedback relating to their rejection.

A company that wastes 1000 people's hours and then ghosts 990 of them should be embargoed by some kind of a computing and technology service workers' union. (You know... the folks who can herd CaTS.)

If the company feels justified in wasting a lot of your time in order to save a little of theirs, before the interviews even start, imagine what they will do after hiring.

mnm12• 6 years ago

Sure if they don't know how to interview, they're going to waste their own time and still not get good candidates (or if they don't have much to offer as most companies don't). Our interviews were an initial phone screen of I think half an hour followed by an interview with the team for one hour followed by the take home test followed by another half hour to an hour interview to discuss the code and allow for further candidate questions. All interviews are conducted over a conference call with screen sharing for the code review.

You're right. I don't like it because what you're describing is generally how shitty companies operate. Often they won't even look at the coding test. Those companies deserve to get the worst of the worst candidates that they're optimizing for. They fit into the category above: nothing of value to offer the candidate.

thih9• 6 years ago

Alternatively: ”How desperate are you to work here?” or ”Are you willing to spend your free time doing unpaid work?”

aphextron• 6 years ago

Take home coding tests are fine. I really prefer them, honestly. The problem is when you do the test, write great code, they love it and ask you to come onsite, and then you get white-boarded by a group of people who never even looked at your code or resume and get rejected for not being able to cough up a DFS algorithm on the spot.

ritchiea3• 6 years ago

I agree with you and there is nothing inherently wrong about a take home test. I do prefer taking home a test and working on a feature and talking about it with the hiring panel over whiteboarding. But the expectations they express about the take home work can still be poorly thought out and make me feel like I'm in the wrong for taking a longer time than they budgeted for me. Or that I'm being secretly tested to see if I will ignore the suggestion and throw myself at this exam to show my dedication to their company.

This joke is a suggestion there is still room for improvement.

ritchiea3• 6 years ago

Oh and I will add, I hate the criticism that a take home test is "free, unpaid work." So is looking for jobs and taking time out of your schedule to interview on site. It is inevitable that in a job search you put in unpaid time to try to find the best possible work experience. But it would be nice to try to find ways to make it less painful for job seekers and more accurate in assessing developers's real ability to be productive team members.

nostrebored2• 6 years ago

I'm always shocked by the examples that come up when people talk about when talking about whiteboarding. DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications.

cblades• 6 years ago

That there is a plethora of implementations of and documentation available for. There is no need to memorize an implementation.

subhobroto27• 6 years ago

Sure, DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications.

I have coded DFS many times in my life, both at high school and atleast a dozen times each during my Bachelors and Masters and far more times at work because I do a lot of NLP and traversing a tree is a regular day at work.

I can code DFS with you today, when out at a bar over a few beers.

What I struggle with is to code DFS:

1. with people I don't know, am not comfortable with

2. who are looking at me expectantly, expecting me to fail at it (because that's what happens in most of the interviews they have been part of)

3. with time running out

They only time I was able to perform DFS at an interview was at my first job out of college because just the month before I had prepared for my Data Structure final exam and could code DFS (or any of the HackerRank style algorithms) in my sleep no matter what even if all I had was a chisel and a slab of granite.

I was not upset that I never had to actually code the DFS, invert a binary tree and find the minimum coin change (all the 3 questions I was asked for the job) ever after that - it was all C++/Python/Java CRUD, for 3 years because I had 0 professional experience with CRUD at that point so I would have never got the job in the first place.

In fact the official policy at that place was to always use existing commercial libraries (and purchase them if necessary) because it was a CRUD shop and they had high turnover so any custom library would quickly become a liability.

mmazing• 6 years ago

I've interviewed several hundred people and gave them a take home test of a simple problem that we had encountered in our work.

We asked them to write a simple API that would return the closest pharmacy, given a latitude and longitude supplied to the API, and gave them a CSV with 10 pharmacies. Language didn't matter, and we never even ran it to see if it actually worked.

During the interview we would have them walk us through the code and explain what each part does and why they did whatever (this makes sure they actually wrote it and understand it).

Then, we would throw curveballs like "what would you need to change in your code if there were 100,000 pharmacies", and "how would you need to change it to return the closest 10 pharmacies instead of the closest 1?"

It's normally really difficult to get into someone's head about HOW they solve problems. I really felt like our method shined a light on their programming abilities. Also, how well they might anticipate future usage by building something smart to begin with and not code themselves into a corner.

subhobroto27• 6 years ago

Take home coding tests are fine. I really prefer them, honestly

Home coding tests are great at gauging real life performance of a candidate, if done right, but in my limited exposure (I don't have a lot of time to work on coding tests just so I can collect data samples to back my opinion), coding tests are handled extremely poorly, with expectations being set extremely vaguely.

Any realistic assignment requires a dialogue between the consumer and producer.

Companies are trying to minimize the time/money they spend on each interview.

Clarifications cost money.

Hoping a company will engage in a "what really are your requirements? are you trying to test my basic coding competency or do you want to see the best I have got?" will elicit a very vague response, if any.

If a company treats a home coding test as a real assignment and provides the candidate with the resources they need, I see no problem.

Unfortunately from my personal experience and anecdotes from others, it's just not the case.

get rejected for not being able to cough up a DFS algorithm on the spot

DFS is easy. BFS is easy. So is Coin changing, inverting binary trees and doing back flips.

If you invest enough time and effort into it, you can do it.

I have done DFS, BFS, Coin changing, inverting binary trees many times before: https://news.ycombinator.com/item?id=22368645

I am beginning to prefer DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews.

I have to learn it once and can regurgitate until I forget them.

I pay the heavy cost once upfront and amortize it.

No such thing for a home coding test!

I now have to pay the same cost over and over again, unable to amortize it because no two companies are solving the same problem unless you view it as a meta problem and in that case you might very well end up redesigning Django, RoR or a similar meta framework.

Financially it would be cheaper to become a full time contractor and get paid to make POCs and demos unencumbered by NDAs.

As a candidate a home coding test is extremely expensive compared to the traditional DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews.

dyeje• 6 years ago

I'll do the take home over a whiteboard firing squad any day of the week.

crimsonalucard7• 6 years ago

What does this say about people who survive and dominate the firing squad?

52-6F-625• 6 years ago

Only that they've practiced dodging bullets in order to get a job designing kitchen appliances?

Frost1x• 6 years ago

If businesses treated take homes with diligence and treated interviewees respectfully, I wouldn't mind reasonable take home assessments. This means the company is required to at the very least to send a "declined appplication" communication and not ghost.

Ideally, a business would give feedback as well but I understand there are people who quibble into lawsuits instead of integrating constructive criticism to improve themselves.

With the current lack of professionalism and respect in the hiring process industry wide, take homes as is are a good way to waste significant amounts of time when job hunting.

MrQuincle• 6 years ago

I don't like them.

However my brother is paying applicants for the home coding test. I think that's different.

Nothing beats (for me) an online track record.

eplanit2• 6 years ago

These stories are why I'm so glad to have spent my career as a consultant, where this kind of BS doesn't happen (in my experience, anyway). I wouldn't dream of applying at any of these places for this reason alone.

moron4hire• 6 years ago

It definitely happens at consulting firms.

subhobroto27• 6 years ago

It sure does but only because the consultant is now an employee.

I made an attempt at sharing how a consultant is an orthogonal role to an employee in response to our parent.

subhobroto27• 6 years ago

Fair enough but a consultant is an orthogonal role to an employee.

This thread is about employees being frustrated with the current state of affairs.

A consultant is a business owner.

As a consultant, my deliverables are rarely code. They are strategy documents, reports, plans, proposals, market surveys and video conferences.

As a contractor, my deliverables are mostly code.

Neither of them is similar to an employee's role, although, as a contractor, my deliverables match up with those expected of an employee.

Which is why it seems to me that you are conflating consulting with contracting.

Perhaps you meant to imply you love working as a contractor not having to deal with what employees have to but employees have to "deal" with it because their role requires it.

eplanit2• 6 years ago

As a consultant, I deliver lots of code in addition to documents. The two are not mutually exclusive.

It's sad that employees have to deal with BS like "whiteboard exams". Their role "requires" it only because their employers have followed that trend like lemmings.

So, yes I'm very happy to get hired (as a consultant) based on merit and proven experience.

I would not call it "orthogonal" (though that does sound smart in those documents), because it's more akin to a parallel than an orthogonal role.

s5ma6n• 6 years ago

It is usually very easy to criticize and not provide any alternative solutions. How should a company process 1000 resumes with a meaningful effort and find candidates to be called for an on-site interview?

If you think a single on-site interview of a candidate, it usually takes a few hours of multiple people from every segment of the company (mgmt, development, HR). Everybody has to prepare beforehand for some time for these interviews as well. So in the end, every on-site interview is costly for a company.

As an engineer, I dislike attending on-site interviews to hire people because I see it as a time that is not used optimally.

Finally, IMHO, the solution is to provide both standard-track and fast-track interviews where exceptional (on paper) candidates can be called directly for an interview and regular candidates would have multi-stage interviews.

varjag• 6 years ago

If you send out a 1000 4-hour take home tests as a screen and 100 people took upon it, you just wasted 396 man-hours of other people time. Creating a bunch of frustration, disappointment, and unnecessarily accelerating heat death of the Universe.

fennecfoxen• 6 years ago

Outside of Madoka Magica (2011), it's pretty safe to say that no human activities we undertake today will have any impact on the heat death of the universe whatsoever.

femiagbabiaka• 6 years ago

Fast tracks for interviews already exist, and they’re not good for the industry. They usually result in a pool of people shuffling from high paid job to job regardless of their competence, all on the basis of who they know. Making it an open industry standard would be disastrous.

An alternative: don’t give coding interviews.

nostrebored2• 6 years ago

Hiring is absolutely one of the best uses of your time in any reasonable environment.

If we're considering hiring for a position with the same expectations as your position, unless that time you've spent nearly doubles long term efficiency over the hours you spend to fill the position, it's easy to see why it's a good use of your time.

altmind4• 6 years ago

limit the number of interviews and only allow the immediate peers and managers to the interview? the people interviewing should be interested in a good candidate. that's enough.

mns• 6 years ago

Here we go again. As this seems to pop up quite often, it's seems like it's an important point for companies and candidates. I think that there are good take at home coding tests, and there are bad ones, there are good in person coding interviews and there are bad ones.

For me personally, I always liked take at home tests, and we are doing this at my current company. We have a small task with two levels, one to complete quite fast and the second level is if you really want to show off and spend more time on it. We use it as the main technical skills evaluation for the in person interview, because it opens up an actual discussion and we can talk about reasoning, see how candidates take feedback and how they can explain what they did. In the same time, there is another department with us that has a take at home task that I wanted just for laughter to try, it took me a whole weekend and it was insane. So I can see where some people are coming from. In the same time, we also had some cases where we paid the people (who requested it) for the time spent on the task, and we hired one, which was one of the most worst decisions that we made, because he turned out to be one of the most selfish and self absorbed people we hired.

In the same time I've been through some ridiculous situations where I was left alone in a room with no phone allowed, no pc, just write on a paper some SQL queries based on some printed tables from a spreadsheet, or, in another case to write on a flip-chart some code to print the first X prime numbers in front of 3 people. This I also find quite ridiculous, but it was 8-9 years ago.

In the end it's also how you feel as a developer about these things, if you are ok with take at home tests, maybe the company is a good fit for you and that is it, if not, you dodged a bullet.

subhobroto27• 6 years ago

We have a small task with two levels, one to complete quite fast and the second level is if you really want to show off and spend more time on it. We use it as the main technical skills evaluation for the in person interview, because it opens up an actual discussion and we can talk about reasoning, see how candidates take feedback and how they can explain what they did

This sounds so refreshing and atypical of the standard take home test that I would love to give this a go.

My email is my handle at gmail.com

No obligation. You don't even have to have a current opening. I want to honestly give this a go only to experience what this feels like. That would be value enough for me.

onion2k2• 6 years ago

Last summer I had an interview with a large insurance company for a front end role. The take home test was to produce 5 HTML+CSS+JS components to their existing designs (basically to take a PNG and turn it in to a web component). The components were a hero element, an information card, a notification banner, a list and a working calendar with selectable date ranges across different months. They were all to be made without any CSS or JS frameworks and they needed to be responsive. They suggested this would take about 4 hours.

I read through it and spent a couple of hours on it before abandoning the test and sending them an email to withdraw my application. If I was making those components properly I'd spend about week of solid dev time on them - 1/2 a day on each of the first four components and 3 days on the calendar..

Maybe I'm a terrible dev and very slow, but all that test really told me was that the developers there hack things together without spending time thinking about what they're doing, and on reflection I didn't want to do that.

simplecto• 6 years ago

I can respect that position. However, as a hiring manager I can see their perspective.

That perspective being, "How fast are you at good enough?"

Food for thought in the future. As a hiring manager I would have accepted all 5 done for 50% of all screen types, with developer comments on where you got stuck or what compromises you chose to make.

In my opinion that should be made clear in take-home tests. "You are allowed to cut corners, but please comment on WHY you did so"

ryandrake3• 6 years ago

Are you really looking for “engineers” who can half-ass a project from start to finish in 4 hours? That’s the quality bar?

If I were in OP’s shoes and feeling frisky I would point out to the hiring manager that for a real world project like that it would take at least 4 hours to just gather the requirements, write a design doc, and provide an estimate. 4 hours to do design, implement, and test something properly is ridiculous. And if you aren’t focused on doing it properly, then——wait, what company are you hiring manager at?

OP’s hiring company was an insurance company, not an Uber-For-Cats startup. The software he or she is eventually expected to write on the job is supposed to work.

twh270• 6 years ago

The trouble is that the determination of "fast enough" is provided by a senior developer who has 2 years of experience with that toolchain, and that's a ridiculous expectation.

Given a weekend, I can come up to speed well enough on an unfamiliar toolchain to start being productive. Given a month, my productivity is going to be high enough to be good enough for any shop that isn't looking for rock stars.

I interview candidates on their fundamental skills and ability to solve problems. If you're reasonably intelligent, can communicate well, and have logical and higher-order thinking skills, I don't care what tools you've used in the past. Many times I've recommended someone who has absolutely zero experience with the tools my team uses.

subhobroto27• 6 years ago

That perspective being, "How fast are you at good enough?"

That's a great question to ask.

However, I now ask you if that's really the question you are indeed asking an engineer who already has a full time job with possibly a hour or longer commute and is attacking your problem at the end of the day or week.

If that's the case, what you are really asking the question "How fast are you at good enough when you are extremely tired and nearly exhausted?"

Either that or you're selecting for candidates who are not challenged enough at work and work at a place that does not fill the rest of their employee's time with busy work to make up for the slack so they have time left over to work on your project.

You have thought of the problem from your viewpoint as an employer but have you viewed it from the viewpoint of an employee who's so frustrated with their current employer that they are willing to put up with hours of looking for job posts, doing background research on companies, filling out forms, following up on applications, answering random trivia questions and random take home assignments at the end of their full work day?

raxxorrax• 6 years ago

Incredible...

I think if you want to develop solid components, I don't think the time spend is high at all, on the contrary.

Right decision to withdraw the application in my opinion. I like it if I can gather some points with an assignment, but I would have mentioned their ridiculous time estimation and how they determined the required effort. If that wouldn't lead to a significant alteration of the task, the interview would already be concluded.

You pay quite high prices to have such components developed.

subhobroto27• 6 years ago

You pay quite high prices to have such components developed.

Exactly!

I would be VERY happy if I got the software I needed for the hours quoted on take home assignments.

I heard about this last year from a co-worker and became so intrigued that I started to fill out some applications to really see if it was the truth!

It indeed is!

What confuses me is why are managers falling for it?

One hypothesis is that they have bad data - something I commented on elsewhere on this thread.

ryandrake3• 6 years ago

I don’t understand these time limits measured in hours. What practical job skill are they testing? If it’s “we just promised a customer a new UI to be delivered working and tested by end-of-day!” then you dodged a bullet by withdrawing.

Maybe I’m just really lucky, but I’ve never had a grownup job where a new, never before seen engineering project was due in a time period of less than a few days.

megous8• 6 years ago

It's a home assignment, not an engineering job. It shows expectation on how much time at most you're expected to waste on it to get hired there.

Though to be fair, they should make the limit strict, otherwise they're selecting for people willing to do more for free, if they just then compare results without comparing the actual time spent.

megous8• 6 years ago

Are these tests supposed to reflect real-world programming? If not then the month view "calendar" widget has just three state variables: current view start, selection start, selection end, and the rest is just calculated from that.

4 hours for everything is a bit harsh, but a basic calendar view shouldn't take more than a few hours using just the today's web tech.

It will certainly take more hours if you try to make it accessible or compatible with too many browser versions, or start adding features like DnD range selection, or start optimizing, etc. But if company expected me to make a fully fledged usable component on a home assignment, that they can just use, I'd send them a quote, lol.

crimsonalucard7• 6 years ago

Why test real world programming when you can test for things that are harder and select candidates who can perform better than "real world."?

subhobroto27• 6 years ago

Are these tests supposed to reflect real-world programming?

That absolutely is the expectation.

These are not tests for basic coding proficiency.

These are not tests to see whether you can build a single component or widget.

These are tests to check if you can build a full suite of components or widgets including a README, tests.

... and candidates are apparently delivering.

I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.

Create new users, tasks, updates, votes?

Check Update users, tasks, updates, votes? Check

Get realtime notification of changes to users, tasks, updates, votes? Check

Apparently there are others who can not only do this but also implement a react frontend that puts the UI support for these as well in just 30 minutes more.

Either candidates are reusing templates they have built in previous projects and not reporting the time it took them to build the template when they report the time

Or, they are grossly under-reporting the time it took for them to complete the assignment.

We are doing this to ourselves. We should not deflect the blame.

If we feed garbage data to companies and then companies base their decisions off that, it's absolutely unfair to blame companies for not being realistic

crimsonalucard7• 6 years ago

I was once given a take home with unlimited time. They wanted me to give them the best possible solution. They would compare all candidates based off of the best work they could produce.

I went all out and wrote a solution that was point free. No variables. Now it is my default paradigm.

altmind4• 6 years ago

And you articulated what are you caring the most in your solution. Giving them a lot of food for thought.

b0rsuk2• 6 years ago

They told me they would mail me a programming assignment, giving me one week but a skilled dev would really only need 2 hours for that. I asked them "what contract"? And they were befuddled I expected to be paid for a week's worth of work. Anyway, I told them, in a non-committal manner, that I would take a look.

The assignment was to write a Django app (module) related to restaurants. Backend, templates, some light pagination, documentation, and at least 70% test case coverage.

I re-adjusted my priorities and did it at my own pace. I didn't bother sending them the finished work, just treated it as a programing exercise. It struck me that with such requirements it's basically production ready and they can use my work with perhaps two or three small adjustments, even if they reject my application.

Entitlement is high among programming reviewers.

subhobroto27• 6 years ago

Upvoted.

You are absolutely correct that this would take a magnitude longer to build from scratch.

a skilled dev would really only need 2 hours for that

Ah. I think that's the problem.

I suspect the "skilled dev" is keyword for "reuse existing project".

I can absolutely see myself reworking a Django app to support a backend, templates, some light pagination, documentation, and at least 70% test case coverage in 2 hours if I had an existing similar Django app with a backend, templates, some light pagination, documentation, and 90% test case coverage and I needed to rework it.

I think your competition cheated in reporting the total time it actually took them

subhobroto27• 6 years ago

You are not accounting for the third option: cheating.

I think there's a major disconnect in candidates wanting to be treated respectfully and companies that offer take homes

I have a suspicion that companies that offer take homes expect the candidate to cheat in some way or the other and begin with assuming the worst.

... or they are simply going off the data that they have.

I think majority of the candidates cheat by simply severely under-reporting the time it took for them to complete the assignment.

99% of the take home offers I receive are "our candidates typically take 2 hours to complete this assignment".

These are not tests for basic coding proficiency.

These are tests to check if you can build a full stack application - backend AND frontend including a README, tests and majority of candidates are typically taking 2 hours to complete.

I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.

Create new users, tasks, updates, votes?

Check Update users, tasks, updates, votes? Check

Get realtime notification of changes to users, tasks, updates, votes? Check

Apparently there are others who can not only do this but also implement a react frontend that puts the UI support for these as well in just 30 minutes more.

Either candidates are reusing templates they have built in previous projects and not reporting the time it took them to build the template when they report the time

Or, they are grossly under-reporting the time it took for them to complete the assignment.

We are doing this to ourselves. We should not deflect the blame.

If we feed garbage data to companies and then companies base their decisions off that, it's absolutely unfair to blame companies for not being realistic

pan69• 6 years ago

They suggested this would take about 4 hours.

Software developers are notioriously bad at esimating.

Personally I have started to refuse take-home tests and whiteboard puzzels. I now use it as an indicator of people I don't want to work with.

subhobroto27• 6 years ago

Software developers are notioriously bad at esimating.

While that's true, understand that the numbers companies are quoting are not esimates but rather actual time candidates self reported after finishing an assignment.

I expanded a bit more on this in a sibling comment.

Ocerge• 6 years ago

I've given up fighting white boarding. If anything, at least it can be gamed, plus if their expectations are bad or you got bad luck on a topic you didn't study for, at least you get to waste their time as well. I'll take that over wasting only my own time on a take-home test.

scarface742• 6 years ago

I live in a major metropolitan area not on the west coast. When I’m looking for a job, I change my LinkedIn status to “actively looking”, call a list of recruiters that I have worked with in the past, and usually have two or three offers in hand within two weeks to a month.

For whatever level I’m at at the time, usually companies pay within a narrow band ($10-$20K). Why should I waste time doing an at home coding test when I can get a job that pays just as much just by answering a few behavioral interview questions and describe my past career history?

I’m no special snowflake. I’m just your average everyday CRUD/SAAS software developer/architect/team lead depending on how the wind is blowing.

spuz• 6 years ago

Would you work for a company that spent 0 time vetting you? There has to be a "right" amount of time for evaluating candidates and the amount of time the candidate is required to give up when looking for a new job is only one side of the equation. Unless you want to work somewhere that has no hiring standards, you should not dismiss the time taken for take-home tests as wasted without evaluating their effectiveness vs other methods.

scarface742• 6 years ago

I had two offers before I took my previous job.

At Job 1: the interview was a white board interview where he wanted me to write a merge sort. He asked a few questions about my background and had to model a database schema and write a few queries to answer done questions.

The 2nd offer was after an interview where I had no language questions, he asked me about how I would design a development process “if I ruled the world”, methodology about dealing with legacy code and integrating it with a mobile app via APIs, my feelings about testing, what trade offs I had to make based on “business realities”, etc.

The first offer was full time, a shorter commute and paid more.

The second offer was contract to perm, if I translated it to an FTE salary it paid less (I ended up making more since I got paid by the hour), and it was a longer commute. They were both supposedly “senior software engineer” jobs.

I took the second offer. I didn’t realize at the time that they were actually looking for a dev lead. When they bought me on perm they matched the offer of the first job.

Are you really not able to tell someone’s experience if they have been in the industry for awhile just by talking to them? The truth is that most developers are “dark matter developers” doing yet another software as a service CRUD app or a bespoke internal app that will never see the outside of the company.

derivagral• 6 years ago

I effectively did this. No whiteboarding, no code stuff, just a phone call and an onsite with coworkers and boss. Small team, range of technical talent, but a much better fit for me compared to the larger (or smaller) standard advertising companies in the area that run 4-6 hour take home tests followed by a day of whiteboarding. You do not need to haze candidates, but you do need to test for basic ability.

As for vetting the company from my side: there's a bunch of mentoring and refactors that needed to happen with prior code, sure, but nothing game-breakingly broken. I've viewed this as a great mentorship setup, not as a "time to post another N+1 select to dailywtf".

subhobroto27• 6 years ago

Spot on.

I think there's a major disconnect in candidates wanting to be treated respectfully and companies that offer take homes

I have a suspicion that companies that offer take homes expect the candidate to cheat in some way or the other and begin with assuming the worst.

... or they are simply going off the data that they have.

I think majority of the candidates cheat by simply severely under-reporting the time it took for them to complete the assignment.

99% of the take home offers I receive are "our candidates typically take 2 hours to complete this assignment".

These are not tests for basic coding proficiency.

These are tests to check if you can build a full stack application - backend AND frontend including a README, tests and majority of candidates are typically taking 2 hours to complete.

I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.

Create new users, tasks, updates, votes? Check Update users, tasks, updates, votes? Check Get realtime notification of changes to users, tasks, updates, votes? Check

Apparently there are others who can not only do this but also implement a react frontend that puts the UI support for these as well in just 30 minutes more.

Either candidates are reusing templates they have built in previous projects and not reporting the time it took them to build the template when they report the time

Or, they are grossly under-reporting the time it took for them to complete the assignment.

We are doing this to ourselves. We should not deflect the blame.

If we feed garbage data to companies and then companies base their decisions off that, it's absolutely unfair to blame companies for not being realistic

crimsonalucard7• 6 years ago

Or: here's a whiteboard, implement some algorithm you last studied 10 years ago and that you will never use in front of 4 people staring at/judging you. You have 5 minutes. :D

I agree with the fact that this style of testing is not directly measuring the skills relevant to the job.

Say what you want, however, a candidate that does solid on the whiteboard despite the irrelevancy of it, is a good candidate because a whiteboard is one hundred times harder than the take home test and most of the problems that will be thrown at the candidate on the job.

mooreds2• 6 years ago

At a previous job we assigned folks a take home test, but paid an honorarium for their time. Then part of the interview was a code review session. Different projects for different experience levels, but never more than 15 hours total (usually more like 2-10).

Seems like the best compromise between flexibility for the employee, seeing actual real world work for the employer and allowing the candidate to shine.

fjfaase• 6 years ago

I guess this has to do a lot with hiring culture. Here in the Netherlands, these kind of tests are rare. You get two or three interviews, which are more about your programming experience and your personal strengths and weaknesses. It is as much about whether you fit the company and its culture. Sometimes you are given a tour through the company or are given the opportunity to talk to some of your potential new colleagues, and usually includes meeting your first responsible manager. If everything goes well, you get a job offer and a one or two month trial period, during which both parties can terminate the contract on any moment. In my life, I only once had to do some coding before a white board. I felt like not doing very well, but they thought it was great. The idea that you can select the best person for a job position by testing is not really based science, because there are so many factors involved that you cannot assess with some tests either performed on spot or at home.

BlargMcLarg• 6 years ago

To attack the take-home test instead of defending it:

The entire problem with these things: the more complex the problem, the longer the assignment generally takes. Half-an-hour assignments are trivial. 1 day assignments can range from trivial to absolute bogus. Now do these 10 times and congrats, you just worked a week or more for free, on things you're not even sure can be carried over to the future. Most of these companies don't compensate you for taking the tests and they don't assist you either (time is money, friend). Most of these take-home tests aren't universally applied, either: test A at company X is irrelevant to company Y, so they will give you test B.

If anything, this take-home test is a symptom. A symptom that companies have gotten lackadaisical thanks to their power, and a symptom that the industry as a whole needs to evolve. Don't punish capable devs because the guy you hired yesterday couldn't do FizzBuzz, and pay your talent accordingly (both in money and respect) so you don't have to resort to these practices in the first place. Most of all companies are there to take risk, not to give it back to (potential) employees. They get paid accordingly. If they dump the risk on us, then they should also lose a piece of the pie. Unfortunately, the reverse is happening: they have their cake and they are eating it too.

And a personal anecdote: most of the tests I've come across have nothing to do with the job at hand, and frequently the people able to clear these tests are overqualified for the job. Don't give a junior a take-home test made for a senior.

RutZap2• 6 years ago

This has really annoyed me recently, I've had interviews with companies that expected me to do a test before even seeing them in person (following a phone interview) and their tasks mentioned I should spend around 8 hours on the test. I think this is definitely unreasonable.

I prefer longer interviews where you're working alongside your potential future colleagues. This way you can also find out if you actually want to work with them or not. Interviews should be a two way affair.

b0rsuk2• 6 years ago

The phone and online coding exercises are called pre-screening. Their goal is not to find out if you're a good programmer, but if you can write code at all. This is to save time and see as many candidates as possible.

RutZap2• 6 years ago

Yes, i get the screening process; but asking a candidate to spend 8 hours on implementing a client library for their API at the beginning of the application process is definitely unreasonable. It's not a coding exercise, but a full blown deliverable.

thebigspacefuck• 6 years ago

I spent a good amount of time on one of these after a phone interview for a company I was mildly interested in. I’m confident it met all requirements, yet I submitted it and got no response one way or another. I could have put more time into it, but it met requirements and I thought, in the real world, would this company need me to make everything perfect or just good enough? However, in these take home tests where the implementation is so simple that submissions are judged on factors other than code correctness, it becomes an arms race between applicants and who has the most time to jump through hoops. I’d rather do a white board test and if I fail, have a chance to redeem myself with a take home test. In my experience it wasn’t worth the time I put into it. At least it helped me make a decision.

msluyter2• 6 years ago

I worked at a company for a while that had a pretty good take on this, IMHO.

First, the take home test was relevant to the sort of work one might actually do in a reasonable time frame. (Read a csv input file, hit an api per record in the csv, then merge that info and output an updated csv.) Could theoretically be done via shell script; my submission was ~100 lines of python.

Second, submissions were graded mostly for correctness. Style/structure and all the other stuff were nice but usually weren't deal breakers, iirc.

Third, the on-site included a session where we reviewed the submission with the candidate, asked questions, etc... This was pretty instructive, because we could point out a missed edge case or whatnot and see how they'd modify the code to handle it.

Fourth, names & personal identifiers were scrubbed from submissions, to avoid potential bias.

So, it was a better than average process, IMHO. That said, after reviewing a fair number of coding tests myself, I'd note that there are still a number of downsides to take home tests:

a) even if the instructions specify that one should take less than N hours, it's obvious that some applicants spend considerable time on them. So we'd see solutions of wildly differing quality; some with detailed unit tests, documentation, Dockerfiles, vagrant files (!), build scripts, etc... etc... some with just a single file.

b) Even though the primary criteria was correctness, there was some room for assessment of code quality, and subjective bias of the reviewer was undoubtedly a factor. A solution that missed an edge case but which was well structured would generally be fine, but "well structured" is obviously subjective. Or, although applicants could use "the language of their choice," those selecting, say, Java, were at a disadvantage. Most of us were Python/PHP/Ruby folks and the Java submissions looked generally bloated in comparison, and seemed to be graded more harshly.

Not sure what a perfect solution would look like. To resolve issue a) one could impose a strict time limit or use HackerRank or whatnot, but that would create time pressure and up the anxiety. For b) one could rely on purely objective criteria, though that may create other unforseen problems.

subhobroto27• 6 years ago

use HackerRank

Please dont. It's a terrible IDE.

My suspicion is that those who do well on HackerRank have invested a lot of time to be familiar with HackerRank.

Is that a valuable and meaningful longterm strategy and worth the investment for a developer?

If you were given an hour of paid time and asked to choose between:

1. learning something valuable and meaningful that would continue to pay dividends for the rest of your life, or

2. learning to use a website that you will probably use a dozen times or less your entire career

... which choice would you, personally, make?

The actual IDE I use at work is nothing like the HackerRank IDE. I have built up muscle memory that just does not work on the HackerRank IDE.

Have you tried to run a Javascript program within HackerRank? Try out how the Map implementation works and how it's different from the Map implementation available when you use Babel.

Timed tests on HackerRank select for candidates who have trained on HackerRank problems inside the HackerRank IDE.

If that's your perfect candidate don't come complaining that your employees solve HackerRank problems all day at work.

I have been doing software for a while and there has been no realistic situation when a business needed a problem solved from scratch without an collaboration, discussion or feedback within an hour.

I have been in situations where companies needed a solution within an hour and it almost exclusively involved extreme team work and cooperation.

msluyter2• 6 years ago

Yeah, I agree about HackerRank's UI. When I was using it I think I just used PyCharm and then cut & pasted my solutions.

remmargorp64• 6 years ago

As time goes by, I'm slowly becoming more and more convinced that the only way to accurately and fairly interview programmers is to give them some real world contract work assignments that match the kind of work you want them to do at the job position, and then see how they perform.

This approach has a lot of upsides:

1. Real work gets done

2. The developer gets fairly compensated for their efforts

3. You get to gauge them objectively, based on their actual results

subhobroto27• 6 years ago

Bang on!

I have been doing software for a while and there has been no realistic situation when a business needed a problem solved from scratch without an collaboration, discussion or feedback within an hour.

I have been in situations where companies needed a solution within an hour and it almost exclusively involved extreme team work and cooperation.

I made a comment on how it would be cheaper for the employer to hire a candidate on an at-will basis once the candidate passes a basic skill and design test: https://news.ycombinator.com/item?id=22368572

altmind4• 6 years ago

i would rather take a coding test than suffer an unknown number of office-visits, talking with people who are not interested in hiring, who are stuck talking with me and who are adamant to stop getting interrupted and get back to work.

as bad as coding interviews can go, they are better than spending more than a day to talk to people who evaluate you on how good you are at talking.

Every software company giving a take home coding test

Related Stories