When hiring developers, have the candidate read existing code

https://freakingrectangle.wordpress.com/2022/04/15/how-to-freaking-hire-great-developers/

by ewag • 4 years ago

996 535 4 years ago

freakingrectangle.wordpress.com

swayvil3• 4 years ago

You could ask them to show you their favorite personal projects. Have a conversation about that. Seems most natural and revealing.

forbiddenvoid• 4 years ago

Unfortunately, not everyone has time or desire to work on personal projects. Being an engineer outside of work should not be a pre-requisite to being one at work.

swayvil3• 4 years ago

That's pretty weaselspeak.

lucb1e9• 4 years ago

We did this in the first interview and they did well on that.

Then we gave an exercise* to write some code. In my opinion a junior (but not noob) programmer should be able to do it: the code should loop through a file that is too large to keep in memory and collect some statistics -- e.g. look up numbers associated with keywords and compute the average (where the keyword list is the large file; the lookup table is like ten entries).

This candidate (like the last one) instead worked around the memory constraint and just split up the file using external software such that they could fit the whole thing in memory. Not combining the partial results later or anything, no, just compute answers based on this one slice. Then in the interview afterwards, they could not describe how this could have been done differently, also not with hints (I understand they're under pressure during the interview). The code also had other problems (e.g. opening and re-reading this lookup file inline in the main loop... which runs a billion times). Even though functional (if not always giving the correct result; mistakes happen), they had neither a concept of memory complexity nor computational complexity.

So that's just anecdotal, maybe it was an outlier. From my perspective, talking about projects and code isn't always the same as them being able to work on the problems you're hiring them for.

* Note this was not entirely my choice, even though with hindsight it was a good idea we should at least have compensated their time regardless of the outcome imo.

jpgvm2• 4 years ago

Or any opensource project they have contributed to.

swayvil3• 4 years ago

Yes, that would do nicely.

Doing it because you choose to. Because you like it. That's key.

anyfoo7• 4 years ago

I'm not looking to interview, but if I would you'd find that my personal projects fall in categories of either being:

a) outdated, because almost all work relevant to my subfield have been at my job for a some years now (and my experience has grown accordingly, since), or,

b) irrelevant, because by now my personal projects are mostly orthogonal to my job (e.g. stuff involving electrical engineering and digital signal processing), which I am absolutely not applying for.

lucb1e9• 4 years ago

That's interesting, I've never heard testing code skills by reading instead of writing.

An example would have been nice though, as I'm not sure how to find a piece of code that does something standalone that is too large to grasp in 20 minutes yet make a reasonable prediction at the output. That combination seems kind of weird.

I wonder how well it would work to modify OP's idea and present a candidate with some code that has a bug in it (you reveal what the bug is to save time, they'll need to understand the code anyhow) and you ask them how they'd fix it. First broadly, like what approach, then actually write part of it.

To take my own advice and make the discussion more concrete, I should add an example. Random script from my github: https://github.com/lucb1e/randomdirectory/blob/master/random...

To prepare for the interview, one could add an argument like -h to show usage (in the same style as -v is currently implemented), then tell the candidate that the bug is that passing -vh executes neither -v nor -h (one would commonly expect it to execute both). Fixing this requires a 'structural' change to this little bit of code, but it's small enough to easily grasp and implement. The candidate might propose to loop over each character after a single hyphen to fix this, or they might propose to throw out this custom reinventing-the-wheel and use a proper arg parsing library. Either is fine, but they should be prepared to write the code (in any language, they can write just the changes in C, Python, JS, or whatever language they're comfortable with).

Has anyone tried such an interview coding question?

Buttons840• 4 years ago

Good idea. I'm not sure I know arg parsing libraries in every language though. Even so, hopefully talking about where to make the change and finding some good documentation on an arg parsing library would be good enough start for 20 minutes.

lucb1e9• 4 years ago

I'm not sure I know arg parsing libraries in every language though.

In my opinion, you should be free to look that up: that's what you'd do on the job as well. I also google it every time I need to use arg parsing in literally any language because I need it fairly infrequently, and frankly that's how this code came to be: too lazy to look it up again, too easy to write myself, and enjoying the writing of code. Any other solution would also be fine, this test would be about seeing if you can read and write code rather than if you come up with the (what the interviewers think) is the perfect solution.

nickcoury• 4 years ago

My favorite front end interview did this exact thing. The interview started with an issue taken straight from the Preact GitHub codebase. The interviewer provided the issue text and the commit right before it was fixed to pull down onto my machine. I had an hour to figure out how to build and reproduce the issue, Take in the high level structure of the code base, figure out how to drill down in a debugger to find where the issue itself was occurring, then propose a fix for it. Took the whole hour but was both extremely satisfying for me, and I imagine very insightful for the interviewer.

mhagemeister• 4 years ago

Preact maintainer here.

That's amazing! I didn't know that our little project is used in interviews. Happy to hear that you liked working with the code base!

developit• 4 years ago

This sounds fun

cldellow• 4 years ago

We used to have an interview process that had a take-home project.

The project was:

- you have two files: listings.json and products.json

- listings.json lists ~20,000 Amazon listings for electronics with fields like title, brand and price

- products.json lists ~1,000 digital cameras, with fields like brand, family, model

- your job is to write a script that, for each entry in listings.json, emits the best match (or no match!) from products.json

Your solution could be as naive or as fancy as you wanted. The main point was to have something for the next stage.

We'd run your submission, and use that to show you some false positives and false negatives. Then we'd ask you to debug why each false positive or false negative happen, explain it to us, propose how you'd fix it, and identify any trade-offs in your proposed fix.

Eventually, we wanted to offer a non-take-home-project interview. We already had a bunch of existing solutions from employees, so we used those to run a stripped down version of the interview that just focused on the code reading/debugging/proposing fixes part.

I think both of these interview approaches were pretty effective at giving candidates a natural environment to demonstrate skills that they'd actually use on the job -- debugging, collaboration, predicting downstream implications of changes, etc.

lucb1e9• 4 years ago

That's a great example that fits almost 1:1 with something we've been doing. I will propose that to avoid having to give take-home projects, thanks for sharing!

angarg122• 4 years ago

My company has an initiative to bring in students for a 3 day event where they attend workshops and talks.

We were brainstorming what type of content would make fun and educational workshops for students. I proposed something along OPs lines. I called them "debugging minichallenges". The idea is to present students with a buggy implementation for a simple problem, and they need to find and fix the bugs. Just as OPs argues, the concept is that reading and understanding code is a fundamental skill.

I have the set of sample implementations in my Github, for example the one in Python [1]. This is our first time running this workshop so I can report how well it did after the event.

[1] https://github.com/angarg12/minichallenge-flood-fill-python

pycache• 4 years ago

Took a quick look and was not impressed to find a pycache[1] in your repo (unless that's one of the bugs the candidate was expected to find :-P )

[1] https://github.com/angarg12/minichallenge-flood-fill-python/...

forrestthewoods• 4 years ago

This article would be much more interesting if they shared the code. They imply they use a new set of question every cycle so it should be safe and easy to share the examples used in a past cycle.

grangerg• 4 years ago

Those are similar to my own thoughts. I like also having them talk about their favorite recent achievements.

I can quickly train a person to have knowledge in some domain,

This part I have a little bit of trouble with. If you have a trivial domain, sure. In my experience domain knowledge takes a while to truly get your head wrapped around. But I expect that's why there's always other things that people do in an interview.

travisgriggs3• 4 years ago

Makes sense to me in general. We all spend much more time reading our own code and others', much more than writing code. In fact, first period a new hire is going to experience is spending a disproportionate amount of time, just getting to know the system(s) they're expected to contribute to.

I'm currently looking for someone to join our own team, I'll be taking this adjusted priority in mind.

gaoshan4• 4 years ago

This is exactly what I started doing at my company a few years back. You are presented with a complete app written in the stack we use. This app has some bugs we will solve to get it working (nothing that's a trick... actual, commonly encountered bugs that have all the error messaging you need to solve them). Once it is working you will walk me through a particular flow of the app, explaining what is going on and why we do it this way. There are optimization opportunities (purely optional, there for you to notice), areas we can dig into if we choose (or if I feel the need to). If you tear through it we can examine a different flow in the app. Perhaps we could discuss the database structure and how it might be improved or changed, maybe we dig into some CSS or into some GraphQL or any other aspect that we need to. I've had people stumped and unable to continue but the vast majority of devs can make a good stab at it, even if they are not familiar with the specific stack. The best devs are barely slowed down by such unfamiliarity and can still reason, logically, about the code.

So far I've had really positive feedback from job candidates. A couple even described it as their favorite interview ever! I feel like it has worked well, given the people we have ended up hiring.

Aeolun• 4 years ago

How long is this interview? I can debug a random React app in 30 minutes, but a whole stack may be pushing it.

gaoshan4• 4 years ago

1 hour. The initial bug takes most people about 5 minutes or less (the console error practically tells you what to do if you actually read it)

HorizonXP• 4 years ago

Are you able or willing to share some samples? I've been shying away from doing technical assessments the "old way," instead focusing on having a discussion about a candidates past projects. I feel like your way could be a good chance to get insights into someone's problem solving and critical thinking skills, which is truly what we value as a dev team.

8note• 4 years ago

I've had one given as a thought experiement.

List as many different ways as you can for how to figure out if a singly list is broken and has a loop

gaoshan4• 4 years ago

I can’t share the code, unfortunately. It’s a full stack CRUD app with some third party API integrations. You’ve got it! Gaining insights into their problem solving and critical thinking is a primary goal (more important than fluency in the various frameworks used).

88913527• 4 years ago

I only once ever had an interview as you describe, and from a candidate perspective, it was my favorite! I had fun with it, and it was far more collaborative with my interviewer. I got to learn from them, instead of just regurgitating straight CS knowledge. It was all around a more human centered process. I got to learn that I would enjoy working with my interviewer, for example (this was a smaller company, so probably less applicable at scale-- unless you can continue to have the interviewers be candidate's colleagues, at scale).

andrewflnr9• 4 years ago

"Bugs we will solve" seems to imply there's still a coding component. Is that correct? That's a different thing from what OP describes (and FWIW I'm pretty sure I like yours better).

gaoshan4• 4 years ago

Yes, that’s fair. However the bug is really trivial to solve and does not require much beyond reading the console error and then making a tiny change to the server code.

gcheong• 4 years ago

I like the sound of this approach insofar as it sounds more like a collaborative effort than simply trial by fire.

jrochkind1• 4 years ago

because reading is easily an order of magnitude faster than writing.

Hmmmm, really?

jbluepolarbear2• 4 years ago

Is it not for you? I’m at least 10 times faster at reading code than writing code.

xboxnolifes• 4 years ago

Does "finding the source of a bug" and "figuring out the correct thing to write to fix a bug" fall under reading the code or under writing the code?

I feel like the answer to those questions determines which is faster. One may say that you only compare (lines_read / time_reading) and (lines_written / time_writing) to find an answer, but if we remove all "time spent thinking" from both then it kind of feels like a meaningless comparison.

nathias• 4 years ago

I think it's takes about the same effort when you're trying to read not just skim (which is probably what you mostly do when coding) ...

bspear• 4 years ago

Really like how this makes the interview simulate day-to-day work

johnqian2• 4 years ago

I sometimes ask candidates "what kind of interview do you feel would best bring out your strengths?" and try to adapt the interview to their response if I can. It's helpful if they want to talk about side projects or war stories, but doesn't pressure them to. I still give my coding challenge after. Wonder why no one else does this.

eBombzor• 4 years ago

That sounds terrible. That just implies, "if he doesn't have an amazing interview now, he's definitely an idiot".

lucb1e9• 4 years ago

On the contrary: I like the idea, but for context, my most recent HN comment contained:

[...] That said, I do find [it] hard in interviews [to talk] about past experience[s] and prefer to just be given a web application riddled with every type of bug and try to bingo them all in the time given to show that I understand them all. That one was my favorite interview.

Now I see johnqian suggesting to ask interviewees how to best see their skills, and it sounds great. If the interviewer doesn't have something like this on hand, we could still go into different types of bugs that they care about.

I've also had interviews that went terribly, particularly when no technical details were asked at all for a technical position (e.g. a manager just asking the standard questions where you need to give platitude answers, such as "why are you specifically excited to intern with Vodafone", methinks: "because I want that degree and you were the only option I could think of within 45 minutes commuting"... but one can't say that so I made something up.. he probed further on that... it went downhill from there), so being able to steer away from that sounds great.

But it might also be that I would feel unprepared for such a question and give a total non-answer, helping neither party.

Asking this in an email ahead of time might be better, though then the effect you're suggesting might be even stronger.

Edit: I'm not the downvoter, to be clear. I think you raise a good point even if I am not sure whether it would necessarily be that way.

burnished• 4 years ago

What? Can you explain how that is terrible? I fear that the implication you are deriving says more about you than it does the method.

quantisan• 4 years ago

how do you ensure consistent criteria between candidates and avoid unconscious bias?

johnqian2• 4 years ago

The standard coding challenge is still most of the interview, so that provides some consistency. I just want to give candidates an additional opportunity to flex, in case they feel there's an important thing that most interviews neglect. Most candidates never really thought about it, and that's totally fine, for them I just do the standard course.

As for avoiding unconscious bias, was that ever an option?

anyfoo7• 4 years ago

It's helpful if they want to talk about side projects or war stories, but doesn't pressure them to.

I like leading with that, and am a big fan of the initial "what's an interesting problem you had to solve?" question, but I made the unfortunate experience that even people with good war stories are sometimes very shaky in the basics of the particular subfield I'm interviewing for. To the point of deeming someone not a good match for our team, but encouraging them and internal recruiters to interview with other teams.

There is ultimately no full substitute for getting down to brass tacks and actually do some "sample work" together. Whiteboard coding is the usual (and, I guess, often poorly implemented) approach, but I've used reading existing code, with added intentional mistakes, to great success so far.

EDIT: Just read your new reply in a sibling thread that you're doing that anyway, and project and war stories are supplemental. It seems like we're in agreement, then!

rTX5CMRXIfFG• 4 years ago

As an interviewer, you should know what strengths to look for and how to test for those strengths.

angarg122• 4 years ago

When hiring at scale we value consistency so this wouldn't work well. However in my company we started an initiative called "candidate experience". In one of the interviews we allocate some time for the candidate to "flex" as you put it. We ask open ended questions such as "what is your proudest achievement?" or "is there anything you wanted to tell us but didn't had a chance to?". These questions are not used to evaluate the candidate and often are no even recorded, and gives a chance for the candidate to relax and put forward their best self.

jarofgreen• 4 years ago

these questions are not used to evaluate the candidate
gives a chance for the candidate to ... put forward their best self.

How does that work? If it genuinely won't be evaluated, what's the point of getting their best self? How do you make sure things you learn in this don't affect your judgement later?

_vdpp• 4 years ago

I really like this.

michaelrhansen• 4 years ago

This is my standard practice - I give a real world data fix script that has been simplified to understand and comment on in 20 minutes. The interviewee is the code reviewer before it gets run in production. It covers things like performance, security, typical syntax mistakes, and working with sensitive data. The best part is actually not the script, but the stories it triggers about past challenges.

lytefm• 4 years ago

The interviewee is the code reviewer before it gets run in production.

Yeah I've also been using this approach for Fullstack Devd: A small page with a bit of CRUD + a small ticket description of what the page is supposed to do.

The code contains various bugs or questionable implementations, the interviewee is supposed to analyze the code and to either fix the issues right away or to write comments.

Nobody is expected to get everything right within the time slot, but I've found it to be a great test on how a candidate might perform in their day to day work.

zeroonetwothree3• 4 years ago

I think it’s fine to have this as part of your interview but I would be worried to have it as the only thing.

lkrubner6• 4 years ago

My one big tip: One vice that I see among hiring managers is an unwillingness to ask tough follow-up questions. If you ask a question and there is any vagueness in the answer, you need to drill down deep until all vagueness is eliminated, so you understand exactly what the person knows. Follow up on what's said, but also follow up on what is not said.

Here’s a real-life example. I asked a recent applicant (for a fullstack software job, where we were hoping to hire a novice-to-mid-level engineer):

Me: How would you improve a situation where a page is loading slowly and you suspect the problem is related to the database?

Applicant: Well, I’d start by checking the HTML, is it correctly done, and then the CSS, is there any redundancy? And then the Javascript, is it correctly written, is it minified? Can we speed that up at all? Check the timeline, the API calls, see what is slow.

Me: Okay, great, that’s a good start, but what else? If the problem is not on the frontend, then what?

Applicant: Uh, well, then, I guess I need to look at the backend database model code. Is my database model code concise? Am I fetching the data needed, without any excess?

Me: Okay, great, that’s a good start, but what else?

Applicant: Uh, what else? Well, uh, we really need to look at that database code. Is the model bloated? Can we slim it down?

Me: Yes, okay, you basically said that already, anything else?

Applicant: Uh, well … uh, you need to check the HTML and the CSS and the Javascript and then, uh … API calls … uh ... the model code, make sure that is cleaned up. That needs to be lean.

Me: Yes, okay, but you said all of that already, anything else?

Applicant: Uh … well … the model code … and uh …

Me: Have you ever worked directly with a database?

Applicant: Uh … not much?

Me: If you get unexpected results from your model code, do you know how to debug the query?

Applicant: Uh … I guess I could … not really.

Me: Have you ever looked at the “slow query” log?

Applicant: Uh … no?

Me: Do you know how to run EXPLAIN or ANALYSIS?

Applicant: Well … uh …. no.

Me: Have you ever written SQL by hand?

Applicant: Uh … no.

Me: Are you aware of any differences in dialect between the SQL of MySQL and the SQL of PostGres?

Applicant: Uh … no.

Basically, they were somewhere between a novice level and a mid-level engineer, so they knew the frontend reasonably well, but they didn’t know a thing about databases. Which was okay, because that was what we were looking for. We still hired them and they turned out to be great in some areas, and they were eager to learn about the things they didn't already know. But obviously, if I'd been hiring a senior-level engineer, and it turned out they knew nothing about databases, that would have been a problem. The crucial thing is that I kept asking the question, over and over again, until I had the full answer. In this case it was easy, but sometimes it can feel aggressive, asking the same question over and over, which can leave either you or them feeling uncomfortable. But you will never be any good at interviewing people until you learn how to tolerate uncomfortable moments. The goal is to find out if you want to hire someone, without wasting their time or yours. And asking questions like this, directly, and digging deep, is a much faster method than handing out homework assignments and then waiting a few days for them to complete it, then reviewing it yourself, then discussing it with them. And such direct, factual questions, as above, are at least as objective and as any “objective” test that you might invent.

jameshart2• 4 years ago

This is a "Guess what I'm thinking" question.

Non-psychic candidates will struggle with this challenge, so you might find it produces some false negatives.

Since you already seem to know so much about debugging slow performing database queries on your particular RDBMS stack, though... what was your thinking in looking to bring on board someone else who duplicates that exact knowledge?

Oh, and it'll be a missing index. It's always a missing index.

lkrubner6• 4 years ago

The point is to start with something open ended, so the candidate can talk as much as they want on the subject, broadly, and then, based on what they say and what they don't say, I narrow in on particulars:

"Are you aware of any differences in dialect between the SQL of MySQL and the SQL of PostGres?"

It's not like they lost points for the stuff they didn't know, I simply wanted to be sure I understood the limits of their knowledge. We hired this candidate and they turned out to be great. But I hired them knowing exactly how much they knew and how much they did know.

abalaji• 4 years ago

The premise here is good, but instead of asking the same question again and again, I would rather supply more and more information until I get the answer I am looking for. (and jot down what prompting was required to get that answer) I agree with the sibling comment, otherwise this becomes a guess what I am thinking exercise.

lkrubner6• 4 years ago

But I wasn't really looking for any particular answer. This was not a quiz. I was trying to figure out what they knew, and what they didn't know.

ipaddr• 4 years ago

If you wanted to check if they can write a query than ask them to create a simple select or insert statement. The drilling down turned the interview into a hostile conversation. Easily ways to test that skill.

lkrubner6• 4 years ago

But I wasn't really looking for any particular answer. This was not a quiz. I was trying to figure out what they knew, and what they didn't know.

aunty_helen5• 4 years ago

I had an interview like this for a senior position, it turned out to be the most enjoyable interview I've ever done. The group lead really showed himself and found out at what point would I stop and say I don't know.

There's nothing aggressive about this approach unless you make it that way and finding out if people will start bullshitting or admitting the limits of their knowledge is a massive advantage to someone you might need to depend on.

lkrubner6• 4 years ago

This is a huge part of it:

"finding out if people will start bullshitting or admitting the limits of their knowledge is a massive advantage to someone you might need to depend on"

One of the best hires I ever made was a former K-12 teacher who had decided to change careers and become a software developer. She went to a bootcamp and then I hired just as she graduated from that bootcamp. When I interviewed her, I very much appreciated how clear she was about what she knew and what she did not know. She felt no need to bluff. And that foreshadowed what our communication was like once I hired her: very straightforward, with no lies or bluffing or indirections or deflections.

0x20cowboy• 4 years ago

I think we should just jump to the end game and make interviewees do ultra man triathlons while solving differential equations. Then at the finish line, they knife fight each other to find the one we let go to the next stage in the interview. That way we can be sure who has stamina and the best competitive programming problem solving skills.

Something like squid game will truely find the A players. I mean, we wouldn’t want a bad hire after all, and everyone lies on their CV.

(Obviously /s)

andrewflnr9• 4 years ago

Right, if you're a big company you can more easily afford to have false negatives than false positives, so why not add knife fighting to the list of qualifications, just in case?

mhh__• 4 years ago

arguably worse for smaller companies. if you hire the wrong person you can't even try and move them somewhere better suited to them

yitr• 4 years ago

post: have candidates read code

and this is your response? did you come up with extremely clever reduction in the shower and wait for the first post that's tangentially relevant (and in fact embodies the opposite of what you're attacking?)

anyfoo7• 4 years ago

I did the reading of existing code thing in interviews a lot a while ago (and somehow forgot about it when switching teams, this reminded me to reintroduce it), with the probably common twist of having intentional mistakes at various levels in the code, and it was absolutely great.

I feel it really gives you a good impression of the level of experience the candidate has with actual coding, and opens up for a lot of related discussion, such as security implications. Without, as the article mentions, the tediousness and artificiality of whiteboard coding (which I don't want to completely dismiss, however).

We're going to be working on a lot of code together, so it makes sense to have the interview about doing exactly that.

monksy2• 4 years ago

The only unbelievable thing in your statement was "oviously /s".. I read that and thought.. someone is taking notes.

quickthrower22• 4 years ago

I interpreted squid game as an analogy for real life. If you staying alive depends on money which depends on having a job, then algorithmic interviews are a mild squid game after all.

legel• 4 years ago

I propose, as well, that we “weed them out” with a week of hunger games, battle bots, and rocket launch tests.

In seriousness, through a decade of experience in AI, I have found that most “famous” and “exotic” algorithms that software engineers love testing (e.g. OP’s references to sorting, tree searching, and recursion) come up exactly never in my day-to-day.

Please, limit testing to the actual data structures and algorithms that dominate 99% of the proposed work. If the proposed work is not in a field you are truly an expert at, then don’t be responsible for technically interviewing this person.

In my branch of AI, if I’m not getting solely evaluated on my ability to work with tensors, vectors, graphs, …, then I realize there is no AI group at this company (or it is inferior in its autonomy). There are so many extremely useful specializations in real-world moonshots, that would otherwise get bludgeoned upon any interview within a general purpose software engineering culture.

ravenstine2• 4 years ago

This would be great because if the code stinks to high heaven then I won't work there. When they think I'm not a fit for not understanding their code, the feeling would be mutual.

innomatics• 4 years ago

I agree with you in the sense that I only want to work places that recognize and value great code.

But everywhere has code that stinks. And it's hard to find people with the skills and experience to get rid if it. Before it can be gotten rid off, it must be understood.

In a sense it might be valuable to use the poor code in a reading screen. Firstly so we can agree it stinks, and the reasons why. And secondly, to find people with the attitude required to clean up messes that they didn't create (good scouts).

Even when the job is greenfield, or extending great code, I'd rather have engineers with the battle scars of doing tough maintenance.

pojzon3• 4 years ago

If you want me to clean up the mess your ppl created over the years -> be ready to pay me hefty premium or equity.

Often times it requires not only a lot of skill but also a lot of work.

I can do it but be prepared to pay me 2-3 times the normal wage.

john_the_writer3• 4 years ago

Honestly.. It's not just the desire to clean.

The management has to be set up to allow for this. People have to be able to work towards something as a team; else we are all cleaning in different directions.

andrewflnr9• 4 years ago

My thesis is that the true measure of reading code is still the ability to fix and extend it. So my current ideal interview problem is still a tiny toy codebase, where the interviewee is tasked with adding some (relatively trivial) feature to it. Like ten lines of code, but where those lines require you to have grokked the other couple hundred or so. Any downsides?

WaxProlix• 4 years ago

Only one I can think of is that a couple hundred lines (depending on content of course) can be quite a bit to grok within a 50 minute span. Especially if there are a number of places where improvements might make sense, but you have one or two in mind specifically...

I've been in that sort of position tbh, where a 200 line codebase has maybe 20-30 obviously wrong parts (creds in code, calls to 3rd party service in code w/o cacheing, maybe that could just be a service? why is this not in a queue, ...) and it's sort of weird that getting 11/14 and talking over pros and cons of the others can be considered a barely-pass.

Still, you're not wrong. Find a good enough, real-life enough example and it'd make for a great question.

andrewflnr9• 4 years ago

You would have to fine-tune the size, taking into account that narrowing your focus to the relevant parts of the codebase is an integral part of your reading skills.

paulgb• 4 years ago

One thing I like about this (as opposed to a “blank slate” coding challenge) is that it shows how well the candidate is able to fit into the broader design style of the surrounding code. Not just the superficial things (like naming conventions), but also the structural patterns.

lucb1e9• 4 years ago

fit into the broader design style of the surrounding code[, including] the structural patterns.

That might require one to be familiar with those ahead of time. If that's what you're hiring for, that's perfect of course, but I personally haven't had much experience with design patterns outside of a few C# ones in school five years ago. Yet I see myself as a competent amateur programmer (and C# isn't even my strong suit), just not one that often works on large code bases. These patterns seem like something I'd learn in a matter of days on the job... but that would not show in an interview.

xtracto2• 4 years ago

Back when I was a hiring manager for programmers I liked setting a "code challenge" where we gave the candidate a simple game and he had to transform it into another game.

For example, we gave you a tetris game, you had to change the code to make a snake game. Or change an Othello game to a "number sliding puzzle" game.

It was very interesting comparing people that really made an effort to use the game code vs candidates that just deleted most of the stuff to implement their logic.

aunty_helen5• 4 years ago

You do miss out on the 'sit down and create a project' aspect though. Which shows things like familiarity with project structure, (python specific) virtual environments and paths etc.

These are good skills to have when a dev is debugging as they could get tied up for a long time dealing with something as simple as a relative path issue.

joshgachnang• 4 years ago

Yup, this is what my current take home (2 hours) project is. Read about 200 lines of code, implement about 30 somewhat complex lines in a real working project. And then there's a code review by a "junior" candidate for them to read and review, filled with about 10 bugs in 50 lines. I much prefer it to giving out algorithms questions.

flockonus• 4 years ago

Would love to see what the assignment is like if the code happens to is available to share / public.

GuB-42• 4 years ago

By doing that you will be selecting for the "cracker" type. That is people who are able to get into any code, even under adversity, and do anything they want. It is a great skill to have even if you job doesn't involve removing copy protection, for example, he may be the only person who will be able to fix a bug in that old code base written in an obscure language when the last person to work on it left the company 10 years ago.

What you won't be testing is higher level skills. It may be easy to hack into a known codebase, and even do it cleanly, you just have to mimic the surrounding code. Getting the big idea and making long term choices is another set of skills, not exclusive with being a cracker, but not something you can see in 10 lines of code.

skydhash• 4 years ago

I've been at both ends. Building apps as a solo developer, as well as fixing some bug or implementing a single feature correctly for clients. I think it requires being able to zoom in and zoom out along the stack of abstraction that constitutes a code base, as well as either focusing on the now or a much larger timeframe.

The higher level skills can be somewhat tested by asking the candidate how it would build some module and the variation that some factors would bring in (like closer deadline, no third party service,...)

andrewflnr9• 4 years ago

Uh, no. The skill here is one used by literally any programmer who starts work on a project they didn't start. Not that there's zero overlap, but come on.

BFLpL0QNek2• 4 years ago

I like this approach.

Far to often I’ve interviewed at places and been grilled by the interviewer only to find out when you start the quality isn’t great, what you where grilled on you won’t be working on “as that’s to hard” or “we don’t do that” despite being grilled on it and the level of skill not to great they just want senior people. It’s the bait and switch.

At least being taken through existing code you know what you are getting yourself in to. Also looking at the current open pull requests and closed pull requests to see the standard and speed of delivery. Bonus points for no PR’s and trunk driven development as that shows a very mature team.

My simpler interviews have often been with companies that have held a higher bar than the ones with tougher interviews. Those companies have often been sink or swim though and if you don’t make the grade you’ll be kicked out pretty quick. My last company had a reputation for new starts disappearing and not great that way, but the team was probably the strongest bunch of people I’ve ever worked with as only do good survived.

bluefirebrand3• 4 years ago

Bonus points for no PR’s and trunk driven development as that shows a very mature team.

I'm not sure what Trunk Driven Development is, could you elaborate?

csmiller• 4 years ago

https://trunkbaseddevelopment.com/

beebmam• 4 years ago

Virtually all game companies use Trunk Based Development, in my experience, including many very large studios. (Outside of game engine development, which isn't really done by game studios anymore with a few exceptions)

dilyevsky4• 4 years ago

I’m assuming based on gp everyone merges into master, no one reviews each other commits. Very uh.. mature

Btw trunk based in general can still have short lived branches and prs. That’s what most public projects on github are doing

bredren• 4 years ago

Do you follow a style of git behavior now? I transitioned from purely git flow projects to most of my time on a trunk-based team this year.

Pretty striking difference.

kozd• 4 years ago

Why no PRs?

lanstein• 4 years ago

I read “no open PRs”

mylons3• 4 years ago

the blog author doesn’t take them through the company’s code base. the author creates code for the interview.

wnolens• 4 years ago

Yea a lot of people in this thread missed that point. It's still contrived problems, just testing comprehension rather than creation. But you can contrive much larger problems than you could ever ask a candidate to code, so you escape the ability for a candidate to practice your exact questions to some degree.

jwatte• 4 years ago

Trunk driven development has worked well everywhere I've used it.

That being said, pull requests / code reviews / merge commits are totally compatible with that. Gerrit, for example, does every commit as a mini-pull-request on top of a branchless HEAD, and when the review finishes, you merge to main/trunk and off to CD it goes.

devoutsalsa2• 4 years ago

I took a job once because they were honest about the fact that the code was a complete shit show, and that’d I’d have to clean it up. I’m sure this is naive on some level, but I’d say you don’t have to lie to people. Just help them imagine doing the job and let them decide if that’s how they want to spend their time.

whatshisface• 4 years ago

Bad code + time allowed to clean it up = perfectly well-defined business requirements + a license to think about code craftsmanship. That's a lot of people's dream job.

akhmatova2• 4 years ago

I’m sure this is naive on some level, but I’d say you don’t have to lie to people.

Sounds perfectly sensible, and not naive at all.

I'd much rather come into a situation where they were both already aware of, and entirely honest about the current shit-show status of the project -- than a situation where everyone thinks what they're doing is great and bleeding edge but when you actually take a good look at their assets, as it were... not only are they manifestly and visibly unwashed -- but no one seems to notice the stink.

zdragnar• 4 years ago

Bonus points for no PR’s and trunk driven development as that shows a very mature team.

Ugh, pass. Trunk development is fine. Skipping PRs just brings back nightmares of SVN. Even if 90% of PRs are approved without comment, it's extremely helpful for everyone to have a second set of eyes on work before it is merged in.

BFLpL0QNek2• 4 years ago

Skipping pr’s is not equal to skipping code review.

If you pair, there’s two sets of eyes, to commit both pairs have to sign a commit. You can also organise a demo/quick mob session before commit. Then there’s a level of trust in your teammates.

PR’s are great for open source projects as act as gatekeeper so not everyone can commit freely. If you need to gate keep your team members then I’d question the strength of your team.

The best teams I worked on who delivered working code fast, efficiently even when they are some of the most complex projects I worked on committed straight to trunk, had a very good build pipeline (super import) and worked closely together for review. The standards where extremely high yet the general feeling was it was less dogmatic, micromanaged or kept behind a gatekeeper.

The projects I’ve worked on with dogmatic pr’s generally failed to deliver anything in any amount of reasonable time. The prs where dogmatic as more junior teams get caught up in superficial things such as names, package structure, syntax preferences rather than what the pr actually does.

wildpeaks• 4 years ago

I agree, the PR is invaluable to preserve context: the trunk history remains simple and linear thanks to squash merges, and you can still see the fine grained commits (and matching discussions/reviews) in the closed PR.

It's also a way to give insight when onboarding new developers: if something is surprising, they can see why things came to be this way, not just accept the result at face value.

intel_brain• 4 years ago

i think that was sarcastic.

boomlinde3• 4 years ago

Where I work, development is trunk based and without pull requests, and all code is reviewed. When I want to submit code, I push it to a staging area that tracks master. This causes the commits to appear in Gerrit where (conflict-free) rebasing can be performed with a single button and the code can be reviewed. During the staging I can change anything about the commits to my hearts content. Once everyone is satisfied, someone with the authority to approve the change approves it and I submit it, upon which the patches are applied as-is on the branch I staged it for.

Creating a separate branch, pushing this to your public copy of the repository and then asking someone to pull from that branch into their master branch seems absurd to me, especially if it's just 1-2 commits, and the idea of reviewing code (which I think is extremely important at team scale) should not be conflated with the concept of pull requests.

auspex• 4 years ago

Workflow and pull request approvals are required for SOC 2 type1/2 compliance.

jjfoooo6• 4 years ago

It really seems like this "no PR" idea is having a moment on HN. I can't tell if it's just engagement bait contrarianism or genuine. That any developer would trust themselves even to just not make simple typo's is baffling to me.

Maybe the kernel of a good idea in the comment: A team that prioritizes reviews, such that PR's are not a significant hurdle, is indeed a sign of a mature team.

pojzon3• 4 years ago

This is precicely why only few companies will do this kind of interview.

Good candidate would instantly notice the team is being labeled as seniors but instead are a bunch of crappy devs with years of experience.

Code does not lie. If you are crap you gonna produce crap code.

Ive recently hit such a mine. Lies during interview. Gonna sit here through vacations and jump the ship. Was sold on working with experienced ppl with over 10y of experience. Turns out those ppl IMHO have less skill than me few years ago with only 2y experience. Legacy bugers that did not improve over the years.

jonathanlydall• 4 years ago

They have 10 years of experience, but unfortunately it's just the same year of experience repeated 10 times.

mstipetic2• 4 years ago

I’ve never understood this. I’ve had Teams grill me on questions I know most of them wouldn’t pass and they themselves said they’re struggling with delivering things. Some weird dick measuring thing

jacquesm3• 4 years ago

Power game.

jseban17• 4 years ago

They want someone to do all the work so they can continue coasting, team effort!

chucksmash• 4 years ago

It could be that, but it doesn't have to be.

If you're on a team that isn't currently having staffing issues, I don't think it's weird to want your next hire to be better than the median person on the team. "Raise the bar," etc. Asking harder leetcode questions seems like a bad way to achieve it, but I get the impulse. If the bar for member nine of the team is approximately "better than the bottom 4" then that 9th person in is likely to have at least one moment of "wait, how did you pass that interview?" at some point after they come on.

Riverheart2• 4 years ago

Or they don't have the time or manpower to fix/learn things and are trying their best to hire someone to help right the ship before bringing you onboard. Such a negative take.

jahnu4• 4 years ago

Last person I hired to work with me in my previous job I walked them through the entire code base, architecture and CI setup. This gave the hire a chance to best understand day to day work and me a chance to know which parts they understood immediately and which they would need to learn. Works perfectly for me.

jseban17• 4 years ago

A for effort!

KnobbleMcKnees• 4 years ago

Riddles posed to me at engineering interviews: ~12

Riddles I've encountered while working as an engineer: 0

jseban17• 4 years ago

despite being grilled on it and the level of skill not to great they just want senior people

Happens to me too, I get tough interviews, and a good salary, but the work is exactly the same as the juniors "a guy on the scrum team". I don't even understand why they hire seniors, or have that title when it means nothing. I guess they expect them to be just faster versions of juniors.

jacquesm3• 4 years ago

I guess they expect them to be just faster versions of juniors.

No, they expect them to be:

- more independent

- able to bring past experience to bear on present problems

- make fewer mistakes

- be able to help others rise to the next level

- possibly make good team leads at a later stage

ignoramous• 4 years ago

Those companies have often been sink or swim though and if you don’t make the grade you’ll be kicked out pretty quick.

I hope they weren't some gatekeepers you ran into... "don't use emacs and fish and the dvorak keyboard? you don't fit in here."

linspace• 4 years ago

It would be quite absurd to dismiss a candidate based on the editor and yet I find it more important that a particular cloud provider, which quite often is a requirement.

kurko• 4 years ago

Bonus points for no PR’s and trunk driven development as that shows a very mature team.

Trunk driven does not mean mature team 100% of the time, but if you have a mature team trunk-driven is more efficient than PRs.

It only works if either everyone is senior, or it’s a project of 1-2 devs, or if people are pairing most of the time.

kolanos• 4 years ago

Those companies have often been sink or swim though and if you don’t make the grade you’ll be kicked out pretty quick. My last company had a reputation for new starts disappearing and not great that way, but the team was probably the strongest bunch of people I’ve ever worked with as only do good survived.

This is not what I would call a "strong" team. This is what I would call a developmentally stunted team. By that I mean they've reached competency as individuals, but they have limited ability to level up fellow developers. As such, they are not a producer of talent and rely on actual strong teams for their hiring pool. In my experience, these teams are best avoided if you actually care about growth.

wsostt• 4 years ago

I've been doing this for a long time now. I present the candidate with a function (on paper) that compiles and runs. It does the job but it does everything poorly: an embedded connection string, leaves the connection open, no error handling, bad variable names, no comments, etc... EVERY candidate can find something wrong with this code and the things they pick up is informative. Juniors can find the easy stuff, more senior-level folks find the deeper flaws about the basic structure of the code like figuring that a lot of this is boilerplate that could be implemented elsewhere.

Silhouette5• 4 years ago

I always preferred this style of technical interview because it has two critical advantages over tests based on coding from scratch in an interview situation.

Firstly, you can work with realistic code. It can use your real tech stack and there can be enough code with a realistic structure to see how a candidate finds their way around.

Secondly, you can make it open-ended. For junior candidates it might be best to stick to simple questions like what does this short function print and see if they can reason through some basic logic. But for more experienced candidates it can be a general code review.

Your example code can include anything from superficial mistakes like typos and unnecessary duplication to strategic problems like inflexible designs or inconsistent models. You can include obvious or subtle logic errors and performance problems appropriate to the level of the role.

Ask each candidate to talk you through what they're thinking about as they read the code and see what level they work on and how much they find in the time available. Are they methodical? Do they flag the trivial stuff but not get hung up on it? Do they spot the O(n^2) algorithm? Do they spot that algorithm but then start talking about worst case vs amortized performance and using a profiler to decide whether changing to a theoretically better but much more complicated algorithm would be justified?

In this kind of environment you can quickly tell if you have an awful or outstanding candidate. For the majority who fall in between you have an objective basis for comparing their performance. And all without any trick questions or LeetCode junk at all.

jra_samba17• 4 years ago

When I did interviews @ Google (I only do hiring committee work now, thank god :-) I usually asked questions around bugs I added to an existing codebase, to see if the candidate can avoid the pitfalls I ran into by making bad assumptions.

As I'm a pure C coder, my starter question was usually something like:

a). What does malloc(0) return ? b). Why does it do that ?

llimllib• 4 years ago

I'm not anything other than a C tourist, and I see that the man page says it returns either NULL or a "unique pointer value that can later be successfully passed to free()."

I'm kind of at a loss about why it can return either of those two things, somebody want to take a shot at explaining it?

jra_samba17• 4 years ago

Both NULL and a unique pointer value can be safely passed to free() :-).

Answers to this question taught me about the candidates taste and understanding of good API design :-).

Both NULL and "unique pointer" are correct answers, but all modern implementations only chose to return one of these. My follow-up question is "why ?" :-).

lelanthran• 4 years ago

I'm not anything other than a C tourist, and I see that the man page says it returns either NULL or a "unique pointer value that can later be successfully passed to free()."
I'm kind of at a loss about why it can return either of those two things, somebody want to take a shot at explaining it?

Any return from malloc, whether it succeeds or not, is a valid argument to `free()`. Hence, it can return NULL because `free(NULL)` is legal, and anything other than NULL has to be a unique pointer, because if it returns a duplicate then calling `free()` no the duplicate will crash.

oconnor663• 4 years ago

Huh. My first answer was "I don't know", and then I googled it and apparently it's implementation defined. So...my first answer was correct? :)

jra_samba17• 4 years ago

True, it's implementation defined. But almost all implementations choose one of the two options. There is a reason for this.

FYI. I introduced a (bad) security bug into Samba a long time ago by expecting the wrong return (i.e. the one the implementations never return :-).

aunty_helen5• 4 years ago

I find questions like this lack an on-ramp. Either you know or you don't. If you don't it gives you 0 indication of the skills.

I once flunked a faang interview because the interviewer mispronounced (or used the correct pronunciation I was unfamiliar with) of "arp protocol"

I had no idea of what was being asked and was racking my brain for something I didn't think I had ever used to down every computer in the library of my high school 12 years before.

jra_samba17• 4 years ago

No, that's not true. If you know anything about C and writing secure code (which I what I was probing for) you know about using malloc(). You know because you have to know something about the internals of memory management.

Imagine you just read a 4 byte value off the network, and it's part of a protocol that specifies how many more bytes there are to read. You might (in error, ahem... :-) pass that value to malloc(). So knowing what might happen if an attacker puts an unexpected value in there is something you need to think about.

If you don't know or can't guess because you don't know how malloc() works, then you're not the person I'm looking for.

teddyh4• 4 years ago

According to GNU libc, malloc(0) can return either NULL or a valid unique pointer value which can later be passed to free(). You seem to imply that one or the other is definitely true.

jra_samba17• 4 years ago

Yes. Write test code for glibc. You'll find it always returns one of the two. There is a reason for that.

ja27• 4 years ago

I did this. I had a folder with a few printed sheets of code hiding some slightly tricky C/C++ bugs from our codebase. I would ask the candidate to see if they could identify the issue. I don't think anyone ever actually got them but at least I got an idea of their thought process and experience. So mixed results.

jasoneckert• 4 years ago

Of course, if a developer comes with experience and personal references from acquaintances, you don't need to test their abilities at all during the interview process.

For example, I hired a backend developer last month. She already had the job because she came highly recommended from a trusted friend of mine, but she didn't know that. Here's how the interview went down:

Me: I see on your resume that you've achieved Grand Master level in Microsoft Solitaire Collection.

Her: Yes.

Me: Well, we won't waste any more time then. Welcome to the team.

Cyph0n• 4 years ago

Completely off-topic: I am 99% certain that I read a very similar comment on HN a few weeks back. Did you happen to share this anecdote on another HN thread?

This is my first time experiencing HN deja vu :)

fma2• 4 years ago

Same here...except I tell up front when they apply that it's more of a formality for HR. I respect their time and don't want them to waste time preparing or getting nervous. I tell them it's mostly for the team to get to know them and for them to evaluate the team and we'll keep the interview light.

I still give my team the option to decline them if there's major red flags - but I have not had that happen.

Also, a reminder...an interview it is a 2 way street. Your 1 question interview doesn't give the candidate an opportunity to interview you or your team and the "Welcome to the team" is a little presumptuous...I assume it was left out for dramatic effect and I'm sure (I hope?) you allowed the interviewee time to ask questions.

staunch• 4 years ago

I still think Triplebyte gave me the best interview I've experienced, and I've done ~100 over the years at all kinds of companies. I've also interviewed people ~1000 times myself.

It was even more impressive because the person doing the interview wasn't super competent and yet they still managed to do a fairly comprehensive technical evaluation in ~2 hours.

One section was a debugging session where you had to get tests to pass. The code and tests were quite decent, which made it quite easy to show off. Every company should do this.

Too bad the Triplebyte promise (not having to do phone screens) was a joke and their business model wasn't good, but that part of Triplebyte had major promise.

ineedasername• 4 years ago

Reading probes the most fundamental skills. Reading code is probably 95% of what a developer does as part of their job.

This is true. Most of the time I'm only reading my own code, and over the years I've been motivated to code better by having to go through crap I wrote early on.

quickthrower22• 4 years ago

In my jobs I am reading other peoples code most of the time. Maybe a consequence of small company/team “jack of all trades” culture and also there are more “not mes” than “mes” contributing to the code base. Hell someone doesn’t even need to be currently employed there for them to have the pleasure of me reading their code (or maybe my pleasure!)

monksy2• 4 years ago

When hiring developers, have the candidate read existing code

Are they intentionally trying to scare devs away?! (Some of the code bases that I've seen have been pretty bad)

a_square_peg2• 4 years ago

Would really like to see one day a post about 'when hiring developers, read their CV and have a technical discussion about their past work in relation to the role required' becoming a thing.

PretzelPirate2• 4 years ago

So many companies are requiring 6+ interviews nowadays. If they decided to take a reasonable approach to interviewing, think of all those people who wouldn’t be able to write “Played an active role in hiring” on their performance review!

sgustard• 4 years ago

I do that extensively when hiring and it is valuable. But sadly, many people can hold a technical discussion without actually being able to read/write code. I'd like my orchestra to have a lively discussion of music theory, but I still need to hear them play the violin.

fma2• 4 years ago

My company requires just easy leet code exercises. If you can pass that and you can hold your technical discussions - the stuff you have on your resume is likely true.

If I interview someone it's because I like their past experience and think they are a good fit and/or have growth potential. I just need to know if they're lying and if they fit the team personality wise.

What knowledge do I have to gain in having them do medium & hards besides that they can solve medium & hard exercises, which aren't really applicable day to day?

This is equivalent to fizzbuzz without fizzbuzz. But that's just my opinion and I do not work at a FAANG...

anyfoo7• 4 years ago

I agree, and it comes up in every discussion about interviewing, here and elsewhere: People claiming that the "coding interview" is unnecessary and maybe even demeaning if the candidate presents the right credentials and can talk the talk, and people who know from experience that they are sadly necessary anyway.

a_square_peg2• 4 years ago

I agree it's not a straightforward thing - it's just that in many cases people seem to be hiring a violist but screening them with live sessions of Guitar Hero.

gilbetron• 4 years ago

That was the way it was done for years and years, and it's poor performance is why we have much better, but still flawed, ways of interviewing these days.

ChrisMarshallNY4• 4 years ago

This makes a great deal of sense, to me.

But I am also the type of developer that would do well at this (experienced and older). Young folks, right out of school, or with just a couple of years of experience, would not do as well.

I’m pretty convinced that one of the goals of LeetCode tests is as a “young-pass filter.” It controls for people close to college age, where those types of problems are common, as well as people that are willing to work very hard, learning exercises that appear pointless.

Not sure that many companies, these days, are actually interested in older, more experienced, developers.

travisgriggs3• 4 years ago

Not sure that many companies, these days, are actually interested in older, more experienced, developers.

I’m beginning to appreciate the value of cohorts as I age. I work in a ver successful triad, ages 51, 58, 61. We recently tried to integrate a young-30s in our group. It did not go well. They recently moved from our team and are working elsewhere with a group of people closer to same skill, aptitudes, and age. Both they and we are much happier this way. It’s a small sample set obviously. I value a diverse world where we tolerate and learn from each other. But years of working experience have made me wonder if we shouldn’t just let age/skill cohorts naturally work together.

harrydehal• 4 years ago

Throwing in my own anecdotal example and tiny sample size: many years ago I was part of a trio of 20s, 40s, and 60s front-end devs – all rare and elusive Bay Area natives who grew up in very different part of SV history with very different backgrounds.

On first glance, we really should not have worked well together. But the group clicked because it shared a value system (be it for writing clean code, good documentation, user experience, collaboration, etc.).

And it was two introverts and one extrovert, to boot!

weatherlite3• 4 years ago

I get your point but its not like recent grads have it super easy getting their first job; you have decades of experience interviewing (part of which is Leetcode which you have probably done many times); for them it's probably the first time doing this. On top of it all many places don't want juniors at all.

ChrisMarshallNY4• 4 years ago

Actually, I never took coding tests, until a few years ago, when I was looking for work, at age 55.

I quickly learned, that if I saw a binary tree test, I might as well give up. Even if I did OK, the company was not going to give me a chance.

What I do have, is a ginormous portfolio, with dozens of repos full of ultra-high-Quality code, for shipping products, and over a decade of checkin history, along with hundreds of pages of documentation, and dozens of blog articles and postings; often directly discussing my design methods and development practice.

It has been my experience that this was dismissed, even when I curated and sent examples, directly relevant to the job.

phone8675309• 4 years ago

The company I work for is desperately hurting for older, more experienced developers. We use very mature tech (to put it charitably) - Java/Spring, Python/Django, C++ for OS interfacing parts - and we are having trouble filling roles for junior and senior software engineers.

Part of that is the company is insisting on going hybrid after the pandemic, so they filter out full remote candidates for long term positions.

To connect to the article, we have three questions in our interview that lasts an hour and a half - in one of which the candidate is given a small program with a bug and we see how they troubleshoot it - the idea is not to fix it, it's to see how they think.

gotaquestion• 4 years ago

This is a clever idea. First, every team has tons of code that a new-hire would see on day one. Might as well see how they perform! Second, this truly is an important skill. I've read hundreds of libraries, if not thousands, and being able to accept not just a coding style, but a thought style, and internalize it is essential. Yeah, the more I think about this, the more I wish I had thought of it when I was hiring!

I'm kinda jealous, I'd love to do code interviews again, but I haven't been a junior dev in ... /* checks notes / ... over three decades?!? cries in yaml*

EMM_386• 4 years ago

Please comment your code, when it is is necessary.

I don't need "we need to loop here from 1 to 50", I need "we have to rate-limit this function to under 60 transactions per second due to hardware requirements", etc.

If you are putting "magic numbers" anywhere, COMMENT it as to what that number is, why you chose it, etc.

I'm 30 years into this game and I still come across code that takes way too long to reason about.

bkuehl2• 4 years ago

Agree completely.

Also, good commenting skills take years to master.

jgauth• 4 years ago

good commenting skills take years to master

As a newer developer, it’s nice to hear you say that. I often find that I spend more time deliberating over comment wording and variable names than I spend writing the code itself!

dwaltrip• 4 years ago

If you are putting "magic numbers" anywhere, COMMENT it as to what that number is, why you chose it, etc.

Better yet, turn magic numbers into constant variables whose name becomes the comment. Of course, comments can also provide additional context :)

srigi• 4 years ago

Agree. I also add to this - name your constants by meaning, not value. Too many times I see

    const ONE_HOUR_IN_MS = 3600000

Instead I would like to see

    const RESEND_DELAY = 3600000

spike021• 4 years ago

100% agreed.

A trend I've noticed the past couple years is that people make the excuse of "code should be self-documenting" so they don't need to add comments. But then very helpful contextual hints that are not otherwise documented anywhere, like the one you provided, are completely non-existent and the original dev has left the team, it's been 2-3 or more years since the last time someone touched it, etc.

Code should be self-documenting, don't get me wrong. But that's more in the vein of showing (what), not telling (why/how). There still needs some be some method of telling.

chrismarlow9• 4 years ago

Not disagreeing because I think comments can help solve confusion immediately in clever code. But when I tell people the code should be self documenting it implies there will also be:

some form of git history to go along with it

some single page architecture type of doc. No technical info, just like a README saying what this thing does and some 101, maybe a diagram.

The "doc as part of the code" for me is really just a easy way to generate a "reference" for function signatures and specific code interfacing. And if you start off that way the reference is at least always up to date, because it's caught in review (ideally).

As for updating the README that's just on the dev team to remember.

I will say my opinion changes a bit with huge polyglot mono repositories. In those cases you need great docs and organization.

satran• 4 years ago

As a general guide I comment the why, now the what. The what can be generally understood by reading the code. But the why is not so easy.

twblalock• 4 years ago

Don't do it unless you are going to treat keeping comments up to date the same way you treat keeping the code up to date.

I've lost count of the number of times I've seen comments that were just wrong -- they applied to the code as it was written years ago, not today.

Write good code, and it should be obvious how it works. Use comments sparingly and only when things are not obvious. Update the comments when you change the code, and make sure your code review process doesn't let outdated comments slip by.

If you can't commit to the above policies, then don't comment your code at all.

enraged_camel• 4 years ago

> Don't do it unless you are going to treat keeping comments up to date the same way you treat keeping the code up to date.

This should be easy enough to verify during code review, no?

pkolaczk• 4 years ago

Code comments that are wrong can be very useful. I've found many bugs that way. If I find a comment that states an assumption that does not hold in the code, that's the first place I analyze when looking for a bug. And very often it turns out the comment was right, but the code was wrong.

TameAntelope7• 4 years ago

You can just name the variable to explain what the number is, and if you need more info, there should be a "Why?" doc somewhere explaining the context generally, WITHOUT tying itself directly to the current choice (otherwise you'll have to update the "Why?" doc every time you update the number in the code, which is almost certain to go out of sync).

No need to interweave documentation and code, in most cases. Sometimes, when we fail to write good code, sure. But let's try to write good code!

jonahx2• 4 years ago

No need to interweave documentation and code, in most cases.

Locality is the reason, and it's a good reason. Of course, I don't want to see paragraphs of explanation inline, but a one-line comment giving context for some choice that might be confusing is much better and faster to work with than a separate WHY doc.

arduinomancer• 4 years ago

This would not work based on my experience

What actually would happen is the next person comes along and changes the code and doesn’t update the external doc because it’s probably buried among 100 other pages on confluence and they don’t even know it exists

If the comment is in the code they will see it and update accordingly

emrah• 4 years ago

Agreed, despite the elephant in the room that when the company is in growth stage, comments rot, documentation rots, things change rather quickly and everything including comments become a liability. I'm not saying they shouldn't be written but they also need maintaining

faangiq• 4 years ago

No. Only leet.

andrew_• 4 years ago

Very glad to see takes like this becoming more popular. This is how it was done when I started out.

Uptrenda• 4 years ago

Except most commercial code bases are tangled rats nests and it would take a substantial effort to understand anything beyond the most basic details. My view of interviewing is it should require no special effort on the part of the engineer to demonstrate their skills. One simple way to do this is to have the engineer explain an open source project they've already built. Obviously this doesn't work for people who have never created open source projects they would be comfortable sharing. But I think we need to be doing more to actually value people's time.

My experience with many companies is they treat people as a 'resource' that needs to be managed rather than a human BEAN. Companies, in the process of funneling around these resources, forget that there is an individual cost to these resources. And if your first introduction to a company is being dehumanized and treated like your time doesn't matter: what are the chances that this company is a good place to work at? And that mans name. Albert Einstein.

megous• 4 years ago

This is so funny. :) Thanks for a good laugh.

VikingCoder• 4 years ago

We had a mix of code written by brilliant people who knew how to maximize cache coherence in template metaprogramming hacks... and interns who generated more compiler warnings than anything else.

Over the years, I found some truly awful code, in our cash cow application.

So I turned those into the smallest representations of what was bad, and showed those to candidates.

I was trying to assess, "How much will I need to mentor this person, and how hard will that be?"

I may have given bad reviews to people who deserved better, but I honestly think I never once gave a good review to someone who deserved a bad one. A few times, the company went against my suggestions, and each time, the candidate wrote some terrible, buggy code that I later had to untangle and fix.

no-dr-onboard2• 4 years ago

I’m personally against this approach. I’m an appsec person. I absolutely do not know how to write most of the code I review. In my past two jobs I’ve been required to read code and explain it during interviews. I’ve never had trouble explaining what’s going on because on a very basic level, most languages use the same conventions.

If this were a heuristic of a good developer, it would let me (not a dev) in and who knows what the effects of that could be :).

reflection99• 4 years ago

Interesting take

no-dr-onboard2• 4 years ago

I’ve got more:

I think one way to iterate on this to weed out guys like me would be to ask process based questions that reveal the candidates experience and preferences:

“Brainstorm with us on how you would you make this messyFunction more performant given $these conditions”

“We are thinking of developing a feature that would call this function more often. Would you turn this into a micro service?”

“Looking at this db call, how do you see this scaling if we increase the number of x and y?”

“Is the following PR safe or sane?”

droopyEyelids• 4 years ago

Maybe you’re seeing a benefit as a detriment.

If you wanted a job coding, and you can currently read code, i think you could do a passable job if you were working at it for 8 hours a day.

In this way, the interviewer is expanding their talent pool.

andrewflnr9• 4 years ago

I suspect people like you, genuinely good at reading code but not at writing it, are not a significant threat to the average dev hiring pipeline. There aren't that many of you and you have better things to do than apply to programming jobs you're not qualified for.

a3w• 4 years ago

Oh Writing Code is easy: copy and modify ;) Pros remove the edge cases that allow exploits from stackoverflow examples, but you seem to be qualified to do that.

nullc• 4 years ago

You might be overestimating the grade of people who make it through coding interviews.

I'd rather hire-- as a coder-- someone that could read code but would fail an interview where they coded, than someone who could pass some trivial coding interview but couldn't read code.

Obviously, people need to do both, but the reading seems less likely to get a false positive/negative in an interview.

Heck, given two candidates one who cannot currently code but reads code fluently vs one who can code (and not just pass a coding interview) but cannot read code-- I'd still prefer the reader. It's the more important skill.

Average industrial programmer is going to write 8 lines of code per day. If you can correctly read a codebase, you can probably manage to write 8 working lines of code per day for it on average. Maybe you wouldn't be #1 one the team overnight, but a fluent reader is at least on track to be average even if they hardly write at all now.

sriku• 4 years ago

I do a variant of this in my interviews. I write code from scratch and talk about the statement and ask questions based on the code before asking them to make modifications to it and talk me through their thinking.

rzimmerman• 4 years ago

I have found a lot of value from putting up real code from our code base (maybe simplified with some parts removed) and asking the candidate to explain what a function does. It’s very fair, low stress, and you can ask open-ended questions about “why this way?” or “what are some other ways to do this?”

rpmisms• 4 years ago

I would like this. I'm a lowly front-ender with some engineering chops working towards a full-stack/backend role, and this would help me convey my knowledge better than getting brain freeze when trying to remember syntax in a language I might not be great in.

MichaelMoser123• 4 years ago

I saw that back in the nineties: my employee had an interviewing task where you had to look at a c++ listing with obvious errors, the task was to find the errors, like memory leaks, buffer overruns, use of stack allocated memory, etc. Actually few candidates would pass this test...

I don't think they will do this: the interviewing process at most places seems to emulate that of the industry leader, nowadays that's google, correct me if I am wrong.

baskethead• 4 years ago

There's only 1 real answer to hiring.

Make sure the candidate's personality is a good fit for the team, have them do a fair coding question, and then hire them quickly. Give them 3 months to become productive and if not, fire them quickly and give a 3 month severance package.

You can probably analyze the data and figure out which of the employees are good at spotting good candidates and lean on them to make the decisions, but overall fast-hire-fast-fire is the best for everyone, except for fake candidates.

This also gives you the opportunity to take chances on borderline candidates or candidates with less experience.

mostertoaster• 4 years ago

I like this idea. My first job out of university, this was actually what they did.

Though many companies don’t care if you’re familiar with the specific language they use, because they assume a decent developer can pick up a new language, and if someone weren’t familiar with that specific language it maybe could put them at a disadvantage. Or maybe that would actually be a good test of their skills??

morelish• 4 years ago

Yeah I had an interview like this recently. First part of the interview proceeded well as they asked me to read different bits of code and how different language features worked.

Then I was asked a brain teaser that I bombed. And that was the end of the interview.

john_the_writer3• 4 years ago

I was once asked how I would sort 1000 bolts.

I said. I have 5 year old twins, I'd give it to them and have them sort it.

haspok• 4 years ago

Problem is, reading is at least an order of magnitude easier than writing.

In terms of natural languages, if you can read and mostly understand a text in another language, let's say you score 8/10. At the same time it is completely reasonable to expect that you would not be able to write the same text, and if you had to, it would be at 5/10. Then if you had to do it in a speech, you would score a measly 3/10.

I'm not saying that focusing on reading and analyzing code is a bad idea, just be careful, and expect these differences in skill levels. Definitely a hundred times better than leetcode though.

pelario• 4 years ago

I suspect you have not read enough code.

cphoover2• 4 years ago

In my professional experience there is a lot of poorly writtem, convoluted spaghetti code, that is extremely hard to follow. Im not sure why people seem think reading code is easier than writing code... this is often not the case.

buf5• 4 years ago

I've interviewed maybe 500 engineers in my career. I'm an early engineer of Instacart, 3rd engineer of Eventbrite, founding engineer of Reforge. Started 3 companies myself.

My interview is always the same:

1. Bring code you've written 2. Share your screen 3. Explain what it does and I will casually ask questions about it

You get so much information from this:

- How they think about code - If they think it could be better - Who they blame if the code isn't the best - Personality - Product dev glimpses - Comms - Sentiment

itronitron3• 4 years ago

A lot of developers' best work is within employers' proprietary code bases, do you consider it a red flag when they share some of that code with you?

refactor_master2• 4 years ago

It should be. As should “bring your own code”. In which other engineering industry can you ask this of people?

buf5• 4 years ago

I've never been put in the situation where an engineer shared something alarming from a proprietary code base, but hypothetically if someone brought with them the code for how to hack superuser privs for an old company, that would be an ethics violation and I would pass.

mbeex• 4 years ago

It hasn't to be the 'best' work. The actual value comes from the conversation.

ProZsolt• 4 years ago

I have tons of open source code on GitHub to scratch my own itch, but nothing that I consider production ready.

jseban17• 4 years ago

Sounds good maybe for a founder or early in a startup, but otherwise this would probably illustrate the mismatch in priorities that exists between a single person's passion project, and risk-averse enterprise groupthink.

aidos2• 4 years ago

I think it’s really important to differentiate the code and the coder. Where we use this technique, I actually say to candidates, “I don’t care about the language / quality / age / immaturity of the code- it’s just something that you should be familiar with”. If someone turned up with an open source codebase to talk about it would be absolutely fine too.

jahnu4• 4 years ago

Surely this will filter out ~50% of people who are good but don’t have any public code?

I have a family and as such no free time for coding so I haven’t written any code I can legally show anyone else in more than a decade. But everyone who has employed me is more than happy with my work. Not to mention code is only half of why you would want to employ any developer.

xarope2• 4 years ago

I agree. I would be happy to provide references of the developers and engineers in my ex-teams, whether collaborators or mentees, who would give me glowing references.

Instead, I'm asked for senior-managers/directors as references... who were the reason why I left in the first place. I guess HR just wants to know I was actually employed there (tick box)

moonchrome2• 4 years ago

Surely this will filter out ~50% of people who are good but don’t have any public code?

It then becomes a take home assignment where you get to pick the topic.

Surely you wanted to protoype some tech but didn't have the opportunity at dayjob - so make that prototype and bring it to review.

Much higher interview value than take home assignment IMO, but you need to be competent as an interviewer to enter into a discussion on something you might not understand yourself.

And it's also higher value to the person applying because they get to protoype stuff they find interesting.

buf5• 4 years ago

Hi, comment OP here. I decided to blog about this. It's cheeky, apologies. https://siliconvict.com/articles/6-how-to-hire-actually-good...

rickosborne• 4 years ago

Surely this will filter out ~50% of people who are good but don’t have any public code?

Not OP, but I follow a similar process when I have to do coding interviews. I work around the "no public code" problem easily: "great, then pick an open source library you use regularly and let's go through and look at some of the things you do with it, what you like about the API design, and some things you stumble over or wish were better".

I've had candidates go through everything from jQuery to D3 to Spring to just parts of the Java SDK.

Also, in my experience, the percentage of people who have _zero_ public code is small. Maybe 10%. Certainly not half.

mountainriver• 4 years ago

Yeah we do this but also give people an option of either a take home test or live challenge. It works really well and fits all developer types.

At the end of the day all your after is signal on how well they understand software. There are lots of ways of doing that

anyfactor4• 4 years ago

I am out.

My good code are paid for and thus is owned by someone who is not me and can't be shared with a third party. The code that own is inherently bad as I want to create things as fast as possible without being bogged down by proper code writing etiquette.

buf5• 4 years ago

Everyone in this thread seems to be hung up on sharing code from an employer or 'good code'.

You could clone an open source repo and use that if you're worried. The value isn't the code; it's the conversation.

"Bring your own code" is just meant to put the interviewee at ease because they are already prepared to talk about their own code.

xarope2• 4 years ago

This is definitely a concern. I've previously worked for e.g. eye bee... and despite having written massive amounts of code in c, java etc, I'm probably violating some major NDA if I show any of that code, or discuss details of a first-in-the-world active-active HA cluster solution in the 90s.

By comparison, my github or personal code would be just a bunch of perl, rexx, sh/bash and python scripts written just to automate some work. I doubt anybody would think those should be cleanly written, highly maintainable, modularised, etc...

throw14082020• 4 years ago

On average, who's going to be better at writing code, the one who has done a side coding project in the past 10 years whilst developing, or the one who hasn't but only worked a job? (assuming everything else is equal) - The one who has more time for side projects might also have more time for work, too. Unfortunately, it does remove good people, since not everything else is equal.

IMHO It's still better than leet-coding questions because most developers don't need to use the skills gained from practicing leetcode when at work, whereas open source contributions and projects are valuable. Also, someone with no time outside work and family would struggle with both leetcode and having side projects.

tiborsaas• 4 years ago

I'd still show them my bad code, since I do pretty much the same thing. It would be a good opportunity to show how I'd refactor the side project if I could dedicate more serious time to it.

tick_tock_tick• 4 years ago

Sounds significantly worse then whiteboarding.

caffeine• 4 years ago

I think this is a great interview. I do something similar.

I’ve done most of my interviewing in finance where:

1. The engineers are mostly really good

2. They almost never have side projects, and everything they have ever done is super secret and proprietary.

For this it makes sense to ask people to do a take-home, and then ask them questions about it when they come in.

I usually also ask them to add a simple feature to their code and just silently watch them code it. You learn a lot from watching how someone breaks down a simple problem, especially where they are expert (ie they are working in their own codebase).

shafyy• 4 years ago

I recently interviewed at Whereby and they did exactly this. It was a breath of fresh air and I really enjoyed it.

If you didn't have an existing code, they suggested you built something from scratch and compensated you for the time spent.

zeroonetwothree3• 4 years ago

Whenever I see a comment that starts out bragging about someone’s accomplishments I always assume the rest is going to be simplistic at best and nonsensical at worst.

I was not wrong this time either.

noisy_boy• 4 years ago

Sure - I'll be happy to show you the scores of scripts I have written to do every tasks like switch pipewire audio sinks, wrapper to youtube-dl, generate/copy rsa token, convert/combine images to pdf etc that were written in the sliver of time afforded to me after I'm done family and kids. Those scripts in no way represent the code I get paid to write in my job but if you find that to be an an issue, then you have found the flaw in your method.

kissgyorgy2• 4 years ago

Good article, terrible advice at the end. I agree 100%, seems like a way better interview approach, but the last sentence is just not applicable to everybody. I enjoy doing side projects, but should not be a requirement and you can be a great developer without even having a public project on GitHub.

lizardactivist• 4 years ago

Related to this, is solving programming-trivia on white-board with a clock ticking and someone watching over your shoulder still a thing?

I think that's leaning to interrogation, and less of interviewing.

KaiserPro• 4 years ago

At first reading, I thought this would be horrible

but then I realised it would tell me as an interviewee how good/bad the code is before I join.

If there are no comments, loads of "clever" optimisations lots of "syntactic sugar" it would be a good time to GTFO.

dagmx• 4 years ago

I'm struggling to think of code examples that would be concise enough to be usable in an interview setting, but that are complex enough to still discern their skillset to code.

If you introduce subtle issues in your code, you're most likely just testing their familiarity with a framework or language nuance.

Big issues could work.

Perhaps just providing a take home codebase that's 80% of the way there, and asking them to take it to where they think the remaining 20% should be is a good middle ground.

jll29• 4 years ago

This does not just work for companies: The University of Cambridge (England) asks for candidate-written code as part of the application package for its Master's program in advanced computing (MPhil).

AlecSchueler• 4 years ago

To be honest as a developer I might have thought twice about certain roles had I seen their code base ahead of time!

Nothing cured my imposter syndrome (self taught, no degree) like walking into a startup and seeing PhDs fail to follow just about everything I'd ever learned was best practice and was doing without thinking in my own projects.

opensrcken• 4 years ago

These kinds of articles always seem to get a lot of comments, and so many hiring processes seem to miss the mark.

If the criteria that you use for engineering hires has nearly 0 overlap with the criteria that you use for engineering promotions, then I'd assert that your company is suffering from some serious cognitive dissonance.

This article describes a hiring process that is probably a lot more useful than what I've typically witnessed at the FAANG's, provided the results can be quantified.

lysecret• 4 years ago

It's kind of funny hearing both potential employers and employees coming up with "concise" or good enough code example. I think this exactly the beauty of this approach. If you don't talk about some simple sorting Algo code is always full of tradeoffs. Talking about these tradeoffs how your potential hire thinks about them if they even see them. This is the real value of this approach I think.

langsoul-com• 4 years ago

I hope it's actually to an ide. They do so much to make code easier to read, like showing the code highlighting, function docs, and links to the source

DeathArrow2• 4 years ago

Nobody sketches code on a white board or notepad as part of their daily work.

Sometimes when I'm trying to plan something which requires a lot of thinking I take a piece of paper, write bits of code, draw schematics of the flow, schematics of data structures. If I am planning the architecture of an application which isn't trivial, I like to go to the whiteboard and draw. Even if I have it in my mind, visualizing it helps me find improvements.

DeathArrow2• 4 years ago

So instead of writing code, consider instead having the candidate read existing code and talk about what it does and how it works.

While reading and understanding code written by others is an important skill, I am not sure it would be totally fair in an interview. The candidate might not be familiar with the particular coding style. The code can be very easy to read or very cryptic. I can write easy to read code, cryptic code or anywhere in between.

So since reading code doesn't solely depend on the candidate 's skills, basing hiring decisions solely on it, might not be the best idea.

I would see no harm, though, if part of the interview consists in testing the comprehension of well written code.

saos• 4 years ago

Nah lets give them leetcode instead!

pierredewet• 4 years ago

This is my preferred option also but I love reading the comments when these sorts of posts appear as it seems that interviewing is still something that causes loads of debate and no matter whether the op is whiteboard, tricky algorithm or <other> focussed, there’s still a hot debate. Interviewing just seems broken

It must surely boil down to the profession aspect. Doctors and lawyers do a period of internship after and during the degree that perhaps mitigates the uncertainty around hiring. I doubt that doctors are asked to bring in a cadaver to operate on during an interview for a new position, or lawyers are asked to jump into court unprepared and defend someone as part of the hiring practice.

It sometimes seems that programming jobs need to be a calling. Ie: you spend 50 hours a week at job doing the thing but then are also asked to have a portfolio on the side that you presumably do in your spare time.

It could just be an American thing, though. Perhaps hiring is seen as very risky because sv salaries are so out of proportion to the standard across the rest of the working environment, in addition to there being very little quality control regarding professionalism in the industry that makes technical hiring such a minefield.

BlargMcLarg3• 4 years ago

It could just be an American thing, though.

It's not. You'll find plenty of anecdotes outside the US. Even just outside SV is enough to put things into question, as salaries outside SV are way lower.

Doctors and lawyers do a period of internship after and during the degree that perhaps mitigates the uncertainty around hiring.

Two problems with this.

For one, this is changing rapidly. Any institute of higher education is becoming a worker factory focusing on what the industry wants in candidates rapidly. Despite this, interviewers continue to complain about the most minor things which in reality are very easy to grasp for people with a degree and some projects.

For two, this problem still persists after having some years of experience under your belt. If an internship mitigates the need for other professions to go through this, surely having verifiable experience should do the same. But it doesn't, and it takes away a lot of time from people to build their own portfolio to get through interviews with too.

It must surely boil down to the profession aspect.

I don't believe so. Hiring is plagued with perfectionism and idealism, looking for the perfect candidates and failing candidates over the most minor things. You could actually be the best candidate in the world, and you'll fail because you said something or did something in a way the interviewer preemptively labels as "unviable".

The problem is actually as you point out: hiring managers are pushing risks onto individuals in the name of "calling", and loads of developers are doing nothing to push back on it. Or worse, they are encouraging it. See also why the average junior requirements includes an entire IT department's worth of skills.

jokethrowaway• 4 years ago

Sounds great for testing the candidate but at the same time it sounds like the best way to get the candidate to look for another job.

I still haven't found a company larger than a few employees with a codebase that didn't make me want to leave.

jacquesm3• 4 years ago

That's a good approach if only because it shows how and how fast someone can build up a mental model of what a piece of code really does.

The problem is that the 'existing code' may well be of poor quality and that in order to understand it you first have to get into what it was supposed to be doing in the first place, and this isn't always obvious. So the writer had better take good care to make the code self explanatory or provide additional documentation to give sufficient context. In a way the underlying assumption is that the code is 'good'.

And that's where the real problem lies: lots of code isn't all that good and plenty of it is probably best described as 'single use', in other words: write only. Trying to read it or trying to make sense of it is more effort than writing it ever was. And given the fact that code is typically read many more times than that it is written it pays off to write it well, but hardly anybody really does. The pressure to deliver the next commercial feature is just too high.

And woe to the interviewee that points out the deficiencies in the code if the author of the code happens to be the interviewer, because people will be people, so this could easily turn into a minefield.

Questions to ask of the interviewer before 'reading' the code:

- who wrote it?

- is it functional?

- are you supposed to debug it or explain it?

- was it written for the express purpose of the interview or is it code from the company codebase that is representative of how they work there? (this alone might be reason to terminate the interview depending on how it looks :) ).

throwaway693• 4 years ago

There is no baseline to measure against candidates as opposed to competitive programming problems

whitesilhouette5• 4 years ago

I do almost the same but a bit differently. Like a lot of people have suggested here that they can't share their company's proprietary code (neither can I). So I have cooked up some sample questions asking people to code for a app involving REST and CRUD (because that's resturant what we do at office). It's not much work and can be done in 2-3 hours. Then I get down to discuss their answers and the 'why' questions around their approach. Always gives me a pretty good insight on their work style and doesn't give the candidates any opportunity to cheat (in these remote times) because eventually they would get caught whole explaining their code.

This approach has given me excellent candidates every single time and also led to a lot of time saved otherwise wasted.

ornornor• 4 years ago

It's not much work and can be done in 2-3 hours.

I bet it’s much more work than that. It’s maybe 2–3h for you, who has reviewed dozens of submissions, have designed the problem, and know what the actual solution is.

For the rest of us, it takes trial and error, implementing, polishing (because of course you want to show your absolute best code when being judged solely on your code) and it’s probably actually taking 2–4x as much time as you think it does. No applicant will ever tell you that because they don’t want to look bad.

Try and have people on your team do the challenge, you’ll see how much time it actually takes.

I personally pass on these. If it’s more complex than “implement array.flatten”, it’s going to take way too long and I decline to go further. If it starts with “implement a web app that…” or “an api that…” I don’t read any further and bail.

whitesilhouette5• 4 years ago

It's 2-3 hours for a guy having a 2-3 years of experience (which is what I'm looking for). And yes, I get those attempted by the guys in my team before sending it out. And yes, most of the time, I end up flushing out a few bits of the question before actually sending it out based on their suggestions.

In reality though, the time spent by the guys varies. I always ask them how much time did they spend on the question after selection. The mileage varies from 30 mins to 4 days (because they had office work/weekend trips and attempted the question only when they were absolutely free). No, the hiring HR makes it specifically clear to not try to give a polished code during the initial call and only work 2-3 hours on it.

I see where you're coming from, lengthy problem solving questions are definitely not worth anybody's free time. I've seen problems that can take 1-2 weeks to solve. But for me 2-3 (or even going by your maths of 4x3... 12) hours... is definitely worth your free time... Because 1. that's the amount time you spent while giving your 4 other interview rounds in 4 other companies paying ¼th the salary. 2. Or for a similar company paying an equivalent salary that takes 4 rounds of interviews spread across 4 different days (counting your commute time as well).

Implement array.flatten can be Googled in under a minute. That beats the purpose of a test. I'd rather take a telephonic round and be over with it. The core idea behind my shift was to ensure candidates can't cheat. (And I hate video calls because a lot of them end up having connectivity issues during the interview, especially during the difficult questions).

I don't read any further and bail. Yes. This. You're the kind of guy I don't want my time wasted with. There are a lot of intelligent candidates who'd have definitely aced a telephonic round (or even this written test for that matter) but bail out at the sight of "implement a...". In my experience, you're the kind of a guy suffering from Dunning-Kruger syndrome. The good work is there, but the headaches that such candidates cause on the other 90% of the times makes it not worth my time.

We can argue this to the world's end but in the end what matters is the demand and supply market forces. You can call my process a waste of your time. I'm sure you'll end up getting a better job the next day. And on the other side of this coin, even I'll end up getting a better candidate than you. In my experience, all the candidates that I've hired in this format have been marvelous, so I'm gonna stick to this.... At least until I keep getting great candidates.

woldenron• 4 years ago

2-3 hours

You pay for that wasted time, yes?

dahfizz• 4 years ago

Do you also demand payment when you have to stand in line at the grocery store?

whitesilhouette5• 4 years ago

Sure, I'll make sure you get your salary on time in case you get selected. You're worth getting selected, yes? Why don't you sit for the interview and we'll find out.

Or may be since your time is so precious, we can just skip this written-test-hogwash altogether. Let's just do the traditional telephonic round, shall we? Ohh and by the bye, so long as we are on this topic, you'd pay me for the time I had to waste on you in case you turn out to be a garbage of a candidate, fair?

throwaway14356• 4 years ago

in my ignorance i wondered.. perhaps it is possible to have some automation compare methods used in the employers software with ones used by the dev in their public repos?

Imagine how interesting a job offer would be if they can prove you will get to do things you know well.

EdwardDiego• 4 years ago

In the past, I've taken existing code that needed a good refactor, checked the unit tests gave a good working/not working signal (i.e., deterministic pass/fail), set up the candidate's preferred IDE/editor, and asked them to refactor it to be better, according to their definition of better.

Then we talked about the things they did and why that was better.

Probably the best "coding exercise" I've ever done in terms of getting to understand how someone approaches a typical code base.

But sometimes candidates were very unsure about how to approach it, they found it hard to proceed without knowing exactly what "better" was.

I stopped using this, as I realised that my approach was unconsciously selecting for a certain type of person, one who resembled myself and it excluded people who could be amazing devs within clear parameters.

TL;DR interviewing is hard to get right, this reading idea is a good one.

datavirtue• 4 years ago

Sounds legit. Also gives me a look at the real day-to-day that I can expect. No contrived code, just real production code at the company, please. That way I can avoid the great waste of life that is: thousand line functions without tests.

dennisy• 4 years ago

This is a great idea! I would love to see some real examples, what can people think of that would work for this?

amai• 4 years ago

In my company we go one step further: We encourage the candidates to „bring their own code“. Then we let them explain their code, discuss possible issues, extensions and so on. Usually simply from looking at the code style one can infer a lot about the candidates level of skills. Also the candidates are less nervous and are sometimes really enthusiastic explaining their favorite side project. All in all it leads to a quite good interview experience for the candidate, but also for the interviewer.

Aperocky• 4 years ago

I have a lot of weekend-ish side project on github (e.g. http://aperocky.com/cellular-automata/), but I also know a lot of capable engineers that does not have that. One of my favorite thing to look at for a candidate are their github, but unfortunately more often than not people don't have it or only CS-X01 repository exists. The interview is in no way affected by this as it is not an indication of weak technical ability.

But if someone does have it, it does get looked at, at least through me.

zeroonetwothree3• 4 years ago

How does this work if they only have coded as part of jobs? Do you assume everyone is working on some side project or open source?

jansommer• 4 years ago

Would make sense to provide it as an option, and approach candidates without side projects in a different way. I would much rather show up with code I've already written and use that as a starting point for conversation.

vmception7• 4 years ago

My bet is that it doesn't work. My experience is that they are assuming that, just like those recruiters and hiring managers that look for “an active github” and swear by that

hyfgfh• 4 years ago

I would probably think twice before accepting an offer if I had a peek in the codebase of some companies I worked on. From weird practices to complete useless tests and straight copies of other projects without any planing.

But for first step a Psychotechnical test on the candidate, and to be fair, the team and manager.

sirsinsalot• 4 years ago

For an interview, I took a copy of the product's source code to the interview with me to talk about issues it had.

They couldn't work out how I got hold of it. It wasn't open source and only distributed as a binary.

It got me the job. Risky strategy.

pipogld• 4 years ago

This is brilliant. Thank you for sharing that. I have always thought that writing code in front of somebody else was not indicative of skills. This is showing a very good and simple alternative.

dccoolgai• 4 years ago

Even better: read the code, understand it and then change it.

pkdpic• 4 years ago

Why don't companies just have prospective candidates work for 2-4 weeks on the actual job unpaid and then select 1-2 candidates from that pool based on performance?

Most job seekers are working for free when applying for jobs anyway and it seems like this would have the nice benefit of providing a free rotating labor force for the hiring companies. It might even allow them to do without a couple of paid positions long-term.

kyawzazaw• 4 years ago

Experienced people don’t have time for that.

nonameking2026• 4 years ago

This - this is important!

padheyam• 4 years ago

I am a solo non-technical founder who hired and fired around 10 developers on a freelancer platform before finding the gem of a developer who single handedly completed the platform. Unfortunately, after a year he has left and now I'm struggling to find a replacement. The web app is still running fine, but further development is stalled. I'm wary of giving access to the entire codebase to the would be new hire. And I being a non technical person, have no idea how to give only partial access to my developer. Just rambling after reading the title. Sorry!

SonOfLilit• 4 years ago

I'm a consultant and former technical founder with a lot of experience in helping non-technical people have control over their software projects. Feel free to contact me (username on gmail) to think about how to proceed in this situation.

Edit: obviously interviewing is one of the issues you're facing, but far from the main one. Trust is hard :)

robbywashere_• 4 years ago

When hiring developers, fetch a stoic subordinate who is A) good at eye contact and B) able to maintain said eye contact without blinking.

Have this subordinate and the candidate participate in a staring contest. This will show a couple of things about the candidate: A) How well they maintain eye contact (very important for communication) and B) How long it takes them to back down from a challenge; this taps into their primal instincts.

natpalmer1776• 4 years ago

I honestly can't tell if this is satire or an interesting look into an alien (to me) work culture.

leaflets28• 4 years ago

Gotta tap into their primal instincts.

I would also sound a fire alarm, to see how they react to working under stress, and measure how much their pupils dilate.

forgotusername62• 4 years ago

I've had managers be critical of candidates with poor eye contact before. I've had to remind them that not everyone is good at that. It certainly isn't required for being a software engineer.

cookie_monsta• 4 years ago

Is this one of those tests where you hire the first person to call out your BS interviewing technique?

amon22• 4 years ago

Haha, good one!

daxfohl• 4 years ago

How do you avoid language familiarity bias? I've had one interview like this and am sure I did much worse just because the language was Javascript, which I have limited experience with.

Intuitively I'd expect that not to make much difference since most mainstream languages are pretty similar, and the interview wasn't syntax oriented. But that's not what I experienced. Signal to noise ratio was there and definitely affected speed of processing the code.

(To answer my own question, for coding interview style algorithmic questions, it should be easy for the company to translate the questions to the applicant's preferred language before the interview. (if not then it's probably not a good interview question). Hopefully this company does that.)

Obviously if the company explicitly wants the candidate to be familiar with deep aspects of whatever language you use, then this doesn't matter. I can imagine hypothetical cases where this would hold, but I'd think it relatively rare.

gavinray3• 4 years ago

A good interview should be language-agnostic, if it's not a junior position IMO.

Most of my code-based interviews have been this way, and when I've been the one interviewing, I say: "Use whatever language/tech you like."

mc4ndr3• 4 years ago

Shoot, just ask them bout their own FOSS changes.

"Why'd you design it that way?"

"Why'd you implement it that way?"

"What alternatives did you consider?"

Interviewers be so lazy in 2022 though. 'uh um do a fizzbuzz in ten seconds no your solution not match my first mental solution bye'

mbrodersen• 4 years ago

We ask candidates to review and criticise 2 pages of really buggy, badly written C++ code. It is a great way to find out how experienced a C++ developer is and how good they will be solving hard problems. Writing new code is easy. Fixing problem in code written by other developers is much more of a challenge.

adave• 4 years ago

I liked the article and kudos to reaching top of hacker news.Your article however does not consider real world problems like - What if this technique becomes popular and now there is LeetCode db for your question? In that case its really easy to just know the answer and get far. - Same for reading the code part. Most interviewers have a set of questions they go with to be fair or what not however once you know the code snippet in question you can talk about it.

The real issue is that one approach does not work for all and there need to be uniqueness to every team so an avg candidate cannot just hack the interview process. This is the problem with the current system as well since everyone uses it and its become its own nemesis.

bgro• 4 years ago

But wait, how is this fair to the millions of software engineers that can only pass interviews by memorizing the question? Isn't that the whole point? To gatekeep people with other learning styles and fast track people who arguably cheated on tests throughout school?

When hiring developers, have the candidate read existing code

Related Stories