Why coding interviews aren't all that bad

https://eli.thegreenplace.net/2022/why-coding-interviews-arent-all-that-bad/

by picture • 4 years ago

33 53 4 years ago

eli.thegreenplace.net

Apocryphon3• 4 years ago

The discourse around algorithms/data structures coding interviews (as opposed to domain-relevant pair programming coding challenges) does seem to be universally either about abolishing them or keeping the status quo.

Then what about keeping those topics but improving the process? How about reducing other pain points? Such as:

* Transparent rubrics about how an applicant is to be judged

* Disclosure of what specific topics are to be tested or not, e.g. avoiding candidates stressing over dynamic programming problems only to be stumped by simple string manipulation

* Actual training of interviewers so that they can be more collaborative and helpful during the interview

* Disclosing to candidates just how many questions they are expected to answer during a round

Companies want to keep their Leetcode question banks, fine. But the coding interview process are painful and scary to candidates not only because of any difficulty in problem content, but because of the apparent subjectivity and opaqueness of everything involved. Maybe the current process can be reformed and systemized a bit so neither party wastes their time.

And another subject worth considering: how could current algorithmic pedagogy be improved? How do you take the teaching of runtime costs and complexity, specific data structures, and general principles like dynamic programming, and be able to solve specific problems like, say, merge intervals, without having seen such problems previously? Maybe there’s a layer of “now that you have a toolset, this is a good set of heuristics for matching your methods to actual problems.”

throwmeawaycs7• 4 years ago

You have to treat it like a creative field, albeit one which has logical rigors to it. I believe the correct way of hiring in this structure is to hire people for the Apprentice/Journeyman/Master artisan categories used in medieval times.

There are clearly master coders out there e.g. Linus Torvalds, John Carmack, etc.

I really strongly feel having the master/apprentice structure is the correct way to hire and consider coders, and that many things business wants to do and hires “new programmers” for needs to be considered in terms of how many people you want to hire who can complete the task on their own, and how much help they will need which can be provided in terms of junior/senior positions below the artisan/architecture level programmer.

amitport• 4 years ago

The core interview of this type is still fairly terrible IMO. You're just masking it with a few nicities as many big companies already do. (Those rich enough to have proper interviewer training)

Apocryphon3• 4 years ago

Those big company interviews are still completely opaque, fraught with uncertainty.

BoiledCabbage2• 4 years ago

Honestly, they are really great points he makes.

There absolutely are flaws with interviews. But if you stick with coding questions using simple datastructures non-esoteric data structures, they are the most objective, scalable and accurate method of evaluating a candidate.

jrjarrett2• 4 years ago

Except that skews the candidates that pass younger. If you’re fresh out of school, you remember binary trees or sorting algorithms.

After doing software development for 20+ years, you’ve been using those structures as a small part of the larger problems you now solve. If you go looking for a new job, you don’t necessarily remember all these tricks to invert a binary tree, and you fail. Even though you have all that experience in what you actually will be asked to do on the job.

ratorx3• 4 years ago

I’m a fairly new interviewer at FAANG. I wouldn’t consider “invert a binary tree” to be a good question.

If a question did require “invert a binary tree” to be part of the optimal answer, then identifying this would be the most important thing. If the candidate then coded an suboptimal version, whilst mentioning that a more optimal one could exist, that’s almost max points.

I think there is a slight focus on “tricks” for coding interviews, but from the questions I’ve done and seen, these are more general problem solving strategies, rather than the slightly esoteric algorithmic tricks that get brought up on HN.

I haven’t seen a current question where a very specific algorithmic trick is required to get a good rating (but that could very well be my lack of experience). I guess dynamic programming may be the closest thing. But I would say that’s a more general strategy and the questions I’ve seen that use it are fairly rare. I’ve never seen a question require it for a good rating, the ones I’ve seen have it as extension type stuff.

BoiledCabbage2• 4 years ago

Agreed that it's not perfect.

But asking to talk about technical topics is what is essentially trivia - is an absolutely worse approach.

jokethrowaway• 4 years ago

What the article says makes sense but he's not disproving doing a coding test similar to what you may encounter on the job vs algorithms you'll implement once in a lifetime.

When I'm interviewing, I ask about their projects to see if they're passionate about something and then I test them on something I need to implement for work (or similar enough). It's often something like, implement an API endpoint that does X with Y database, implement this UI component. I'm happy to be flexible on language and technologies the candidate is familiar with.

If you think the job will be much more than that, you're simply wrong.

The closest I got to implement a leetcode exercise at work was when I had to implement pathfinding for a 3d game in a language that is not commonly used for games. Turns out my naive solution was poor, I duckduckgoed 5 minutes and implemented a way better algorithm from Wikipedia.

Besides: If I need to prepare for two months for your interview, I won't apply for your company. Even doing a take home exercise takes way less time than that, I can crank most of those in a couple of hours. Still, I'd prefer a real world exercise at the interview, so I don't need to prepare anything in advance.

martinbooth• 4 years ago

The questions are designed to allow an interviewer to get as much signal in the 45 minutes spent in the interview.. that's why they're contrived rather than your typical piece of work. They're not designed to test you on something you would never encounter though. If a company is asking these kinds of questions and the job only ever requires dealing with npm packages that shields the engineer from this complexity then they shouldn't be doing that in my opinion. But most of the companies asking these questions are in the business of building software that requires this lower-level knowledge

TrackerFF• 4 years ago

My only real problem with coding interviews is that you never know whether or not people lack knowledge, or if they're just folding under pressure. I've seen very competent people start to fumble, when they're being put on the spot - which in turn makes a bad first impression, for the people that have just met them.

Some people are very good at whiteboard interviews, others at take-home exams.

sokoloff2• 4 years ago

If the folding under pressure is episodic/random, then those very competent people will likely make a correctly good impression in the next 1 or 2 interviews they attend and end up landing a job soon. From their perspective, it's still fine.

From a company perspective, if you really can't tell whether they lack competence or just folded under pressure, you pretty much have to pass on them and try again with the next candidate.

keskival3• 4 years ago

Why would it be random? I have seen people describe this as related to performance anxiety, and maybe some part of it is that.

I have myself come to conclusion that this is largely due to brain architecture. Notably the same part of the brain is responsible for coding as for speaking. Asking people to explain their thinking to an unknown person with unknown criteria takes a huge amount of brain capacity exactly from the parts of the brain one needs to program.

Additionally, the normal way experienced programmers approach these types of problems is to utilize background processing in the brain. They first encounter the problem, then they aim to understand it well. Then they go take a coffee, or even sleep on it, while the brain is working in the background unconsciously, and then at some point an innovation comes almost unbidden. The coding interview set up doesn't measure this real problem solving capability in any way. This is why experienced people become stumped, because it is not the way they actually solve the problems in the real life, and they might not even be aware of why their brain seems to not work suddenly.

This might even flare up their impostor syndrome which is a common professional hazard if they think they have failed in something they should have succeeded in. It is not only a bad way to interview and screen people, it is discriminatory and materially abusive.

throwmeawaycs7• 4 years ago

I personally implemented major APIs and libraries that are currently being used in deployment by over 25m+ systems with 99.999% uptime. When I get called into interviews and some guy starts trying to get me to implement or work on some random crap in some random IDE and doesn’t want me to use a search engine to do so?

Guess what, I just turn off the laptop and go hiking.

For a year. And then after that year I’ll try again. You’re not playing the game you think you are.

asd88• 4 years ago

Hot take: Having false negatives is perfectly fine as long as you’re minimizing false positives.

AtlasBarfed3• 4 years ago

That is a bullcrap phrase all interviewers and HR say.

Look, how would you know your interview process isn't filtering critical numbers of good engineers?

You have no control group for that assertion. None.

And engineers are known to be neurodivergent more than your usual meatsack.

For example, your assertion works really well if the subconscious goal is to only hire white people. After all, black people sue for discrimination to better to be safe and avoid them.

keskival3• 4 years ago

The author exhibits huge confirmation bias in their example of an otherwise good applicant stumbling on a technical test. They assume without any doubt that this means they narrowly avoided a mishire.

Testing for simple algorithms knowledge biases for recently graduated people who have these in their active memory. Experienced people rarely encounter these details in practice and are more likely to have their heads full of deep knowledge no one in your organization knows about. If you screen these people out, it will be quite literally your loss.

Technological fields are blue oceans where only junior people spend significant time near the safe harbors. Experienced people have sailed far away and seen lands you don't know about yet.

People also have this inherent bias of treating everything they know well as some common knowledge everyone else knows as well, or at least should know. This bias of underestimating the vastness of the field is largely because of inexperience. People who haven't seen how deep it goes are liable to think that the basic university exercises are the important things.

jrjarrett2• 4 years ago

Yes, exactly-this is what I wanted to get across in my comment down post.

marssaxman3• 4 years ago

Testing for simple algorithms knowledge biases for recently graduated people who have these in their active memory.

I have never understood this argument. With 29 years of experience, I am about as far from being a "recently graduated person" as anyone could be - in fact, I never graduated at all! Not only did I drop out after three semesters, I never took so much as one computer science class.

I have, according to your assumptions, never had algorithms knowledge in my active memory - and yet, these sorts of questions have never been an obstacle for me. How can this be?

I think many people must mistakenly believe that interview questions are the same sort of questions one might find on a university test. They appear to believe that the interviewer expects them to regurgitate a correct answer, on the spot, from memory, and that their suitability for the job is rated based on the speed of recall.

That would be absurd, of course. If anyone is actually interviewing people that way, they're doing a terrible job.

I have never answered these questions that way - and, when I ask such questions in an interview, I never expect them to be answered that way.

Rather, the point of an algorithms problem is that you have to think your way through it. I solve them in the same way I would solve any problem I encountered in my day-to-day programming work: by thinking about it, poking around at different approaches, roughing out a simplest-possible approach, then refining toward a more efficient process.

If I am interviewing you, and I pose you an algorithms problem, I'm hoping you will not have memorized an answer. If you already know the answer, your skill at writing it out on the whiteboard tells me nothing. The whole point of asking you to think your way through an algorithms problem is that I want to see how you think. I don't even care whether you find your way through to a correct answer in the time allotted, so long as I can see that you know how to think about a problem, work your way toward a solution, and communicate about your process as you go.

The common idea that experienced devs ought to struggle with algorithms questions dismays me. What are people doing that does not require them to maintain basic problem-solving skills? Or... perhaps... is this all a big misunderstanding...

bcbrown2• 4 years ago

That, of course, would be absurd. If anyone is actually interviewing people that way, they're doing a terrible job.

Perhaps you have never experienced that sort of interview, but it doesn't mean it doesn't exist.

I was once asked a question where the solution involved implementing a trie. The naive design involves a 26-element array to hold all the possible next-letter entries, which obviously takes a huge amount of memory. The interviewer expected me to know the trivia answer on how to avoid allocating all that memory, and was clearly disappointed that I didn't know (using a linked list instead of a fixed-size array: https://en.wikipedia.org/wiki/Trie#Implementation_strategies). This was a principal engineer interviewing another principal engineer, FWIW. The company was Qualtrics, a survey/customer satisfaction company, not a bare-metal algorithms-heavy company.

Obviously you've managed to have a successful career without encountering interview questions that hinge on algorithmic trivia, and I'm not doubting your experience. But saying "it's never happened to me so therefore it can't exist" isn't an especially strong argument.

gcheong4• 4 years ago

That’s how it should work but it seems to have largely gravitated toward, not rewarding memorization exactly, but definitely pattern recognition gained from drilling on a wide arrangement of leetcode style problems, and as much as they say they want to understand your thought process your performance is heavily weighted towards whether or not you arrive at the optimal solution in the time given, not how well you might explain how you might get there even if your code is ultimately lacking due to time constraints.

bradlys• 4 years ago

How often do you interview and how often are you doing LC interviews?

There’s no real time to “see how one thinks” when the expectation is to pump out two medium answers in <45 minutes. You’re basically just reading the question, going through a couple hoops, and then implementing/explaining the algorithm. There’s barely any time to “see how one thinks” in these modern interviews.

Maybe you can show three problems you were asked recently and how long you had to solve them.

bcbrown2• 4 years ago

Technological fields are blue oceans where only junior people spend significant time near the safe harbors. Experienced people have sailed far away and seen lands you don't know about yet.

I really like this analogy.

kag0• 4 years ago

The author asserts that simple coding interviews

involving nothing more than simple and fundamental data structures

is the objective way to evaluate a candidate. Because those

do require a good understanding of programming fundamentals

But in my experience, someone who can memorize those basics can also memorize their implementations. So it is quite possible to have someone code a perfect BFS, but have no grasp of programming fundamentals.

ovao• 4 years ago

I can appreciate the author’s reservations around take-home assignments given how easy it is to cheat, but why not as an interviewer simply protect against that by asking the interviewee to detail certain aspects of their implementation on a follow-up call?

Whenever I submit a project for review, I’m prepared to answer questions about why I did X, why I used this tool/library/approach versus some other tool/library/approach, and what I could do to improve the implementation further. On the latter subject, I’ll raise that discussion even before being asked. If your applicant panics on questions like this — or is obviously clearly making shit up on the spot — there’s your indicator to reject.

If you ask me to implement a B-tree, I’ll describe what a B-tree is and tell you I don’t off-hand remember how to do it or what an ideal approach would be in a given language. I’d tell you that I’d research it a bit before diving face-first into it. Which is what almost everyone would do in real-world contexts anyway.

throwmeawaycs7• 4 years ago

Ah, to me that sounds like you are a good engineer and what they actually are seeking is a good “follower/code-servant”.

You see it’s good to filter applicants like you out because you may do nasty things like “question management decisions” and “think creatively”.

We want obedient slaves with social media accounts, Damn it!

hackerfromthefu• 4 years ago

Yes, this is one of the things that arduous processes filter on.

doktorhladnjak• 4 years ago

That coding interviews are objective is no argument in their favor. The author provides no evidence that what's being "objectively" evaluated matters at all. They could just as soon choose the candidate whose name comes first in the alphabet or who's the youngest. Those are also objective, but not meaningful hiring criteria.

gcheong4• 4 years ago

What’s missing from all these speculations about the best way to hire is any tracking of the applicants performance after the hire. If you showed me data that said that the highest percentage of candidates that do well at the company all did well on their algorithm interview vs any other screening technique then I would be more convinced. Google claims this is the case and the industry seems to have just followed suit without questioning it. I personally think the best way is probably a short simple fizzbuzz type problem to screen out people who simply can’t code at all then a take-home or in-house coding assignment with a discussion after or (or during if in-house). If the claim that avoiding a bad hire is far better than rejecting a good candidate is to be believed then it behooves the hirer to get as much signal around something that closely resembles the actual works as possible and actually discussing the code at hand would probably eliminate most problems around cheating.

hansvm2• 4 years ago

behooves the hirer to get as much signal around something that closely resembles the actual works as possible

Is there any reason why closely simulating the work environment should be the most efficient way to glean whether a candidate will perform well once hired?

gcheong4• 4 years ago

I wouldn’t think it would be the most efficient but if you’re worried more about accepting a bad hire than rejecting a good one efficiency isn’t the primary metric.

stitched2gethr• 4 years ago

I'm going to be honest, I'm a backend developer and I've never written a tree in production. I understand the structure, I've written them for practice, but for real world code it's just never come up.

It seems that for the overwhelming majority of developer roles these data structures are outsourced to a database, which is usually closer to the data than my service anyway.

It's not that this knowledge is irrelevant, but there are so many aspects of writing, and maintaining, business logic with a team of others that are skipped entirely.

kevindong• 4 years ago

I've personally experienced three kinds of coding interviews.

1. Trivial questions that can be done very quickly; e.g. does this list of integers contain any duplicate integers?

2. Non-trivial questions that require a large amount of time to solve if you haven't memorized the solution; e.g. the skyline problem https://leetcode.com/problems/the-skyline-problem/

3. Practical questions; e.g. a question that more-or-less represents something you might actually encounter in your day to day job, for instance querying multiple APIs and joining the data together into a particular expected format

---

1 isn't bad due to its level of ease. 2 is awful since you mostly just need to grind many hours of Leetcode to be able to consistently solve those kinds of problems. 3 is enjoyable since the problems given might actually teach you something you'll use day-to-day.

AtlasBarfed3• 4 years ago

The tree mirror problem is funny. You don't even need to recurse. Just define a wrapper for the node object that gets left when right is accessed and right when left is accessed. Voila!

Right and left are arbitrary anyway.

throwmeawaycs7• 4 years ago

Actually in our problem we wanted you to do it without a wrapper so while this works, 0/100

AtlasBarfed3• 4 years ago

Can I rate your question? Why am I asking permission. 0/100

seanmcdirmid• 4 years ago

As an interviewee, I like coding interviews for another reason: they are so straightforward to prepare for. I have no idea what they say about my abilities as a SWE, but 4 coding interviews loops beat open ended loops where I don’t know what to expect. Yes, they have made intense practice of coding problems and prepping mandatory, but it isn’t a mystery.

But I don’t think they yield quality hires much more than the other ways, which all seem to be crapshoots anyways. Credentials and referrals seem to still give the best results, but they are necessarily biased to those who network.

Why coding interviews aren't all that bad

Related Stories