“Design me a highly resilient database”

https://nikogura.com/DatabaseDesign.html

by donutshop • 5 months ago

53 45 5 months ago

noemit• 5 months ago

Hiring good engineers is not that hard. Why do we keep screwing this up?

keyle• 4 months ago

If anything it shows the applicant was over qualified and it probably wasn't a great fit; to have this guy, who has battle experience, in a place where the interviewer was looking for a one-size-fit-all canned type answer.

rockemsockem• 4 months ago

Hard for a bad engineer to hire a good one maybe? If so, not the biggest problem to have.

pinkmuffinere• 4 months ago

Hiring good engineers is not that hard

I feel this is misleading. I think a reasonable definition of a 'good' engineer is engineer in the 75th percentile and above for their craft. You can debate other definitions, I'm not saying this is the only reasonable one. But by this definition, I think you can immediately see that it is hard -- given 4 engineers, we expect that only one of them is "good". And what if your standards are higher? What if you only want _great_ engineers? I can't believe that it's easy.

carefree-bob3• 4 months ago

It's hard because people are threatened by good people and don't want to hire them, and also objective standards leave less wiggle room to sneak in your friends or others in your peer group.

Many, many forces lined up against meritocracy. All the incumbents, for example, tend to have something to lose, but it's the incumbents who decide if you are hired.

It's a miracle that there are any good companies once the original owners are gone.

mcny2• 4 months ago

If I am hiring, I need these characteristics:

1. Person who cares 2. Person who is knowledgeable 3. These two must be the same person!

kulahan• 4 months ago

Please explain how to hire good engineers. This is a question literally only you have figured out, so your knowledge would be extremely useful here.

tintor• 4 months ago

You are better off not working there.

scrubs• 4 months ago

The interviewer question was adversarial... don't think it puts them in a good light.

More importantly I appreciate the OP for describing lessons learned through this hard experience. I worked in finance: Cassandra is no kind of answer for systems of record

Thanks

andersa• 4 months ago

At Apple Pay, we processed millions of daily transactions across 30,000 servers

Why does it require 30,000 servers to process millions of daily transactions? Is there a typo in the amount?

dworks• 4 months ago

Maybe globally spread out POS servers if that's how it works?

carefree-bob3• 4 months ago

This can be read in two different ways

wrs• 4 months ago

I really appreciate that your response to being interviewed by an idiot is to write, not an angry rant, but an educational blog post. You are a good person and they didn't deserve you.

joeconway• 4 months ago

Having worked with Nik in the past, yes he is a great person and one of the best. Any company missing out on him is genuinely foolish

wewewedxfgdf• 4 months ago

The best interview question to find great developers is this:

"What number am I thinking of?"

Smart developers - really smart developers are always right. Truly great developers have seen this sort of problem before and know how to attack it - they need no fancy questions or "deeper understanding". They simply hit the whiteboard, do the logic and output the answer.

Don't settle for anything except A players.

doublerabbit2• 4 months ago

    % isIt3
    10
    0
    6
    4
    8
    0
    9
    5
    I found three
    ----
    proc isIt3 {} {
     variable 3 [expr {int(rand()*12)}]
      switch $3 {
       3 { puts "I found three" }
       default { puts "$3" ; isIt3 }
      } ;# end switch
    } ;# end proc

The number you're thinking of is 3. (-:

bitwize2• 4 months ago

So how well would a candidate do if they looked at that and thought "whoa, is Tcl really tail recursive?!"

tmoertel• 4 months ago

"What number am I thinking of?"

I don't think this is a good interview question, but I do think it is interesting as a thought exercise.

I bet that perhaps 25% of candidates could actually answer the question correctly, even if they didn't know anything about Schelling points. It could also lead to some nice discussions about how to solve an open-ended problem, probability distributions, strategies for maximizing payoffs when making decisions in the face of uncertainty, and so on. The question is so bad, it's actually kind of good.

fogzen• 4 months ago

What answer are you looking for?

jasonwatkinspdx• 4 months ago

I'm glad you ask this, as it signals very clearly the need to get up and walk out of the room.

Same with your last sentence.

roncesvalles• 4 months ago

The solution to this entirely depends on whether you are willing to answer other questions about the number and of what nature. Would you answer: is it higher/lower than X? How about how many digits there are?

But assuming you are unwilling to answer any additional questions, the correct solution is to simply brute force every number up from 0 and its negative.

mooreds• 4 months ago

_thinks: "gotta be a sarcastic comment"_

JSR_FDED• 4 months ago

I think it’s a great interview question. Open-ended, rewlly leads to a good discussion that can give you a sense of how the applicant approaches a problem and thinks it through.

Disastrous in the hands of an interviewer who’s looking for a one word answer.

LPisGood• 4 months ago

I’m not usually one to complain about AI being used to generate content because I find this an especially interesting topic, but it really took me out of it.

More relevant to the topic at hand, I feel like I didn’t put that much thought into designing my interview questions, but I’m always astounded when I read these sorts of descriptions of other interview questions. There should perhaps be some formal training or certification or something for asking good engineering interview questions.

Freedom2• 4 months ago

There was one person on this site who looked down at candidates (for a normal software development job) who didn't answer his interview question well. And by well, I don't mean being curious and working it out on the fly, I mean that if the candidate didn't know the answer of the top off their head, the HN commenter deemed them uncurious and unsuitable. The question?

"How do mobile phone towers work?"

glenpierce• 4 months ago

Electricity goes in and phones come out, duh.

Joel_Mckay• 4 months ago

Interview questions get absurd when people are gatekeeping for various reasons.

People steal/phish resumes off sites all the time, and it is really hard to remain polite while listening to someone lie. I've met 300lb round cons that claim they regularly climb broadcast towers =3

getnormality• 4 months ago

Good time to stand up, shake hands, say "we'll be in touch!" and leave.

Sometimes you gotta fire your interviewer.

8note• 4 months ago

something is off here.

naming "cassandra" is not designing a database.

it feels like there was some miscommunication along the way

bitwize2• 4 months ago

Bahahahahahaha.

You sweet summer child.

The things I've seen at certain worksites...

skydhash• 4 months ago

I'm glad I read the "Designing Data Intensive Applications" book because I get the full context of the blog.

And the lesson of the blog is why forcing LLM usage is stressful to some engineers. Most implementations are in a quantum state and the back and forth to have clear requirements is what can make the wave function collapse. We know all the possible answers , but all of them are equally valid.

We could go with the LLM route and just do a quick guess on the answer, but "Accountability" and "Professionalism" prevent us from doing so.

tantalor• 4 months ago

/dev/null is pretty resilient

carefree-bob3• 4 months ago

I like your style

mcny2• 4 months ago

The database is resilient. Your data is not

doublerabbit2• 4 months ago

The answer he was looking for was ‘Cassandra’. “You’d have a hard time having it NOT be resilient.”

A "database" that starts to fall if the ring in the cluster becomes choked? Draining a broken Cassandra ring and under high load is hell and ball clenching.

Especially hoping that another Cassandra node doesn't fail under high load. Pish, Erlang and mnesia.

tombert• 4 months ago

At a job interview in 2016, I was given a fairly typical "design Twitter" whiteboard problem, so I started doing a fairly typical thing involving load balancers and Nginx and the like.

The topic of the database came up, and I asked "Do we care more about consistency or availability?".

He responded back with "we want both".

I said "I mean, I would like both too but I'm pretty sure we can only get one"

He disagreed and we went back and forth, and eventually I pull out my phone and pull up the Wikipedia for CAP theorem. He eventually (and begrudgingly) agreed and said "umm, I guess availability", so I used Cassandra and moved on with it, though five minutes were wasted with this pointless argument.

To be fair, they did make me an offer, and it was even a decent enough offer (though I ended up taking a different job), but I remember being kind of flabbergasted that I had to explain CAP theorem to a person who had like a decade more experience than I did. It's not like CAP theorem is this super deep arcane topic; I'm pretty sure it's one of the first things you learn about when getting into distributed systems. It was certainly one of the first thing that I learned about.

nubg2• 4 months ago

Not because it’s trendy. Because ACID compliance
The CAP theorem is not a suggestion. It’s a mathematical proof.
this is not a philosophical inconvenience. It’s a compliance violation.

@dang, i thought we just banned AI written blog posts?

I know I will get downvoted for this everytime I post this comment, but I'm sick and tired of 80% of everything being written/heavily edited by AI.

Can the author not just use his own words? Is he really that bad at writing? But somehow the content is supposed to ne worth my time?

I'd rather read the prompt instead, so I know which parts the author actually meant to say, rather than wondering which parts the AI invented to make it sound flowery and punchy, the Tiktok style of prose.

gnabgib• 4 months ago

@dang doesn't work

Can you post more substantial comments than?:

https://news.ycombinator.com/item?id=47360241

https://news.ycombinator.com/item?id=47355553

https://news.ycombinator.com/item?id=47355564

https://news.ycombinator.com/item?id=47345091

https://news.ycombinator.com/item?id=47343706

https://news.ycombinator.com/item?id=47337595

https://news.ycombinator.com/item?id=47333400

https://news.ycombinator.com/item?id=47269468

https://news.ycombinator.com/item?id=47258669

https://news.ycombinator.com/item?id=47228776

nubg2• 4 months ago

I reject your accusation that they are not substantial.

Leave me alone.

Edit:

lmao at stuff like https://news.ycombinator.com/item?id=47359650

very substantial!

seer• 4 months ago

When I’m interviewing I never ask a question about something I know super well. I circle around what the candidate signals he has great passion and understanding at, and start deep diving into that.

If I know about it as well, then we can have a really deep discussion, if I don’t- then I can learn something new.

The aim when interviewing is to check how well / deeply the interviewee can think through a problem.

If we pick a topic that they don’t have deep knowledge - they can either stumble and freeze emotionally, or hallucinate something just to appear smart. At this point it is an examination not an interview. And sure some people are capable enough to get to an answer, but that’s more of a lottery than a real analysis.

It usually boils down to how often have they interviewed before and been in a similar situation. And “people who have interviewed a lot” is hardly a metric I want to optimise for.

Now picking something they know or have expressed interest or passion in, this means we are sure to have more signal than noise. If the interviewee’s description is more of a canned response - then I delve deeper or broader.

“I’ve managed to solve this issue by introducing caching” - “Great, are there other solutions? How do you handle cache invalidation, what are the limits? What will you do if the load increases 10 fold, would you need to rethink the solution?”

OutOfHere• 4 months ago

For the sake of argument, a smart engineer can run Sqlite more resiliently than a dumb engineer can run Cassandra resiliently.

ofrzeta• 4 months ago

"Kubernetes backed by S3-compatible object storage for WAL archiving — giving you 11 nines of durability on your backups .. " - that might be the case with AWS S3 but maybe not with Minio or even Ceph when you need to operate it yourself.

Anyway that interview question is so stupid it hurts. I would never expect just a simple answer to an interview question and mark it off as "done". It's much more valuable to get into a dialogue and see how the candidate is approaching things.

anal_reactor• 4 months ago

Imagine you have a small modeling agency. You have like 10 girls, most of them hover around 6/10 in looks department, you send them to do ads for small companies, assist small events and so on. One day you see an application - 10/10 girl with tons of experience. Way above what you're doing here. What is the correct move? You should reject her application - having ten 6/10 girls and one 10/10 girl will create lots of drama, which will offset any gains from having her on board. If you want your agency to improve, you should hire a 7/10, maybe 8/10.

Exactly the same principle applies with engineers. If you have a team of idiots and you hire one of top programmers, you'll create friction in the team.

“Design me a highly resilient database”

Related Stories