dondraper36

The idea sounds related to the distinction between essential and accidental complexity.

https://ferd.ca/complexity-has-to-live-somewhere.html

I really like the article above in that complexity does have to live somewhere. If you have a hard problem to solve, there is a minimum level of complexity (essential complexity) that can't be further reduced, only moved from one place to another.

And yet we often add accidental complexity by making bad choices (thinking of you, microservices). If your choice is bad enough, you get all the disadvantages (challenges with distributed systems, complicated debugging), but without any of its advantages (a distributed monolith doesn't get you any deployment or team independence).

Telling one from the other, well, is something I'd like to learn and that seems to require both experience and some sort of intuition.

If I understood correctly, the queue implementation in the blog post holds a transaction while an operation is in progress.

I see the advice to make it as short as possible, but why can’t we update the status column to, say, “processing” and avoid potentially long transactions at all?

Not a novel idea, but it always resonates with me when I hear that the goal of every software engineer is to manage complexity.

That said, I still find it hard to formulate for myself how to tell the difference between accidental and essential complexity.

It’s a fantastic performance booster for a lot of mundane tasks like writing and revising design docs, tests, debugging (using it like a super smart and active rubber duck), and system design discussions.

I also use it as a final check on all my manually written code before sending it for code review.

With all that said, I have this weird feeling that my ability to quickly understand and write code is no longer noticeable, nor necessary.

Everyone now ships tons of code and even if I do the same without any LLM, the default perception will be that it has been generated.

I am not depressed about it yet, but it will surely take a while to embrace the new reality in its entirety

As a person with ADHD, I feel personally attacked.

As Martin recently posted, now the production should take about 4 more months until we can buy the paperback version.

That said, if you have an O'Reilly subscription, you can already enjoy the new edition.

+1

I'd also like to understand whether there are still any cases when MongoDB is the right choice

Also, take a look at windmill.dev. It’s a beast

12 out of 13 chapters are already available on the O'Reilly website in the "raw and unedited" format.

The book is scheduled for publication in February 2026, but if you have an O'Reilly subscription, you can already access the new content.

That's exactly the "simple vs easy" difference that Rich Hickey emphasized in his famous talk.

Even in this thread, there was a comment (now deleted) saying that a staff engineer at GitHub is unlikely to know what real scale is.

Not directly related to the article we're discussing here, but, based on your experience, you might be the ideal kind of person to answer this.

At the scale you are mentioning, even "simple" solutions must be very sophisticated and nuanced. How does this transformation happen naturally from an engineer at a startup where any mainstream language + Postgres covers all your needs, to someone who can build something at Google scale?

Let's disregard the grokking of system design interview books and assume that system design interviews do look at real skills instead of learning common buzzwords.

That's where the familiarity factor steps in.

Until you get enough experience for your own good judgment, you need some rules of thumb and guidelines from more experienced peers.

As long as you understand that everything is a trade-off and, unfortunately, that the modern field is based on subjective opinions of popular and not necessarily competent people, you will be fine.

I see your point, but, taken to the extreme, all it leaves us with is "everything is a trade-off" or "there's no free lunch".

Some generalizations are necessary to formalize the experience we have accumulated in the industry and teach newcomers.

The obvious problem is that, for some strange reason, lots of concepts and patterns that may be useful when applied carefully become a cult (think clean architecture and clean code), which eventually only makes the industry worse.

For example, clean architecture/ports and adapters/hexagonal/whatever, as I see it, is a very sane and pragmatic idea in general. But somehow, all battles are around how to name folders.

The author is a staff engineer at GitHub. I don't think they haven't worked at scale

Isn't reading the article before posting comments considered cool anymore?

... and nevertheless at the end of the article, the author does offer their understanding of the terms

Such a familiar feeling. Articles similar to this one make lots of sense to and I do try to embrace simplicity and not optimize prematurely, but very often I have no idea whether it's the praised simplicity and pragmatism or just a lack of experience and skills.

I don't think the author (or anyone else) could come up with term definitions that would satisfy everyone.

Absolutely. To put it differently, unfortunately not everyone has a chance to be part of a product's organic evolution from "all we need is Postgres" to "holy crap, we're a success, what is Cassandra by the way?"

I believe even at FAANG-like companies, only a lucky minority is involved at that level of scale. Most developers just use the available infrastructure and tools without working on the creation of S3 or BigTable.

Yes, I would probably phrase it like this. "Under the current load, I would go super simple and use X, which can work fine long enough until it doesn't. And then we can think about horizontal scaling and use Y and Z". Then proceed with a deeper discussion of Y and Z, probably.

After all, interviewing and understanding what your interviewer expects to hear is also a valuable skill (same as with your boss or client).

Unless it's encouraged by the modern technical interviewing culture, which it partly is.

There's another reason for that. Deep in my heart, I would love to be part of a team that works on truly data-intensive applications (as Martin Kleppmann would call them) where all the complexity is justified.

For example, I am more of the "All you need is Postgres" kind of software engineer. But reading all those fancy blog posts on how some team at Discord works with 1 trillion messages with Cassandra and ScyllaDB makes me envious.

Also, it seems that to be hired by such employers you need to prove that you already have such experience, which is a bit of a catch-22 situation.

This also happens because plenty of candidates learn the buzzwords and patterns without understanding the trade-offs and nuances. With a competent enough interviewer, the shallowness of knowledge can be revealed immediately.

I enjoyed reading this book (it's a short one), even though the prose is very, well, special :)

Yes, and this is exactly why LinkedIn-driven development exists in the first place. Listing a million technologies looks much more impressive on paper to recruiters than describing how you managed to only use a modular monolith and a single Postgres instance to make everything work.

Not really disagreeing with you here, but that "later" never comes for most companies.

Especially given that generating documentation and tests (of course, with manual revision) is so much faster with, say, Claude Code.