Microservices [video]

https://www.youtube.com/watch?v=y8OnoxKotPQ

by abhisuri97 • 6 years ago

261 52 6 years ago

youtube.com

saamhaz• 6 years ago

I’ve watched this only about 50 times nbd

woofie113• 6 years ago

For every microservices system I've seen, this feels about right.

It oversimplifies things a bit, though.

It's odd; I've never seen a "monolith" simplified by moving to microservices, but everyone seems to be doing them.

KorematsuFred• 6 years ago

Microservice design was not meant for [only] a simpler system but ease of dividing people into smaller teams where a team could focus on a single objective without worrying about rest of the company.

If the microservice architecture has made you less productive than the top leadership is to be blamed.

ramenmeal• 6 years ago

I think most slow downs are due to single teams owning many (20+) services. Usually it's their own choice too.

owenmarshall2• 6 years ago

This is absolutely right in my experience.

Microservices are all about scaling development agility in exchange for increased technical overhead.

In many cases this tradeoff is atrocious. If you can say "we can easily take on (large feature XXX) and deliver that", microservices are all pain and no gain.

Where they really start feeling good is when your "two week sprints" turn into about two days of available dev work because the remaining time is spent regression testing your changes and trying to keep the build from breaking because of tests you didn't touch failing (spooky action at a distance is real and it hurts). A clear API contract that both sides of the fence can develop against means that you can keep making progress.

At least until it's time to integrate against the real world, not your mock ;-)

Johnie• 6 years ago

Conway's Law applies here [1]. Poor architectural design can be rooted to how managers organize the team. If you want to see the architecture designed a certain way, organize the team structure along that line and the architecture desired will naturally emerge from the teams. If you have a misalignment of teams and architecture, you're always bound to hit friction along the way.

As you move up in leadership, the tool that you have is organizational design. Doing it right and you have a highly efficient org. Do it wrong, and you'll constantly hit roadblocks.

[1] https://en.wikipedia.org/wiki/Conway%27s_law

bartread• 6 years ago

...ease of dividing people into smaller teams where a team could focus on a single objective without worrying about rest of the company.

How's that working out for you?

In most cases, and I'm talking right up to the scale of very large enterprises[1] you don't need that much complexity to scale technically and, if you avoid the complexity, you don't need that many people either.

And I don't think you can ever afford to be myopic if you want to effectively deliver solutions to real business problems.

[1] 90k employees, specifically, but also, specifically, not a tech company. A traditional company with a tech function.

jayd16• 6 years ago

I've seen the same thing in monoliths when I was a young contractor. Inheritance hell is usually also involved at that point.

bob1029• 6 years ago

We simplified a monolith by having discrete services within it. Each a self-contained vertical unit of functionality, but in a shared namespace with perfect type visibility & enforcement across every service. So, we can dispense with all of the fanfare and just do direct method invocation across 1 DI scope in a single fat binary. It works perfectly. No contract issues to negotiate and developers can work in parallel with zero fear of breaking shared components, as long as intended interface changes are discussed up-front.

From a high-level perspective, is there really any difference between DMI and HTTP/gRPC/et.al. if you could hypothetically run all of your services on a single logical node? If your entire production stack runs on your macbook, why cant you just wire it all up as a single binary and throw it on a nice powerful server? I guarantee it will run better without all that ridiculous network glue in the middle.

If I am being modest, I would say that we feel really, really stupid from where we are sitting right now. Primarily for having previously spread this implementation across 10+ services talking HTTP for absolutely no good reason. I am growing into a mindset where the entire business system needs to fit into 1 binary image. Perhaps duplicated across more than 1 physical host with differing features enabled per, but at least sourced from the same solution/repository.

woofie113• 6 years ago

That's how I like to code. That's just called good abstraction.

What microservices allow is for different teams to use different technologies and to upgrade components independently. On service might by Python+postgresql, another Ruby, and third also Ruby but two versions ahead.

It feels fast and agile in the short-term to decouple teams like this. In the long-term, it's almost always a maintenance nightmare. You end up with way too many different technologies.

I usually break up my projects into libraries (in Python, independent pip-installable packages) with their own unit tests, and some pretty weak glue in the middle. The service layer is omitted. I design things to scale horizontally too.

I've built apps with a few services, but I've never seen a good example of a microservice architecture that beat good old-fashioned clean abstractions in libraries.

Every few years, someone comes up with a new way to do abstraction, and compares that to no abstraction (rather than good abstraction with the previous model. Microservices seems like the dumbest of the bunch I've run across in my career.

occsceo• 6 years ago

hits too close to home.

Mountain_Skies• 6 years ago

The current trend of adding microservices all over the place feels an awful lot like DLL Hell. Don't know if it will ultimately end up that way, but it sure sounds like a rhyme.

fugazithehaxoar• 6 years ago

This was my exact experience working at Nike 4 years ago. In this scenario, I was the guy sitting. I got this type of "talking to" when I submitted a bug ticket because jQuery was being loaded 3 times on the same page.

The Nike.com software group was over 1,000 employees working on a microservices mess that performed horribly on the customer side. They could have built a much better product with about 50 competent software professionals.

capableweb• 6 years ago

Brooks's Law in effect, same as in all the other places where managers believe that 9 women working together could make a baby in one month. https://en.wikipedia.org/wiki/Brooks%27s_law

pdr2020• 6 years ago

Webpack 5's modules federation ;)

fareesh2• 6 years ago

I remember reading a blog post some months ago here from a former AirBnb employee where they talked about how some senior manager in the company wrote an incredibly detailed spec for a CRUD feature which required them to invent a datatype because they insisted on using no more than N bytes to store some inconsequential field. There was also a description of how the author had to crawl through a parking garage to escape a meeting wherein he insisted that nobody was allowed to leave.

I feel like there is no accountability for these clowns. They'll inevitably switch jobs and ruin the work lives of so many developers with their stupidity. There is a culture of not wanting to name your abuser for some reason. It's weird.

khaledtaha• 6 years ago

I believe this was at Snap Inc. I’ll try to find the link later.

maxmouchet• 6 years ago

This is it: My Time at Snap https://news.ycombinator.com/item?id=21122123

fareesh2• 6 years ago

My bad!!

Nextgrid• 6 years ago

I feel like the technology industry as it stands promotes over-engineering and encourages it. There are a lot of developers out there that built their entire careers working on (or designing) over-engineered balls of mud and thus will bring that way of thinking into any new role they join, thus spreading the problem to their next company (if they don't have this problem already). There can't be "accountability" when nobody considers this behaviour a problem to begin with and actually hires based on how many over-engineered things you delivered in the past.

At a previous employer I remember they had someone build a React front-end which talks to a GraphQL proxy which itself talks to a REST-based API and seemed proud of his creation. There was absolutely no reason why the front-end couldn't talk to the REST-based API directly (it was on the same domain, and the front-end already had a valid auth token), but now there's one extra moving part in the system that can (and did) fail.

At another place they for some reason decided that they needed React despite only a single page of the application actually needed to be "real time". 9 months later they still didn't even have a functional registration form and then it took them over a day for the basic feature of displaying the QR code for two-factor auth enrolment. The same signup form would take me less than a day to do in its entirety using "boring" HTML pages and forms (which our framework already had lots of helpers for, so the backend code for it would've been around 100 lines).

tabtab• 6 years ago

Surgeons usually recommend more surgery and pharmacists more pills. It may not be conscious bias, but human nature tilts each to favor solutions toward their own skill sets.

My manager want's "pretty URL's" even though it's not a good idea to bookmark CRUD detail pages. It creates internal code messes to keep them pretty. It's yet another me-too feature that bloats the stack. I proposed a compromise, but he really wants them pretty.

There are too few K.I.S.S Cops in IT; the Bloat Mafia reigns. Warren Buffett says part of the secret to getting rich is confidence and conviction to say "no" when the logic and numbers don't support an idea but peer pressure says "yes". https://news.ycombinator.com/item?id=22795294

_y5hn• 6 years ago

Like with any cult, the victim get double abused, evicted and the people remaining justify why it had to be so. This is the normal behaviour.

You really have to work within the system, and in the end, become the same system.

stronglikedan• 6 years ago

There is a culture of not wanting to name your abuser for some reason.

No one wants to burn bridges. The world gets really small at the most inconvenient times.

nkassis3• 6 years ago

This was linked to me right after coming out of a meeting with my product team where I was the engineer in this video. Worse, I had a large hand in defining how our software architecture is designed so I can't even blame other people. It's largely my fault.

makstaks• 6 years ago

Thanks for sharing this, not easy to admit. In retrospect what would you have done differently though?

bsaul• 6 years ago

"why i screwed up" is the kind of talk you don't see often, and yet it's probably as instructive as the other ones.

nkassis3• 6 years ago

That's a good question, a lot of the issues we hit now are mostly due to early assumption informed by our business requirements at the time that didn't hold true long term (argh blaming the business trope sorry). Our company being a a startup still learning market fit for our product went from being SASS first to doing mostly on prem and private cloud installations for example. That changes a lot of how I would have approached the design to reduce operational complexity had that been obvious early on.

Similarly, we did a lot of learning from the tools we used that is probably very much just something you earn with battle scars. For example the behavior and issues we hit with our choice of core stack components (celery, redis, rabbitmq, kubernetes,...) are often things you learn by using them and gaining experience.

The next thing is there were a lot of conscious tradeoffs that I would still argue we would take today. We prioritized speed of building features to make the product viable and attractive to customers at the expense of quality of those features and reliability. Doing the hard real world testing instead of relying on simple unit/integration tests before shipping. That's something we now are having to address but the question is was that something we should have done differently? No sure I would, we may not be around now if we hadn't built the feature we needed to sell the product and prove market viability.

On the topic of this video. I think the issue it not using microservices or building a monolith but controlling complexity. The biggest thing I dislike about our current infrastructure is the cognitive load and complexity that a new engineer joining has to deal with to be productive. That can happen in any architecture if you aren't careful. Sure having to run multiple services, tracing and testing acrross them (basically the usual gripes about microservice architectures) dealing with inter-service calls, performance tradeoff etc.. is not as good as I'd like it to be but it could be improved. What sucks most is there a lot of raw edges we left exposed that trip new people and that I think was a mistake we should be prioritizing fixing those as they come up and think of them while buidling up our core framework and components more thoroughly.

raghava2• 6 years ago

Is there a method/framework to measure/manage architectural complexity?

Please share any pointers you might have!

nkassis3• 6 years ago

Observe how well and quickly new engineers come up to speed in your system. What problems do they hit? Think about them while designing. For example what hinders them from solving issues as they come up with customers.

Their experience in my view is a representation of the architectural complexity and cognitive load you are inflicting with your design.

techsin101• 6 years ago

Hey let's design looking backwards and ignore all future needs. Everyone will be happy. Because they all are about saving face and give minimum effort, enough to keep things working as is. It doesn't matter if it sacrifices future opportunities nobody can test that. Then it will be just a matter of fact. We will all just pretend this was unavoidable inherent complexity.

Imagine playing competitive RTS game and decisions were made with this attitude, tomorrow-not-my-problem as others don't have domain expertise to tell otherwise or they don't simply care to put themselves as opposing force and do your work to find better solution. In game you'd be doing seemingly ok and then get destroyed all of the sudden.

gshulegaard• 6 years ago

Sounds like League of Legends to me.

(Edit: at the casual level)

Jugurtha• 6 years ago

We haven't gone through the "microservices" craze for pretty scientific reasons: I dislike smug talking "Chief Wizards/Architects/Scientists" who've been writing posts and giving talks on architecture, clean code, and refactoring, based on that one project in the eighties. Reminds me of those phony martial arts "gurus" who are all about abstract ideas and "energy" channeling, complicating the simple to sell seminars and subscriptions.

Anytime something is discussed, we ask ourselves if that change will improve users' lives. I am sure these are useful for other people at different maturity levels of software, but it is not for us, right now.

This is why we don't use React, Angural, Webpack and a ton of other things and libraries that are useful for others but add no value to us and only increase. We avoid adding complexity unless it's really worth it, and worth it for the users, not to give us pleasure of using a new library. We have side projects for that.

stepbeek• 6 years ago

I've recently been using StimulusJS [0] and found it to be a really nice mid-point between spaghetti and a heavier framework. It has conventions out of the box, it does things in a reasonably disciplined way then it disappears.

[0] https://stimulusjs.org/

Hokusai• 6 years ago

I love the joke. But, it is a serious problem.

One key part is that naming is important. This is always a fight I am willing to take. Acronyms, fantasy names, names of teams or legacy names that does not represent the functionality of the micro-service plague many companies.

If you will not name a method "Jimmy" or a local variable "Galactus", why developers are naming their micro-services in such a bad way?

I do not know what I find more frustrating, the bad naming per se. Or the fact that the same developers would lose their shit if they saw this:

  class Galactus {
    boolean Jimmy;

    void turnOn() {
      Jimmy = true;
    }

    void turnOff() {
     Jimmy = false;
    }
  }

Sometimes it feels that software development is more a religion that an engineering discipline. That there is rights and wrongs based in tradition instead of sound practices. That is why worse than the naming is the lack of self-introspection that let us here.

cgarvis• 6 years ago

PG has an essay on keeping your identity small. It high lights the fact that when there is no way to prove one answer is correct, it’s then subjective which is the “right” answer. He uses the example of politics. Good read.

Yhippa• 6 years ago

One key part is that naming is important. This is always a fight I am willing to take. Acronyms, fantasy names, names of teams or legacy names that does not represent the functionality of the micro-service plague many companies.

This has been happening way before microservices became popular but it's gotten out of hand. Teams have nonsensical names as do their services. Outsiders coming in have a huge hill to climb to learn the domain because god knows why.

contravariant• 6 years ago

I reckon people might be copying the trend of giving nondescript names to particular software/services. Which isn't bad per se (I mean there's only so many ways you can rephrase 'webserver' before you run out of options), but if you split up your organisation and do the same thing to each subpart then it gets confusing.

hirsin• 6 years ago

We have something between 3 and 8 user profile services that you might interact with on a daily basis depending on where you sit. If they all had "good" names they'd all have roughly the same name. It's dumb for onboarding, but I have to say I see how "racoon" became a name.

monocasa• 6 years ago

So I'll throw out the argument that responsibilities of a service evolves as the product evolves. The fact that names like Galactus are blank slates can be a benefit so that you don't have to explain to some poor new grad:

"Well, you see, the identity service isn't actually the source of truth for identities anymore ever since the oauth service started taking that responsibility and the plan was to move all the endpoints into the other service but some other priorities came up, but that was five years ago..."

That is, if agility of architectural development is one of the goals of microservices, names that don't get out of date because they never meant anything except "name for this codebase" in the first place might have some benefit.

Does that overrule the benefit of a common lingo you can talk about without the domain knowledge of your system? Probably heavily depends on way many variables than even HN can hash out, but there's an argument to be made here for these naming conventions at least.

SarikayaKomzin• 6 years ago

One key part is that naming is important. This is always a fight I am willing to take. Acronyms, fantasy names, names of teams or legacy names that does not represent the functionality of the micro-service plague many companies.
If you will not name a method "Jimmy" or a local variable "Galactus", why developers are naming their micro-services in such a bad way?

There was a post on the front page today about the "principle of least astonishment." It could not be more applicable than this example.

For those that missed it => https://wiki.c2.com/?PrincipleOfLeastAstonishment

codr7• 6 years ago

Modular software is a great idea, always was; and fundamentalism doesn't make software better, never did. Good ideas are compromises.

The latest system I designed professionally was an ordering system that's divided into two parts; a local application with a database and an HTML document and a PHP script on a server.

The application itself doesn't need internet access to work.

The data needed for the online customer order module is uploaded to the server via FTP as needed. Online orders are stored on the server and polled via FTP on request from the local application.

The result is modular, very easy to maintain, and performance is good enough to be considered a feature.

But the reasons for the choices I made are all about the problem being solved, not organizing developers and keeping them from stepping on each others toes.

haolez• 6 years ago

Not exactly related to the video, but why do some people choose gRPC and others choose event sourcing for microservices communication? Is it just a matter of unneeded complexity in event sourcing? Are there performance issues as well?

jbreiding• 6 years ago

It's the difference between service orchestration and choreography, respectively.

There are pros and cons to each approach, and the details matter with regard to how services will be scaled and the coordination of that scaling happens.

There are lots of finer points in the details, but knowing the difference between the two approaches can help with the decision making process on which path the adopt.

socksy• 6 years ago

I can heavily recommend the book "Designing Data Intensive Applications" by Martin Kleppmann for a great overview and deep dive into the trade-offs in both of those approaches (and others)

raghava2• 6 years ago

As an engineer, I am finding i very difficult to decide on things like capability/modularity boundaries, layering constraints etc w.r.t architectural decisions for a B2B, cloud native, SaaS software solution. No matter what resource I refer to, there's a lot of conflicting info.

Is there an equivalent of "cyclomatic complexity"(McCabe count of code) for cloud/distributed systems/μServices architecture? Cant find any such objective measure, apart from a few dated research papers.

IMO, it could be useful for many of us. Sometime back, I had tweeted tagging @copyconstruct and @kelseyhightower asking this question, but haven't got a response - may be it went unnoticed.

Would love to get to know if there's any such method/framework to measure and manage architectural complexity, for the world in cloud.

2T1Qka0rEiPr• 6 years ago

I think if you don't know, don't. That's a good place to start :)

alixedi• 6 years ago

The only 2 valid reasons I have come across for using microservices based system architecture:

1) There is a codepath in the application that has radically different load characteristics.

2) We want to give each team in the organisation ownership of (most of) their stack, release cadence, SRE etc.

(2) seems like an attempt at engineering but is often about management - empowering the teams yada yada.

There is a thread here about: "Technology opinion-sphere is a FAANG monopoly" aka "Dude, we got like 3 paying customers!". I'll leave that for another day.

imtringued• 6 years ago

Actually 1) isn't a good argument. There is nothing wrong with having regular sized services. Most applications already do that. They separate databases from web servers.

Microservices [video]

Related Stories