Distributed big balls of mud (2014)

http://www.codingthearchitecture.com/2014/07/06/distributed_big_balls_of_mud.html

by joeyespo • 11 years ago

48 22 11 years ago

The "headline" of this submission is actually the final phrase of a comment on a year-old blog post, way down at the bottom.

dang• 11 years ago

Ok, we changed the title from '“We're leaving the idea of independently deployable services and not looking back”' and added a 2014. It's not entirely off limits to link to an interesting comment rather than the root article, but in this case perhaps it doesn't add enough over the article itself.

There's certainly nothing wrong with an article being a year old if it's good.

threeseed• 11 years ago

a) If the time and complexity to deploy 3,10,100 microservices as opposed to 1 monolith is significant then your build pipeline is flawed. Simple as that.

b) Maybe the commenter has only worked on small applications but I've worked on enough million line code monstrosities to know that one change can have far reaching consequences. It is simply too easy to couple components together e.g. be reusing a shared utility or service class and your code coverage will always be lacking in some way.

c) Microservices enable far more resiliency and scalability simply because you can deploy more of the ones that are critical or non performant. A sensible service discovery strategy can make this a one button, trivial exercise. Doing so on a complex monolith is often far from trivial.

d) This idea that your entire system goes down if a microservice goes down indicates your architecture is flawed. You should be able to still operate in a degraded state or at least provide some functionality.

e) If you can't test your service independently then your testing strategy is flawed. You should be able to trivially mock/stub out any dependencies and if you can't then you need to rethink your API design and the nature of your coupling.

zobzu• 11 years ago

The reality of things is, though, that at many (most?!) companies the pipeline IS indeed flawed BUT must work.

In other words for various reasons it's been built flawed. Speed, idiocy, politics, whatever. At some point, you have various choices to pay your technical debt:

- actually pay it(50%+- work force full time on this, forever - hey, it works long term, but only short term gains means bonus pay check, and making things square isn't as fun as building new stuff)

- patch shit up so it kinda works and hope to leave before it's too horrible (what most do)

- dissolve the company/product (eventually happens, almost inevitable over long enough periods of time - specially when people got tired of the shit and rebuilt everything anew - then the cycle starts again)

This blog post uses the 2nd solution (the most common).

coldtea• 11 years ago

a) If the time and complexity to deploy 3,10,100 microservices as opposed to 1 monolith is significant then your build pipeline is flawed. Simple as that.

Hardly.

The actual issue is that, since the 3,10,100 microservices are all needed for the core offering, then having 3,10,100 microservices is not any different than 1 well designed monolith.

With the exception that you've replaced method calls with REST and what have you, and have all the BS that comes with a distributed system, for few of the benefits.

dave_ops• 11 years ago

This is one of the things I love about the Erlang ecosystem (Erlang, Elixir, Lisp-flavored Erlang).

Since the whole thing was designed to be an OS unto itself, it has the concept of "Applications" and "Releases" that come from OTP.

It's trivially easy to make things properly decoupled by dividing logical functionality into "Applications" and making a game-time call about whether or not they should be bundled in the release and consumed sorta like a library or run on a completely different VM/node and consumed like a service. You're architecturally incentivized to make "microservices", but fully able to couple or decouple them at will while maintaining the same semantics in how you consume them.

crdoconnor4• 11 years ago

Microservices enable far more resiliency and scalability simply because you can deploy more of the ones that are critical or non performant.

You can horizontally scale and set up failovers for non-microservices too.

The main difference is that you need fewer load balancers, your architecture is simpler and you have fewer points of failure.

ryanelfman• 11 years ago

Sometimes I feel like microservices is pushed by hosting providers to get more money from all the additional deployments...

inthewoods• 11 years ago

I'm not a computer programmer or scientist, but I work with a group of them. The argument I've heard them say - and I may well be misrepresenting it here - is that microservices are often used as a way to avoid getting better at parallel processing/programming. They're working on a huge amount of data processing using Go if that provides any context. I'd be curious what others think of this idea.

felixgallo• 11 years ago

I think it's likely that the recent faddishness of Go, which upon actual use turns out to be difficult to use for anything /except/ microservices, itself is causing microservice adoption.

lukeschlather• 11 years ago

Microservices do allow you to an extent to avoid multi-threaded applications, which is considered the most "pure" way to do parallel processing. However, even though it might be considered "impure" I think microservices are a really effective way to manage parallel computation. Especially when you're using languages which have leaky threads that end up accidentally sharing state you don't intend to share.

hyperpallium• 11 years ago

Yes. partly it's easier parallelism. Generally, easier ways of doing things are successful.

And partly. it's more efficient use of resources because finer-grained, therefore cheaper.

TickleSteve• 11 years ago

Microservices enforce the removal of shared state by virtue of the subsystems being physically separate. This is why they're enabling concurrency.

The same effect can be had without actually creating microservices. All that is needed is well defined and controlled interfaces.

rtpg• 11 years ago

but the design thinking and decomposition strategy required to create a good microservices architecture are the same as those needed to create a well structured monolith

I don't think that this quite captures the utility of something like microservices.

For example, if you split your frontend and backend into separate projects, then the barrier for entanglement becomes so much higher. Instead of just importing some frontend class for the backend, you're loading in stuff from a totally different project and making 2 coordinated patches for them. Doesn't that feel wrong somehow?

The appeal of microservices (much like functional programming) comes from imposing restrictions that make dangerous things harder. Reducing the possibility of breaking your own rules on clean programming.

(Disclaimer: I don't think microservices are worth it for smaller projects though)

dietrichepp• 11 years ago

The appeal of microservices (much like functional programming) comes from imposing restrictions that make dangerous things harder.

But, don't microservices make safe things harder too? What's the point in simultaneously making both safe and dangerous things harder?

crdoconnor4• 11 years ago

The appeal of microservices (much like functional programming) comes from imposing restrictions that make dangerous things harder.

It makes everything harder for no particularly good reason. Dangerous or otherwise.

Worse, if you do happen to create a distributed ball of mud you've almost certainly created a far bigger maintenance nightmare for yourself than if you created a "monolithic" ball of mud.

aryehof• 11 years ago

To me, microservices address for many the continuing need for structures that hide complexity behind small, understandable interfaces. Black boxes if you will. Unlike components, which have no standard for connecting to and among each other, microservices do - in a relatively standard and easy to understand way. That's why they are so attractive.

I find myself very much agreeing with the author, but being involved in modeling complex business problems into object-oriented code, I'm more frustrated that in a problem domain, we have no effective modular concept for groups of collaborating objects. We can't walk up to an object graph and immediately understand the functionality it performs and the services it provides.

crdoconnor4• 11 years ago

To me, microservices address for many the continuing need for structures that hide complexity behind small, understandable interfaces.

You can have loosely coupled software with or without micro-services and you can have tightly coupled (ball of mud) software with or without micro-services.

All microservices do is multiply the problems caused by creating that ball of mud.

If you want to hide complexity behind small, understandable interfaces that's laudable goal, but it is not one that distributing your application across a network is going to help you with.

DropkickM16• 11 years ago

I think your first point is obvious, but I disagree with your second, at least 'at-scale'.

In the case where you have loose coupling but are representing multiple entities that scale in different ways, microservices allow you to separate concerns and separately scale those concerns relative to their requirements in terms of memory/CPU/disk/network/etc. The best factored code running in a single horizontally-scaled layer will be inefficient if 90% of requests are manipulating entity A, and entities B, C, and D have a lot of intricate business logic but are rarely touched (they are better off if separated and scaled individually)

The overhead you allude to is definitely something to take into account. If you're a 5-20 person startup without a serious need to scale up or lacking people who have built the tools that make microservices easy, you should avoid the issue for now. But ultimately, decoupling services so they can horizontally scale independently is a huge win.

Distributed big balls of mud (2014)

Related Stories