Monorepos are changing how teams build software

https://vercel.com/blog/monorepos-are-changing-how-teams-build-software

by gk1 • 4 years ago

8 19 4 years ago

zaphar3• 4 years ago

I love the concept of monorepos. With the proper tooling and support I think they can be force multipliers for high functioning engineering organizations. However almost none of the publicly available tooling for CI/CD, IDEs, Source code hosting, or any of the other force multipliers out there support it properly, so for most organizations I think it's a wash.

It's a shame really since if they did support monorepos better or even just stop assuming too much about the repository then a whole lot of things could be unlocked for engineering teams. As it stands though going the monorepo route means you'll be building a lot of bespoke tooling to support it.

MuffinFlavored• 4 years ago

With the proper tooling and support

Could you elaborate/provide some examples?

Jensson• 4 years ago

Google is the main example, their monorepo setup works and scales really well. It is basically a remote filesystem directory that automatically checks out the files you need to build your project and not other files.

zaphar3• 4 years ago

In a monorepo you usually only want to build portions of the codebase at any given time in your CI/CD. Most of the CI/CD tooling out there are accidentally hostile to this.

As just one example it is often difficult to impossible to rebuild your code when a dependency changes. Build systems often don't expose this information usefully, so you'll find yourself either hand rolling something that looks at git changes and tries to infer what should be rebuilt and what shouldn't. Or you just rebuild everything in the entire repo every night. Or you can adopt Bazel everywhere which can be a hard sell in many companies but does give you better tools.

Another example: Most source code hosting products don't give good support for path based permissions which you need in order to ensure that folks with the right context review code before it is merged in. Which pushes you to segregate your repositories along ownership and responsibility lines.

Mikeb85• 4 years ago

Are monorepos cool again?

Dunno, all my software development experience is in monorepos (Rails, Gamedev). Or simply one-off scripts...

pythonbase• 4 years ago

From single PHP file to cover both back-end & front-end to multiple layers & repos to monorepo. Another full circle in SDLC.

JustLurking20222• 4 years ago

Having worked in companies with large monorepos and those without, I would never advise implementing one.

The complexity as a function of repo size grows at something greater than linear rate. Code becomes tightly intertwined and the dependency graph becomes unwieldy, making refactoring a difficult proposition.

Eventually, the codebase always grows to a size that makes it impractical to replicate on each developers' machine, so you start having to implement workarounds, and you realize that just keeping basic build operations working requires a substantial team to support. For the big players, they can eat the cost no problem, but it's still not money well spent and certainly not advisable for orgs without that level of resourcing.

tmerr2• 4 years ago

Code becomes tightly intertwined and the dependency graph becomes unwieldy

One way to combat this is to use a build system with a notion of visibility, https://bazel.build/concepts/visibility

I think you might be over-extrapolating from your experience, but it's true going off the well-trodden road leads to surprises. And not that many smaller companies know the ins and outs of monorepos.

JustLurking20222• 4 years ago

I have... as mentioned, worked in one of the large tech companies with a monorepo, and they all use fairly similar tooling.

In practice, it doesn't solve the problem because most people default to making everything public. Nominally this avoids duplicating logic, but may result in more stuff being pulled in than was really necessary. These days, when NodeJS developers discuss how bad things have gotten with dependency management, I just laugh.

tored2• 4 years ago

How would you handle versioning in a monorepo? Just a new directory for major versions?

If we have multiple teams you can’t really refactor other teams code and sometimes you need to do breaking changes thus I imagine that some versioning must exist.

smitty1e• 4 years ago

Ours is essentially a dumping ground, a disorganized network share. Oh, how great it would be to make the problem smaller.

ohCh6zos2• 4 years ago

I've worked at companies with monorepos and it pushed me pretty hard in the opposite direction. My philosophy has been to go through the packaging and dependency process early and often, and split off as many things into their open repo as possible.

tored2• 4 years ago

Any specific reason you came to that conclusion when working with monorepos?

ohCh6zos2• 4 years ago

I'm a bit scatterbrained on this topic, maybe I should sit down and try to organize my thoughts. Every company I've been at that has used monorepos has fallen into a hell of dependencies where changes break things unpredictably. The other big issue I've seen is companies with monorepos ended up not trusting their engineers. Everyone being able to see everything results in lowest common denominator code, and style fights that would never matter without one big repo.

Monorepos are changing how teams build software

Related Stories