Branchless development

http://www.tedunangst.com/flak/post/branchless-development

by luu • 11 years ago

77 59 11 years ago

tedunangst.com

joshbuddy• 11 years ago

How would this development style be compatible with code reviews?

jacques_chester4• 11 years ago

It's not, in my view, unless you count pair programming as a sufficient substitute.

I've seen branch-and-review used in anger. It can easily go bad: it drastically multiplies the inventory of work-in-progress. Everyone becomes blocked on someone else, moves on to another task, then loses context on the previous task or tasks. Stories with soft dependencies wind up sprinkled across several branches, making user testing a bit of a merge-and-pray event.

It sounds great. Reviews! But efficient!

And it turns out to be a tangle of interlocked gears.

I imagine somebody has made it work. I have not seen such a case.

My personal preference is a single development branch^, test-driven, pair programming, with rebase and integration test before pushing.

[^] If it's a private project, this is master. On github, the master branch can't be used as a true master, because it is a de facto release branch.

pvg• 11 years ago

Isn't that a little at odds with the idea that the author is describing some variant of the OpenBSD development process (unless I misread the thing) and beside being a large and ongoing project, they've been reviewing code since before it became fashionable?

4lejandrito2• 11 years ago

Sounds to me that we work for the same company :)

I bought into feature branches and all that stuff right after uni and tried to apply it at my first job without all the success I was expecting. Then when I moved to my current job in which we do TDD, Pair programming and single branch development I was a bit reluctant. After almost 2 years I haven't come up with a single scenario in which this approach has given us any troubles and in hindsight I think is what I should have proposed at my first job.

ben_bai2• 11 years ago

Big changes (mostly) need to be split into small and easy to review patches. There are almost no commits without at least one OK from another main developer.

glass-2• 11 years ago

Developer changes code and create a diff. This diff is sent to the mailing list or other developers to test and review. The patch only gets committed when other developers have ok'd it.

yxhuvud• 11 years ago

Why would it be a problem? Actually it is favourable, since the reviewers always see the actual changes to the code base in each commit, without having to deal with merge commits.

batou• 11 years ago

That's fine until you have a 4 MLoC behemoth with 50 developers (I use that term loosely) that isn't separated into logical concerns properly like your average "enterprise product".

I've seen those things and what you end up with is a dev team in tears because some chump has fluffed the source tree, the integration build is still running so no one else knows and checks out the trunk followed by boom and 2 days of paralysis while everyone works all their changes back into it for a checkpoint build.

All mature products tend to evolve into this state unfortunately.

Ergo, that works on small things but doesn't scale. Minimally, using feature branches and integrating them in turn is the only situation that works on those sorts of products.

Psyonic• 11 years ago

Facebook seems to be managing with 1k+ developers and 50+ MLoc (https://secure.phabricator.com/book/phabflavor/article/recom...).

I'm not entirely convinced either, but the argument isn't without success stories.

maximegarcia• 11 years ago

For long lived branches, I prefer to track and review the work done on master and merge master into the branch every often, I do every day. You'll handle the conflicts more easily, as they come and you'll get fresh insight/explanation from your coworkers. All at once at the end, and you end up in a nightmare of "Do you remember when you XXXX N weeks ago ?"

Diverge and merge at the end is quite utopist on complex projects.

techman92• 11 years ago

The thesis of this post is that merge conflicts basically negate all of the other benefits a branched workflow provides. I'm not sure how the workflow the author suggests deals with concurrent changes to code otherwise though, and I feel like merge conflicts are preferable to changes that overwrite other changes. Not taking advantage of branching destroys much of the benefit a DVCS provides.

In order for this workflow to be reasonable for any development team of a size greater than about 3, each commit would need to move all the code in to a working state. While a reasonable pattern in theory (Don't TRY commit breaking changes!), it is frequently useful to track code in an intermediate state. Sometimes (read frequently), this code may not be representative of the final state of the technology or may not even work as it should. Not maintaining individual branches means that these changes are exposed to the entire team and can be very very messy.

All in all, what a silly approach to revision control. Take advantage of the latest patterns and a system that provides the loosest coupling and highest distribution. If we do as the author advocates, we might as well all just switch back to CVS...

Negitivefrags• 11 years ago

We have a team of 50 and single branch works well for us.

merge conflicts are preferable to changes that overwrite other changes

Nobody is saying that. The difference is really merging as you go vs doing one big merge at the end. I personally fall on the side that it's far easier to merge as you go.

techman92• 11 years ago

You merge as you go in branched development too?

Master is typically periodically pulled into the feature branch to ensure you're working against the most current version of the codebase.

EDIT: Yes, this can create an ugly revision history, but that's what rebasing is for, right?

ben_bai2• 11 years ago

Branching/merging in DVCS is only easy if there are no merge conflicts.

Checking in code to the VCS that makes the tree NOT build will get you screamed at (at least within the OpenBSD project). This requires proper planning for big changes, and to break them down into small, easy to review, patches.

OpenBSD uses CVS as main repo because it fits their development style. Bevore you knock down CVS look at AnonCVS and cvsync. http://www.openbsd.org/papers/asiabsdcon2009-release_enginee...

Taniwha• 11 years ago

I'm also a big fan - for a tight team anyway, might be harder for a more distributed project - you should eat your own dog food - more incentive to not check in crap and an incentive to fix broken stuff fast.

Having an autobuild system to build every checkin is a big plus.

I worked on a chip once where we decided to make everything a branch, at one horrible point we didn't have a mainline, we didn't know what we were building, we had to put people on full time merging trying to figure out what to test.

riquito• 11 years ago

We make tests to solve subtle merge bugs like the article's example about dialogs, and watching what the other team members are merging help miles too.

radicalbyte3• 11 years ago

..and the circle is complete.

If you're a fan of this workflow then I can recommend using Visual Source Safe. It even has exclusive locks which completely eliminate merge conflicts.

I've been building software professionally for almost 20 years, I've seen (and experimented) with almost every VCS/SCM strategy imaginable.

My conclusions:

This only really works for tiny teams - 2-3 people who work very well together. It needs high bandwidth communication.

For teams of 3-10 short lived feature branches are effective with a little bit of planning to minimize merge conflicts.

For larger teams you really have to split the system into self-contained modules or microservices, each module having clear owner. These modules are then shared as source, library or service depending on the actual needs.

I've never had good experiences with longly lived branches. They can work whilst adding experimental modules which require only a couple of tiny API changes to the main system; but then splitting the module into a separate project/repository and making the API changes (behind an #IFDEF) makes more sense.

LoSboccacc• 11 years ago

same here. the only long lived branch we have is the deployed code branch, which I found more convenient than just a tag

it also matter average team skill. if you have to make do with what you find/can afford, then feature branches are good at isolating devs from each other non working commits.

koonsolo• 11 years ago

Sorry, but your experience of Visual Source Safe is totally different than mine. I started working at a small company in 2002, where they used Visual Source Safe. Next to the VSS database file getting corrupted every few weeks, the 3 developers were always asking each other to unlock a certain file because someone needed to make a change in the same source file (but completely different method/location).

I introduced Subversion. No more corrupt files (shiver), it just kept on running. And you could actually work on any file. If you divide your work properly, you can work in the same file without working in the same code. And if the small chance of changing the same line occurs, the merge wasn't that complex to fix.

We never looked back, and therefore I am completely disgusted that someone actually suggests using such an unstable, non-efficient tool.

logiclabs• 11 years ago

I suspect there might have been a touch of sarcasm in that VSS remark

pavel_lishin• 11 years ago

It even has exclusive locks which completely eliminate merge conflicts.

I assume those locks can be overwritten by an admin, right? I'm exactly the sort of forgetful jerk who'll lock an important config file before going on a two-week vacation.

zeendo6• 11 years ago

In my experience, if you have high bandwidth communication (e.g. a team room) then trunk based development scales to well beyond 2-3 people. I've seen it work on at least one large distributed teams (3 team rooms, 3 time zones, ~30 developers)..and it regularly works on teams of 6-10 developers.

Regarding VSS and exclusive locks. No. Trunk based development with no exclusive locks is fine. Pessimistic locking was unwarranted pessimism when applied to source control.

Also, note that Google is trunk based...with a single trunk across the whole company. I think Facebook is too?

radicalbyte3• 11 years ago

In that case you were absolutely having merge conflicts - only you were forced to resolve them the second you updated your local working copy.

Over the years I've lost so much time to small errors resolving merges - particularly the type which involved cross-file refactoring (where two people have moved methods and then refactored them).

The beauty of using branches - especially with Git - is that you as developer gain a lot of power over how you apply merges. The "pre" state is also under version control, so you can always go back. Plus if you're using Git then you're implicitly doing branch-based development (git clone creates a local branch).

My current team of 6 developers switched from TFS trunk-based to Git with short-lived feature branches and they've not looked back. We've greatly reduced merge issues and downtime caused by bad check-ins. Both still happen, only less than before.

ExpiredLink• 11 years ago

Also, note that Google is trunk based...with a single trunk across the whole company. I think Facebook is too?

Yep!

http://paulhammant.com/2013/04/05/what-is-trunk-based-develo...

OldSchoolPro4• 11 years ago

And still somehow feature switching works great at Facebook, Google and Goldman Sachs with big teams.

Merging complex changesets across branches is impossible to do 100% correct every time. Not unless you're a 10 feat tall code crunching robot.

jules• 11 years ago

Branching or not is somewhat of a red herring. If you are working on a commit then while you haven't pushed that commit, you're effectively working on a branch. If somebody else works on the same code at the same time, then those changes need to be merged. Whether your VCS does or doesn't call that a branch is immaterial. What matters is that two developers can work on the same code at the same time, and that you need to solve concurrent editing conflicts (logical ones, not just the textual ones that your VCS will find).

The pertinent question is how often you merge. On one end of the scale you merge on every keystroke. Every time Joe types a character it gets put into Anna's files. Clearly this doesn't work. On the other end of the scale every developer works on their own copy, and all the work gets merged just before a release. Clearly this doesn't work either. There is a scale from maximal disruption minimal effort merging to minimal disruption maximal effort merging. You want to find a point on that scale that minimizes the total amount of work.

aidenn04• 11 years ago

This is super-important; it is far better for a developer to push a non-working feature on a feature branch than for them to leave it on their machine. They ought to do whatever the equivalent is of a git rebase every morning or evening, so that it is still at least nightly integrated for them.

zeendo6• 11 years ago

Keep in mind though that if everyone is working on something that takes multiple days to complete (and your definition of non-working is incomplete) then all the work is happening in feature branches and master is quiet. Those rebases are easy because no work is going on there.

Feature toggles remove this problem by keeping everyone in trunk/master. You can then have CI builds with whatever set of toggles you want.

glass-2• 11 years ago

People thinking and saying that this only works for small teams should take a look at how many devs are listed in OpenBSD's latest release announcement.

Hint: it's more than 2, more than 3, and more than 50.

icebraining• 11 years ago

How many have direct commit access? If you're sending patches over email, guess what: you're doing merges, whether you call them so or not.

zeendo6• 11 years ago

Or, say, Google and Facebook?

http://paulhammant.com/2014/01/08/googles-vs-facebooks-trunk...

Confusion• 11 years ago

I'll fully accept that it's Teds experience that this works for OpenBSD. I'd be hesitant to draw any conclusions from that concerning the viability of the same process for any other project or organisation.

An open source project with highly motivated and capable developers and a benevolent dictator in charge functions differently from other projects and organisations. A process needs to be attuned to the people, and the relations between those involved, involved.

haches• 11 years ago

People in academia have proposed a number of tools & approaches on how to enable concurrent development while mitigating the "risks" of branching.

To name just a few:

* Palantir by A. Sarma et al. [1]

* Crystal by Y. Brun [2]

* CloudStudio (project I worked on); Video demo: https://youtu.be/R3Fz0Tcdz0Y

To the best of my knowledge, however, none of these projects have found any widespread adaption. A lot of it also has to do - I think - with culture and personal preference.

[1] https://scholar.google.de/citations?user=shMjCasAAAAJ

[2] http://homes.cs.washington.edu/~mernst/pubs/vc-conflicts-fse...

joesmo2• 11 years ago

So if I'm understanding correctly, this terribly written article (he tries to make a point about merging but doesn't actually make it amongst other points he fails to even state) suggests using feature branches inside the code instead of feature branches within source control. It also requires mini code reviews and merging at every single commit. You cannot get rid of merging. Instead of doing it once when merging a branch, however, you're now doing it every commit. Since you're merging smaller chunks, it's impossible to see the bigger picture of the whole feature as you merge/review, and it removes the ability to do meaningful code reviews on features before they actually get into the main code base. Not only that, sometimes you end up in a race with your teammates, knowing that you're working on the same piece of code and that whomever checks it in last will have to merge. Stupid, but it can happen.

To me, the ability to have logical separation of the features being built and do meaningful code reviews far outweighs any proposed speed benefits of not branching, benefits that I assume show up only in larger groups as they certainly don't show up in small groups. Which bring me to my last point, the "slow is fast" section, which doesn't provide any evidence that slow is fast. Other than users not having to decide which branch to run (which takes all of two seconds, BTW), development is much faster with feature branches and the other negatives do not show up in my experience. What I'm trying to say is that "fast is fast." Slow is not fast and the author doesn't make his point there either.

tl;dr: All in all, he points to no actual pros to branch-less development.

OldSchoolPro4• 11 years ago

Heh, there's always opposition to new practices. Branching has been mainstream for a long time, it's only fair to expect resistance to any alternatives.

Give it a few years and everybody will use feature switching, except the old man enterprise dev :)

There are a million things that feature switching let's you do. Enable and disable instantly, enable for .00001% of the users, integration test different combinations, etc.

https://secure.phabricator.com/book/phabflavor/article/recom...

joesmo2• 11 years ago

In my experience, branching has only been mainstream since git became popular a couple of years ago. Before that, when using svn and other similar systems, branching per feature was unlikely in my experience and a major pain.

amelius• 11 years ago

It would be nice to have some kind of monitoring system that warns me when another user is editing (or has edited) the same file(s) as me. Then we could talk, or one of us could wait, to prevent merge conflicts in the first place.

Of course, this would be even more useful if it only alarms the users whenever they have created a merge conflict.

leighmcculloch• 11 years ago

Branchless development is a terrible idea. I think what the author is wanting to avoid is long lived branches. Keep your branches small, isolated, get your changes into the main branch as soon as possible, but develop each atomic change independently of it. We do this at Gumroad, keeping our branches as small as possible, and it's amazing.

solomatov• 11 years ago

I work without branches but with a large number of good tests, and it works like a charm. On the other hand, I know people who can spend several days to merge their one week changes back to the master which is a great waste of time and really boring.

Branchless development

Related Stories