The Architecture of Git (2012)

by wheresvic1 • 8 years ago

355 78 8 years ago

devy• 8 years ago

Very informative. This explains well about why Linus once said signing every single Git commit is unnecessary and it only means you have automated the signing process. [1]

[1] https://news.ycombinator.com/item?id=12290873

ArchTypical• 8 years ago

To understand Git's design philosophy better it is helpful to understand the circumstances in which the Git project was started in the Linux Kernel Community.

I don't understand this sentiment. It's not helpful to know the history at all. At best, it romanticize the choices made. Stating the goals would be an intro that shows some level of analysis.

gmueckl3• 8 years ago

No, the author is right. Git us in its core a database for managing patches. Understanding the needs of Linus Torvalds as his role of Kernel maintainer is about the only good way to understand why git is so strangely designed.

mbfg2• 8 years ago

curious why you say it's strangely designed. seems pretty rational to me.

avar2• 8 years ago

Darcs is a database for managing patches. Nothing in Git inherently cares about patches. To a first approximation it's a database for managing full snapshots of trees of files.

mycall• 8 years ago

I am fine with having extra sentences to read.

dahart• 8 years ago

The goals are stated, just a few lines down in the same section you quoted. What makes you think design goals and philosophy aren't formed by history? How does knowing the history of a project romanticize it? Personally, I'd assume exactly the opposite, that knowing history is the only way to understand the goals and philosophy, and that romaticizing only happens when a history is not understood and/or the story is changed or told through a tinted lens.

alkonaut• 8 years ago

If it had been started in e.g the game development scene there would have been some other design decisions made. The idea that the source tree is rather small and the history is short and there are few binary assets managed are definitely showing in the design.

jayd16• 8 years ago

By understanding the real world motivation you can reason about both the stated and unstated goals of the project.

frou_dh• 8 years ago

Git is the VCS most suitable for bottom-up learners. Which is the VCS most suitable for top-down learners?

pritambaral• 8 years ago

Dropbox, perhaps.

MordodeMaru• 8 years ago

Plastic SCM.

blattimwind• 8 years ago

Copy/paste.

reificator4• 8 years ago

C:/Users/Billy/Desktop/Backup/New Folder (39)/IMPORTANT/My_Project (OLD)/My_Project.sln.bak

crispyambulance2• 8 years ago

    > Which is the VCS most suitable for ....

Git.

It doesn't matter what your question is. :-)

I mean, come on, it's not like you have a choice. You gotta use whatever your teammates/coworkers/organization is using.

That said, yeah, there's a lot of pedagogic problems with git stemming from the INSANE inconsistency of command-line. The only redeeming quality? It works and it's popular.

reificator4• 8 years ago

It works and it's popular and the underlying model is easy to understand.

I think step one to replacing git will be to launch a tool that completely replaces the git CLI but uses the same model below. I've not looked into EasyGit much, but I don't think it goes far enough.

qznc2• 8 years ago

People once said that about Subversion.

I believe Mercurial has a chance if they get support for large binary files right before git does.

pfranz5• 8 years ago

I really hope Git isn't the end of the road for VCS. I work in visual effects and video games. Most video game studios still begrudgingly use Perforce for project data. Many also have a separate Git server for code. Perforce or SVN is still the go to solution for binary assets. Trying to explain Git to programmers is difficult enough, for artists who have never used the command line it is unreasonable. Every time I've seen an "asset library" it's usually written from scratch instead of built on another VCS. I've made some crude attempts at building on Git and the data just structure isn't appropriate.

I know it's a bit unreasonable to expect the same tool for everything (but that's what you were implying). The binary problem is more manageable for code with a few UI assets (like icons), but isn't great.

WorldMaker• 8 years ago

Darcs, probably? Darcs has one of the easiest UIs in the DVCS game and was always very easy to teach people how to use.

But then of course if you want to learn the details of how darcs works under the hood you wind up in Haskell, and feeling a need to read a lot of academia papers.

ThrowawayR2• 8 years ago

I'm a bit disappointed by the flippant replies. Mercurial is a very solid alternative VCS (Facebook uses it internally) and is friendlier (or at least has less gotchas) than git.

reificator4• 8 years ago

As a flippant replier, it's partly motivated by not really thinking other VCSes have a chance.

As confusing to use as Git's UI may be, I think the only way we're getting away from it is if someone massive pushes an alternative.

Alexa.com has Github trending down six places to the 66th most visited page on the internet. Worldwide. That's not just programmers that's everyone.

Too• 8 years ago

Mercurial is the obvious first alternative. It supports many of the same concepts as git but with a lower barrier to entry. A lot of this has probably been said many times before but I feel it's worth reiterating.

1. No staging area. This makes commit+push a 2 phase operation rather than 3 phase and removes a lot of confusing concepts, state and commands. Selectively commiting files and chunks of files is still possible but it is all done in the commit operation. (It doesn't track the history of an edit as git does where you can skip the last edit in one file but who really uses this feature? If you are such advanced user you might as well use stash or amend your last commit)

2. Branches are not just pointers to the tip of a graph. This makes it easier to understand what was master and what was dev after a merge and master can't suddenly point elsewhere.

3. All command line commands are sane by default as opposed to git where you almost always need at least one flag to get correct behaviour. And there is usually only one way to do things right.

4. Incremental version numbers (locally only) makes it look more friendly than commit hashes.

5. Much better GUI tools available. Dont know why since the market is smaller but probably because of a less complicated internal model it's easier to make a GUI representing it.

Even with all those i still don't know if it is easy enough for non developers, or if any scm with a commit/push/pull model ever can be. The whole concept of merge conflicts is very complicated and even seasoned developers need to go get an extra cup of coffee when a conflict pops up in the middle of what you expected to be a trivial rebase.

The only way to avoid merge conflicts fully would be instant edits such as Google docs but I don't think that would work very good for code where different people would break the build constantly.

umanwizard• 8 years ago

What are top-down and bottom-up learners?

nine_k• 8 years ago

The best (known to me) informal intro into the architecture of Git is The Git Parable: http://tom.preston-werner.com/2009/05/19/the-git-parable.htm...

svat• 8 years ago

Also greatly worth reading: “Git from the Bottom Up”:

* HTML version: https://jwiegley.github.io/git-from-the-bottom-up/

* PDF version: http://ftp.newartisans.com/pub/git.from.bottom.up.pdf (via http://newartisans.com/2008/04/git-from-the-bottom-up/)

The great thing is that after reading and understanding these, one's mental model matches the reality of the Git program, so one can both try bolder things, and get unstuck from any mess.

freshhawk• 8 years ago

I recommend this constantly, it is excellent.

Not to outright beginners, but anyone past that point should read this. It is a great job of clearly presenting an accurate mental model that helps you use git. If you are already a git expert this is just a good thing to learn from in general and with how to explain and teach git usage in particular.

The only documentation I know of that can turn people from cargo cult git users to people who just do the version control things they need done with the parts of git they need. That is damn useful.

malingo• 8 years ago

I've always liked the low-level explanation at https://git-scm.com/book/en/v2/Git-Internals-Git-Objects

It's a great exercise for the reader to recreate the core functionality of making git commits with just a few command line utilities (echo, wc, xxd, shasum).

alexchamberlain• 8 years ago

IIRC,that's how they were implemented in the first place.

dabei• 8 years ago

The whole website is a treasure trove for learning about system design.

xrd2• 8 years ago

I still prefer my book's history of Git in which I compare Linus rejection of Monotone to the movie "Back to the Future":

https://github.com/xrd/BuildingToolsWithGithubBook/blob/ca7f...

agbell• 8 years ago

Interesting, what am I reading?

xrd2• 8 years ago

This (a book about building tools with Git and the GitHub API) was published by O'Reilly in 2016, and we just released it under creative commons. You can read it completely free at https://buildingtoolswithgithub.teddyhyde.io or get the repo of the book contents above.

adamkl• 8 years ago

I've found this online course by Paolo Perrotta to be the best introduction to Git's architecture, by far: https://www.pluralsight.com/courses/how-git-works

He takes you pretty much from first principles all the way up to how remote repositories are tracked. Its been far more useful to me than simply trying to learn the CLI.

I realize its behind a paywall, but I'd recommend signing up for the free trial just to watch this course (if you're curious about the inner workings of Git).

royalghost• 8 years ago

I still recommend people to watch this video from the man himself, Linus Torvalds, if they really want to understand the architecture and data structure behind git - https://www.youtube.com/watch?v=4XpnKHJAok8

If I remember correctly, he mentioned that he wrote the MVP in 4 weeks and the data structures he used are quite simple. Never got a chance to look at the source code but I guess they are in github (at least the mirror copy.)

nickthemagicman• 8 years ago

It only took 4 weeks plus 30 years of programming experience and a genius mind.

sfescape5• 8 years ago

I'm surprised that it's not mentioned in the article that one of the most interesting architectural aspects of git is that it's a blockchain system.

beagle3• 8 years ago

It was written before “blockchain” was a well known term.

sfescape5• 8 years ago

git predates the creation of bitcoin and 13 years after the invention of blockchain.

erwan• 8 years ago

That's arguably not the most interesting architectural aspect of Git.

Or does any back-linked tree data-structure becomes interesting if the nodes keep a hash of their parent instead of a raw reference? I don't think that's the case.

It might be a bit heretical but I don't think Git has a super interesting internal architecture. I'm not downplaying the fact that Git was very innovative, especially considered the landscape of SVMs at the time. The tool as a whole is great and has desirable properties but its internals don't strike me as particularly innovative. It's a clever composition of solutions to well established problem domains. And in that aspect it is a beautiful engineering solution although there is room for a lot of improvement in terms of UX.

And in addition to that, I would argue that it would be a very weak definition of "blockchain". The innovation in Bitcoin is the incorporation of proof-of-work and resulting alignment of incentives such that it can achieve probabilistic consensus in an adverse setting and with some degree of asynchronicity.

The underlying structure of the data is an obvious choice because it is simple and "captures" the idea of aggregate global state, but it's also hardly an important innovation. UTXOs are more significant.

And also, recall that the textbook example for state-machine replication is always an append-only log for example. So that's not the crux of it, or of blockchain, in my humble opinion.

sfescape5• 8 years ago

I said one of the most. I didn't declare it the most...

ysleepy2• 8 years ago

It's a merkle tree, not every hash tree or chain is a blockchain thing.

IMO blockchain describes the combination of a hash chain and the no-trust agreement protocol on top.

sfescape5• 8 years ago

I don't agree with that. The oldest blockchain known has its hash codes published each week in the New York Times. I would agree that having the ability to prove the history hasn't been modified by externally retaining the hashes could be a component, but git qualifies for that as well.

rbonvall• 8 years ago

This is from 2012, so it makes more sense to be surprised that it doesn't mention git being a crowdfunded, gamified, phablet-compatible cybertool as-a-service on the cloud.

50• 8 years ago

Not sure if it’s of any relevance but I recently discovered https://gitup.co and must say, it’s pretty cool.

dilap• 8 years ago

Frickin' amazing UI design, and some great pieces of functionality (edit the commit graph directly; ability to undo most operations).

Unfortunately development seems mostly paused, and it still has some big gaps...

Still tho by far my favorite git client. (#2 would be Sublime Merge.)

findjashua• 8 years ago

Fork (git-fork.com) is pretty neat as well

dmoreno• 8 years ago

I loved the AOSA books, but there's been sometime that they don't publish a new one.

Do anybody know if there is something new in the works?

And of any other similar books?

emmanueloga_2• 8 years ago

Have you already read all of them? I was calculating how long it would take me to read and understand every chapter of every book. I usually like to write small snippets of code or look for references and stuff for books like these.

Anyway, if I read, say, one chapter every Monday, it would take me about two years to complete the books!

Probably not a problem that there's "nothing new" since the books seem to be more about timeless design principles and less about novelty.

emmanueloga_2• 8 years ago

Ok, time to contradict myself. Other two books in my back burner in the same ballpark:

* Beautiful Code

* Programming Pearls

macspoofing• 8 years ago

"The Architecture of Open Source Applications" is great. I highly recommend their other articles, as they are all interesting and informative. If you can, throw a few bucks their way by buying the PDF or paperback versions.

chmaynard2• 8 years ago

The fourth paragraph in section 6.2 refers to "BitMover" with no previous context. Copy-paste error?

sitzkrieg2• 8 years ago

there is context in the intro (see bitkeeper)

chmaynard2• 8 years ago

I found nothing about BitMover here:

http://aosabook.org/en/intro2.html

vram22• 8 years ago

I had come across this gem about understanding the vi editing model a while ago:

"Your problem with Vim is that you don't grok vi."

It's the top answer on this StackOverflow question:

What is your most productive shortcut with Vim?

https://stackoverflow.com/questions/1218390/what-is-your-mos...

Wonder if there is any such post about Git that cuts to the chase, even for a part [1] of the Git model, and explains it clearly [2].

[1] I had also come across a StackOverflow post that explains a part of git very clearly, like the vi example I quoted. I think it was about how to roll back accidental changes using "git reset --hard" and variants. Saved it, but don't have it handy right now.

[2] Note: I said "clearly", not necessarily "simply". I like to quote the (probably out-of-print) book by Abbe Dimnet [3] called The Art of Thinking, in which he said something like this (while deploring the trend of books that try to make things artificially simple, a.k.a. dumbed down):

"French grammar cannot made simple. It can be made clear."

[3] https://en.wikipedia.org/wiki/Ernest_Dimnet

He also wrote a book on that same topic (French grammar made clear). Googled the former book recently and saw about the second book. I've read the first, long ago, which I found in a second-hand bookshop. Good book. Apparently it was a best-seller at the time it came out, according to Wikipedia.

Quotes by him:

https://en.wikiquote.org/wiki/Ernest_Dimnet

The Architecture of Git (2012)

Related Stories