SQLite Doesn't Use Git

https://matt-rickard.com/sqlite-doesnt-use-git

by bkq • 4 years ago

181 171 4 years ago

matt-rickard.com

72deluxe2• 4 years ago

I use Fossil at home since it's just me. The single file approach takes up less space than the comparable git tree.

Fossil has the ability to push to git, which I do periodically just for curiosity.

teknolog• 4 years ago

Your dev environment is so resource constrained that the size of the repo is a factor?

bayindirh8• 4 years ago

Some of us likes to optimize for footprint, and that's OK.

I run an OrangePi Zero with 512MB of RAM as a home server since it both works and is a fun experience.

samatman2• 4 years ago

There are a lot of things which make Fossil attractive for solo and small developers.

An example would be the excellent diffing tools accessible through localhost, as well as things like issue tracking, which git doesn't understand as a concept.

There were some deal-breakers for me (kind of esoteric) and I switched back to git, but not without some annoyance.

riskable• 4 years ago

On Windows git performance isn't the greatest due to the sheer number of tiny little files (NTFS is absolutely awful). If you have a lot of small commits the number of tiny files grows and grows and grows and git can feel really slow compared to something like fossil. Especially if you're regularly copying your repo from one filesystem/machine to the next (without compressing it into something like a .zip first).

gwbas1c2• 4 years ago

At times a single large file is easier to handle than many small files.

Ever try and copy the .git part of a git repo? When it gets big, the copy gets bogged down by the sheer number of files.

72deluxe2• 4 years ago

I have 11809 files in my repo. Opening a git tool (GittyUp) on this repo takes minutes but seconds on Fossil (on a Pi4 with USB3 NVMe).

I use Windows + Linux machines + Pi and share between them so don't use the Pi4 exclusively but have noticed a difference in its ability to handle and open the repo.

hprotagonist2• 4 years ago

I’d use fossil more for personal work if i wasn’t hopelessly addicted to magit.

w0m• 4 years ago

I feel like the (outside-in) discussions of magit/orgmode are a stealth campaign for emacs adoption. Whenever I experiment with a vim/neovim migration it always seems... underwhelming, and I chalk it up to 'bad port'.

hprotagonist2• 4 years ago

pretty much :)

there are lots of people who use emacs for those two things and other editors/ides for other things.

i use emacs for more than the two, but certainly not for everything.

sshine4• 4 years ago

SQLite also doesn't use CVS, SVN, or Perforce.

If this title were "SQLite uses Fossil", it'd be more apparently just another ad for Fossil.

bayindirh8• 4 years ago

The author likes to have this slightly edgy tone, as far as I understood by reading a couple of his posts.

He likes to "engage" his audience in a way.

rat9988• 4 years ago

Which is not something I hold in high regard.

citrin_ru• 4 years ago

There is an element of click bait but for many young developers Git is the only VCS they ever knew so Git in the title would give more hints about the article content.

account-5• 4 years ago

Also because of the mention of "fossil" would have meant nothing to these young developers. The use of "git" immediately brings to mind version control and the question: what does sqlite use then?

oneplane• 4 years ago

On the other hand, the internet isn't just for young developers, and even if there are a billion other VCS options to pick from, the ones with the most traction are the most relevant. If you put "I can do Fossil and Subversion" on your CV but everyone is hiring for Git, you're not a great match.

jmt_• 4 years ago

As a younger developer, I agree with you. I've heard of Subversion & Fossil before but know practically nothing about them. By the time I hit the stage of learning version control in my development as a programmer, there was no questioning git would be what I would learn, any discussion I saw of other version control systems was written with a tone that suggested they were antiques to no longer be used. Sort of how psych majors learn about Freud for the historical significance but don't actually use his theories anymore.

guilhas• 4 years ago

Agreed, still Git too popular and Fossil is very interesting

donatj2• 4 years ago

I still believe that a squash+merge (or sometimes, rebase) workflow pairs better with pull requests

Unpopular opinion, but I find the value of having the actual surrounding context in which the code was developed more valuable than having a flat history.

discussDev• 4 years ago

I'm not sure this is too unpopular. In our groups not rebasing seems to be winning.

cube002• 4 years ago

It depends on that context, if they're the end of day "work in progress" or "typo" commits, I don't see the value in retaining those.

A non-linear commit history is much more difficult to run a bisect on which was enough to sell linear for me.

donatj2• 4 years ago

The bisect might be easier to do, but it's vastly less useful if you're looking for why and not what since the commits are not actually the authors originals and the state of the project you find is not the state the author authored them within.

mildred593• 4 years ago

typo commits might be really useful, especially if you can bisect a bug from one of those commits. it happened to me.

WorldMaker2• 4 years ago

`git bisect --first-parent` is just as easy as a bisect on linear history. (It just follows the "linear" line down merge commits via their first parent, which by convention should be their "base" parent.) Plus it gives you the option to bisect "inside" a branch if you need to inspect something "low level" once you've narrowed down the "branch" based on merge commit. I think if `--first-parent` were the default to git bisect a lot more people would care a lot less about "linear commit history". But it is still an easy flag to apply yourself.

beermonster• 4 years ago

I find the value of having the actual surrounding context in which the code was developed more valuable

But only if those commits are real atomic changes of code which compiles and not a series of 'oops', 'tests pass once again!' , 'forgot to commit this file' type inner-monologue type history.

And then there are shops where less technical authors are contributing solely via the GitHub WebUI. So you get a commits of the ilk 'Updated <Filename>' or ''Files added via upload'.

samatman2• 4 years ago

Author confuses SQLite's development model with Fossil's at the end, the latter welcomes contributions: https://www.fossil-scm.org/home/doc/trunk/www/contribute.wik...

pxeger1• 4 years ago

I think it's just ambiguously written. The way I first read it, I thought the author was indeed referring to SQLite.

ddevault• 4 years ago

Important note mentioned here is that sqlite doesn't accept external contributions and has a very small team. The Fossil folks and the sqlite folks are also essentially the same team. Fossil is a boutique VCS which essentially only exists for sqlite.

gwd• 4 years ago

...and that also has certain consequences for Fossil's design and workflow; namely, an aversion to the "rebase"-style development [1]. A project with a handful of known contributors could reasonably use Fossil; for a project with dozens of contributors, which also wanted to be able to accept "drive-by" contributions, Fossil would be unworkable.

[1] https://fossil-scm.org/home/doc/trunk/www/rebaseharm.md

sigzero• 4 years ago

Exactly. The Tcl Core Team (TCT) uses Fossil.

int_19h10• 4 years ago

I really like the notion of having the whole repo including issue tracking in a single file that can be easily copied around. But Fossil's stance on history - specifically, the deliberate refusal to implement history-editing functionality like squash and rebase - is a showstopper, and their rationale for it reads very much along the lines of "you're holding it wrong".

ARandomerDude• 4 years ago

Apparently it does support squashing.

Fossil does not support rebase. Here's an article from the author titled, Rebase Considered Harmful. While Fossil has the ability to squash merges, the primary workflow supported is merging.

masukomi5• 4 years ago

Rebase Considered Harmful article is here: https://fossil-scm.org/home/doc/trunk/www/rebaseharm.md

thesz4• 4 years ago

They actually right from my perspective.

It so happen that I usually do commit squashed changes to many files over code base with long and heavy legacy. I am afraid that squashed changes lose their utility for bisection search for a bug.

Yet, I required to do so. Mainly, because I can.

kevin_thibedeau• 4 years ago

The whole point of squash commits is to maintain bisectability while having breaking commits on your private branches.

thunderbong• 4 years ago

It does, but in a roundabout way. You can create a private branch, keep committing to it and when you're ready, merge it to the public branch with a single commit message.

IMHO, this solves all issues in a very elegant way - my local repository shows all the commits (both private and public) but everyone else's shows all the commits as intended by the developers. Additionally, the public history is maintained consistently without wondering if someone messed it up.

https://www.fossil-scm.org/home/doc/4d8aecdf/www/private.wik...

the_duke2• 4 years ago

How wouod you do code reviews then?

Git style mailing of patches?

int_19h10• 4 years ago

Very interesting, thank you! Unfortunately, the doc page doesn't make it clear that the merge would produce a single commit; I wish this was clearer.

Also, do I understand it correctly that it's a single branch named "private" per repo?

bonzini• 4 years ago

Squashing the whole development in a single commit will lose granularity though, and make it hard to review the changes later. You could just use quilt (patch queues) for development, it's not optimal to have two tools though.

ajross4• 4 years ago

It works for SQLite because SQLite is tiny. A quick check[1] shows that they have just 1360 commits over the last year, 76% of them from just one author (and just five authors total!).

You don't need complicated branch refactoring tools when you're only sharing with a handful of people. It's like working with RCS back in the day: history is still linear and no one needs to worry about cross-ports between feature branches.

[1] Ironically done using a github mirror, because those are the tools I know and they're fast.

justin662• 4 years ago

It’s not true that SQLite is tiny. I don’t know what portion of the test harnesses you’re seeing in the GitHub repo you looked at, but a portion of SQLite’s tests are proprietary. [0] The size of the SQLite test code is legendary, and exceeds that of the program code by an order of magnitude.

[0] https://www.sqlite.org/testing.html

RockingGoodNite3• 4 years ago

Squash and rebase are doing it wrong.

dtech• 4 years ago

why do you care about my 10 "wip" commits on a feature branch? Why am I not allowed to package a change in a nice set of commits once it's done/reviewable?

MichaelCollins3• 4 years ago

Do you find that you often have authoritarian takes, or just in matters pertaining to software development? If the latter, I suggest you pause and reflect on that. There is no one right way to do things, different workflows work for different people. Tools which accommodate the most number of people will be the most popular.

masukomi5• 4 years ago

their rationale for it reads very much along the lines of "you're holding it wrong".

Keep in mind that Fossil was written specifically to support the SQLite team's workflow. It was never designed for _anyone_ else's needs. They don't care if it doesn't work for other people's work styles because they didn't write it for other people.

flohofwoe• 4 years ago

...and Git was designed specifically for the Linux kernel development workflow, I think this is a small but important detail that's lost on many people who think that Git is the only versioning system in town. A lot of other versioning systems didn't or don't allow to rewrite history either, and they're doing their job just fine.

rkeene2• 4 years ago

This isn't a stylistic choice for Fossil, it's a technical one. In Fossil, all clones have identical artifacts (modulo private ones) so rewriting history by changing the set of artifacts creates a much harder synchronization problem.

Fossil DOES allow you to modify history, by adding new artifacts which retroactively change how previous artifacts are interpreted. They are called Control Artifacts and the set of things they can change are limited, but there's no reason additional ones cannot be added.

mvonballmo• 4 years ago

Agreed. I wrote up some thoughts about the "Rebase considered harmful" article in one called "Rebase considered essential" <https://www.earthli.com/news/view_article.php?id=3864>

"Who decides where one level of granularity stops and the next begins? I think it’s the author of the commits. My workflow over the last ten years is based heavily on being able to massage commits so that I can prepare what I share to the server repository, where it can no longer be changed. I agree that there should be an unalterable history, but disagree with the author on where that history begins."

"My workflow is considerably different than it was before I used Git or had access to rebase. I would now be much less efficient if I didn’t have rebase. It would make me constantly focus on cleaning up commits before I really care to. You could make the argument that cleaning up afterward takes more time, but I haven’t experienced that to be the case. Instead, I want to be able to set the priorities rather than worry about committing something that I cannot undo."

samus3• 4 years ago

Cleaning up commits can also make it easier for reviewers to audit your changes. For example by breaking up a larger feature into steps that don't warrant their own pull request because they are too granular. Or by separating changes that I maybe did only to make things look more neat.

masukomi5• 4 years ago

re "Rebase considered essential" article: thanks, and I agree completely.

ghoward2• 4 years ago

IMHO, both Git and Fossil are wrong here.

Fossil requires users to either commit stuff they don't want to see later, or hack away until finished, cherry-picking changes as necessary. This isn't a good workflow for everybody.

Git allows users to do anything they want, but your Linux master branch may not be the same as Linux Torvalds' master branch, and history can be changed at anytime, while merge commits can be completely rewritten, losing the actual merge information.

Both of these are wrong.

There should be a way to make WIP changes without committing it nor committing to it. (Yes, those are different things.) But there comes a point at which a change should be committed and committed to, to never change again. (With the exception of the nuclear option of erasing accidentally committed secrets and the like.)

Fossil is too rigid. Git is too flexible.

There is a middle ground. My design for such a VCS would have two kinds of commits: saves and full commits. Saves would be WIP stuff; anything goes. They would also be on auto-generated branches. Once WIP stuff is done, it should be to be rebased, squashed, merged, and/or cherry-picked into actual full commits on the branch that the saves created a branch off of.

But once those commits have been created, they are there forever. They should be a commitment. (I love the double meaning of the word "commit" here.)

With this design, people could have their cake and eat it too: they could have WIP stuff that they can still massage into a history they like, but then they would also have solid, reliable history with proper merge commits.

Tainnor• 4 years ago

There will always be a need for fully rewriting history, e.g. secrets that were accidentally pushed, code that was included in violation of copyright, etc.

benreesman4• 4 years ago

The SQLite people not using git or hg is like the Bell Labs people and three-button-mouse heavy sam/acme: when you’re right about everything 99/100 times, that 100th time you’re really going to double down on some dumb-ass thing.

In what possible world is telling the most successful set of design decisions about document management in the history of documents or management to get fucked not just “I’ve got goodwill to burn, let’s ride.”? SQLite is indeed badass, but cool it djb, I’m already stressing about the zero bugs ever thing.

HelloNurse• 4 years ago

Do you know SQLite is a few years older than Git?

benreesman4• 4 years ago

Indeed. I started paying serious attention to SQL in 1994 once I had read Stroustrup so many times I could recite it aloud, when did you get into databases?

Edit: I’d like to apologize for mouthing off like that. I’ve earned some amount of cranky old-timer cred, but not enough to just be a dick, which that was.

moeris• 4 years ago

three-button-mouse heavy sam/acme

Acme is great. I haven't been able to find another editor that integrates as tightly with the operating system. The closest I've gotten is Kakoune. And I wish mouse chording were a thing more generally. If we're going to use mice, we should optimize their utility.

benreesman4• 4 years ago

https://www.nbcnews.com/technolog/how-fast-fast-some-pro-gam... claims that effective APM can hit 600, that sounds high to me but not by a lot. 10hz on the keyboard is probably not a realistic sustained rate for a hardcore vi/emacs user.

I easily hit 4hz at the end of a 20 hour coding bender, and I’m old: the hot shit kids on modafinil must be doing 1.5x my decrepit ass.

If you can really move on the keyboard? Touch the mouse? I’ll grab a coffee while I’m at it.

Edit: It turns out that “fast emacs” doesn’t turn up a video of a real pro easily on Google. You can see a code God like Russ Cox being slow and clumsy as fuck if you want: https://youtu.be/dP1xVpMPn8M.

It’s a good thing thing he gets everything right on the first try (not sarcasm, Cox is an alien life form optimized for doing the impossible) because if he didn’t he’d finish pound-defining everything by about next March.

at_a_remove• 4 years ago

Programming existed before git. Programming will exist after git.

benreesman4• 4 years ago

Maybe? Technically we don’t know anything about a free software world without Linus in charge other than TMI about Stallman right?

hbrn• 4 years ago

What might look like a most successful set of design decisions for you, can be viewed as a set of unfavorable tradeoffs by somebody else.

To squash or not to squash is a never ending debate precisely because of that. There isn't a right way to do it, each approach has a set of tradeoffs.

Maybe they are reinventing the wheel, but at their scale it's a tiny decision that doesn't mean that much. If they find out they are wrong, they can choose to migrate to Git instantly.

Cyberdog• 4 years ago

Assuming by "the most successful set of design decisions about document management in the history of documents or management" you meant Git, I see what Fossil does as an iteration no those ideas, not so much telling them to "get fucked." Recognizing that things like issue queues are commonly used together with version control repos and then integrating the latter more tightly to the former is a step forward rather than a complete refutation.

MichaelCollins3• 4 years ago

They aren't burning any "goodwill" by choosing to use something different or odd. If you feel goodwill being burnt, I think you're taking their choice in VCS too personally.

SQLite doesn't solicit contributions from programmers outside their organization, so it's really no skin off your back.

headsup2022• 4 years ago

The single file repository is super convenient for one person projects, but just a word of caution to fossil users - backup the repo file often. A while back I downgraded the fossil binary (by accident) and committed a change to a local repo which caused it to be corrupted. It was a repeatable thing, and was annoying at the time, but not a showstopper. I only lost a day's worth of work. I wish fossil had done a version check between the binary and the repo to refuse to alter the repo if the binary version was incompatible.

RockingGoodNite3• 4 years ago

The sound of it from the article reminds me some of https://trac.edgewall.org/

gwbas1c2• 4 years ago

Anyone use Fossil for professional / team development?

I'd love to hear some real-world experiences.

indymike• 4 years ago

I use it for payment MSP software (Python based) that I license to a couple of companies. I'm a mostly solo on this project, and switched off Github and git about two years ago. I prefer prefer a "merge only, if possible, please" workflow in git, so fossil was pretty easy. Fossil does not have ubiquitous ide support like Git does, so if you are an IDE lover, it may not fit well. Also, for solo use autosync is nice - commits are automatically pushed.

Like SQLite, Fossil is very nice software.

vouaobrasil• 4 years ago

To be honest, I hate Git. The commands don't really make sense outside some basic stuff. For certain organizational setups it seems like overkill. I am not a full-time programmer but I do programming for my job and for personal projects sometimes and I never enjoy using Git.

meesles• 4 years ago

It's not intuitive, but it's an amazing tool that has stood the test of time for decades. I used to struggle with Git, but after putting in a little extra time and studying it more, now I feel like it's a crucial tool in my toolbox and I can use it for much more than _just_ storing my code. Git helps me understand what my colleagues wrote, gives me incredible control over breaking my work apart, and gives me tools to troubleshoot issues (bisect).

cholantesh• 4 years ago

I keep hearing that Mercurial does all of this and is more ergonomic, which begs the question as to why it never captured mindshare the way git did.

simiones4• 4 years ago

After moving to Git from Perforce for ~5 years now, I still miss P4V's Timeline view so much when trying to understand when and why some code changes happened.

It's like a kind of blame + log in a single UI, in Git terms, and is just wonderful for quickly moving through the entire history of a file.

pc86• 4 years ago

This doesn't really address any of the GP's points.

There is no consistency in the language or syntax of the commands when taken as a whole. Even individually some of the commands don't really make any sense and harken back to seemingly random unix CLI flags. "I wasted enough time to memorize them" doesn't mean they couldn't be cleaned up and made a little better.

If you're a single developer git is indeed overkill (not that there's a much better alternative).

Existenceblinks• 4 years ago

It should reduce its features, collapse options to one-way-to-do-thing. Undo mistake should be first class citizen, I've seen lots of gymnastic commands here, and they won't work for all cases.

masukomi5• 4 years ago

Richard Hipp (SQLite guy) spoke about the thinking behind writing Fossil instead of using Git on Corecursive episode 66.

If i remember correctly, it wasn't that they had any real problem with git, but that git didn't really fit with how they wanted to work, so they created their own thing.

https://corecursive.com/066-sqlite-with-richard-hipp/

2143• 4 years ago

Fun fact: Fossil uses sqlite. And sqlite uses fossil. Think about that.

SQLite is what I consider a well-engineered product.

And Richard Hipp is an excellent engineer. Listen to his podcast episodes on the changelog. It's fun. His mentality is interesting.

https://changelog.com/person/drh/podcasts#feed

vitiral• 4 years ago

It has bootstrapped it's VCS!

adaga• 4 years ago

Alt title - SQLite doesn't Git it

Great read :D

dier• 4 years ago

With regard to rebase and fossil vs git when you have a small team of trusted developers rebase is harmful because you lose information that could help you understand the _why_ of something. In contrast if you are working with thousands of untrusted developers you need a way to increase the signal-to-noise at the cost of some granularity.

This comes about from their development styles (see: http://www.catb.org/~esr/writings/cathedral-bazaar/).

- Use fossil if you prefer the cathedral style of development SQLite is open source but does not accept contributions.

- Use Git if you prefer the bazaar style of development. You want to accept hit-and-run style contributions from anyone.

cwilkes• 4 years ago

I don’t understand the dislike of rebase — do you want hundreds of commits that say “linter fix” and “reformatting”?

If there’s commits like “fixed off by one issue in refactor” then it should be a comment in the code so that future will explorers won’t make the same mistake.

Rebase by itself isn’t bad, it just has to contain a good description of what was done and why.

Edit: after posting I realize that I confounded rebase and squash — which is kind of ingrained in my head as that’s how I use GitHub to merge PRs.

Double_a_92• 4 years ago

They are better than one single commit with 100s of changed files. Especially when you are debugging things and want to know why exactly some line was changed in that way and by whom.

SQLite Doesn't Use Git

Related Stories