pmeunier

Pijul natively handles monorepos btw, that's the whole point.

Many Rust projects have done cool new stuff. Alacritty is possibly the greatest terminal emulator on the market today. I'm not into cryptocurrencies, but Parity and ZCash are doing cool stuff in that space thanks to Rust.

I'm also the author of Pijul, a much simpler and more scalable (yes, both! why choose?) version control system, and of Sanakirja, an on-disk transactional allocator to write persistent datastructures (like B trees, ropes, radix trees, HNSW…).

This is indeed one of the issues of Pijul, but patch application is extremely fast, which makes it easy in the vast majority of cases (we don't apply patches to files directly, but to an on-purpose on-disk datastructure).

We also have a tag features, allowing you to pinpoint versions and go back instantly, and we have plans to make that feature even lighter on disk space.

The only ones I know of are Pijul and Jujitsu which you mentioned. They're both quite new.

There are other Git frontends like Jujutsu: Gitless, StackedGit, GitButler, Sapling…

Even the idea of "SVN is the next Git" (which is the thing here) isn't quite new, PlasticSCM did it already.

Nothing like Pijul though, defining the actual problem carefully and rigorously, then actually solving it.

Sort of... But actually, as soon as you go offline it's distributed.

Even online is distributed: Google Docs needs stuff from distributed computing such as OTs and CRDTs.

One of the motivations behind Pijul was to manage custom versions of Nixpkgs while still benefiting from upstream commits. One issue that's hard with Git is that when you also want to contribute multiple changes back, you have:

1. A branch pointing to the latest nixpkgs head.

2. A branch with commit A (let's say commit A introduces a new package to nixpkgs).

3. A branch with commit B (changing some config file).

4. A branch currently at in use for your own machines, with branches 2 and 3 rebased on top of branch 1.

Every time you do anything, you'll have to remember the flow for getting the commits fetched/rebased. Which is fine if you have a DevOps team doing exactly that, but isn't too cool if you are anything other than a large company.

In Pijul, you would have a single channel (branch sort-of equivalent) and two patches (A and B) instead, which you can push independently from each other at any time if you want to contribute them back.

Darcs does the same but wouldn't scale to Nixpkgs-sized repos.

Also, the conflict resolution is just another patch (Pijul patches aren't just regular diffs, they have a lot more information), so should you decide to merge it back upstream after all, you can also cherry-pick the conflict resolution along with the conflicting patch, and also without changing the hash.

IMHO there are different ways to design a version control system:

1. The SCSS/Git way, aka the hacker way: look at what we can do with existing stuff, and use that to build something that can do the job. For example if you're one of the world's expert on filesystem implementations, you can actually produce a fantastic tool like Git.

2. The mathematician's way: start by a model of what collaboration is, and expand from there. If your model is simple enough, you may have to use complex algorithms, but there is a hope that the UI will match the initial intuition even for non-technical users. Darcs did this, using the model that work produces diffs, and conflicts are when diffs can't be reordered. Unfortunately this is slow and not too scalable. Pijul does almost the same, but doesn't restrict itself to just functions, using also points in the computation, which makes it much faster (but way harder to implement and a bit less flexible, no free lunch).

3. The Hooli way: take an arbitrary existing VCS, say Git. Get one of your company's user interviewer, and try to please interviewees by tweaking the command names and arguments.

The tradeoff between 1 and 2 is that 1 is much more likely to produce a new usable and scalable system fast, but may result in leaky abstractions, bad merges and hacks everywhere, while 2 may have robust abstractions if the project goes to completion, but that may take years. OTOH, method 3 is the fastest and safest method, but may not produce anything new.

So, I am the main author of Pijul, and I also don't quite see how to do much better (I'm definitely working on improvements, but not technically radical). But the causal relationship isn't the one you may be thinking: it is because I thought this was the ultimate thing we could have that I started the project, not the other way around.

I've done very little Zig, so this may not be very informed. Indeed we don't use Rust intrinsically that much. With associated type constructors, Rust would be the best language (I know of) to write Pijul in. Without it, most of the polymorphism uses macros anyway. Zig or Rust would make little difference for Sanakirja I believe, and this is where the entire CRDT thing is done.

Yes indeed, we would not have started Pijul without the hope that at least theoretically, patch-based designs could be faster than snapshots. "Patch-based is slow" without any other argument is not a very informed claim, Pijul is actually faster than Git in some cases (in fact Pijul is faster where it matters most IMHO: large files and/or large repos, conflicts and blames). Not because we're better at tweaking C code (we're definitely not!), but because we designed our datastructures like theorists, and only then looked at how (and whether!) to implement things. One advantage we had over Linus is that we had no time pressure: we could well use Darcs, Git, or Mercurial to write Pijul initially (we used Darcs, actually), and it didn't matter much if we failed.

It took a little bit of work to get that down to actual fast code, for example I had to write my own key-value store, which wasn't a particularly pleasant experience, and I don't think any existing programming language could have helped, it would have required a full linear logic type system. But at least now that thing (Sanakirja) exists, is more generic, and modular than any storage library I know (I've used it to implement ropes, r trees, radix trees…), and its key-value store is faster than the fastest C equivalent (LMDB).

Could we do the same in Haskell or OCaml? As much as I like these two languages, I don't think I could have written Sanakirja in a garbage-collected language, mostly because Sanakirja is generic in its underlying storage layer: it could be mmap, a compressed file, an entire block device in Unix, an io_uring buffer ring, or something else. And the notion of ownership of the objects in the dictionary is absolutely crucial: Sanakirja allows you to fork a key-value store efficiently, so one question is, what should happen when your code drops a reference to an object from the kv store? what if you're deleting the last fork of a table containing that object? are these two the same thing? Having to explain these to a GC would have been hard I think.

I wouldn't have done it in C/C++ either, because it would have taken forever to debug (it already did take a long time), and even C++-style polymorphism (templates) isn't enough for the use of Sanakirja we have in Pijul.

remember the "poop" paper about mmap for databases, right? Well, guess what: having a generic key-value store implementation allowed me to benchmark their claims, and actually compare congestion, throughput, speed between mmap and io_uring. Conclusion: mmap rocks, actually.

Not the same category of tools: Pijul and Fossil have radically different designs, whereas jj is a Git frontend, and Sapling a Mercurial fork.

It's mostly about Rust adding lots of features I am not interested in, and not adding the ones I need for the project. Sanakirja was hard to write in Rust, not a single concept of the language matched what I needed, in the end I had to write tons of macros, and the API is hard to use. Zig would have probably made it more natural from the beginning.

There are other things related to the community/zealots/Mozilla/Rust foundation, but I'm not sure this is the proper place.

Edit: Git zealots are worse than Rust zealots, I attribute this to Git being "harder to learn" (i.e. never really does what people think it does) than Rust.

In Pijul the head is a CRDT. Having used it for years to develop itself, I can definitely imagine!

Pijul handles binary files natively, using a number of mathematical tricks removing the need for extra layers (layers like LFS). If you're interested come talk to us!

At the time we thought our tools were good, and really meant it.

Now it's different, we think our new tools are good, and we really mean it. But it's different.

Wait, nobody in the Pijul team brags about being in Rust. If Pijul were written today it would probably be in Zig, for many reasons.

I can answer that question, as both the author of Pijul and as someone most Rust zealots usually don't like very much.

Nothing magical in Rust, at the time it was the only language that worked seamlessly on Windows and Linux, and made it easy to manipulate on-disk datastructures, which is what makes Pijul fast. The new take on version control that Pijul brings can be roughly described as seeing "files with conflicts" as a CRDT. It was therefore crucial to be able to manipulate complicated datastructure without loading anything, and Rust seemed to let us do that without having to do C++ and spend our time debugging allocation issues in our tricky algorithms, while never being really confident that we were done.

The library I wrote for this, called Sanakirja, is actually faster than the fastest C equivalent, but this isn't because "Rust is magical", far from it.

You're describing working on your own single-author project, in which case there is indeed little difference (Pijul has less tooling).

In practice on actual real world cases, there are lots of differences when you start working with others: even on a small project, you don't have to plan your feature branches anymore, conflicts are solved once and for all, you get free cherry-picking of bugfixes to your production branch, etc.

When your project scales, there are even more differences: commutativity handles large repos for free, patches describe large files much more efficiently than by giving their whole contents (which snapshots do).

Git to Pijul is a much smaller change, it is much more difficult to justify.

Having used both extensively, I don't think this is true at all. I don't see as much difference between SVN and Git, as I see between these two and Darcs/Pijul (even though Darcs has scaling issues).

why would you (or the authors of Pijul) care about this extremely rare case?

I'm the main author, and my answer is: because it allowed to to model with great mathematical rigor what conflicts are, how to represent them and how to treat them in the most intuitive and accessible way. The rest is indeed less essential, but still nice to have (I like doing my backups on an external hard drive using Pijul to copy my software projects).

This question is really deep. The "distributed" nature of Git makes some things easier than SVN: you can scale to large teams, work offline, split a repo into independent subrepos for a while (managed via branches; good luck with the merge!).

However, this isn't really what Pijul calls "distributed": in Pijul, this term is about the work that people are doing when working collectively on a shared file. Which datastructures allow asynchronous contributions to happen? How to represent conflicts? Those questions belong to the field of distributed computing, together with things like CRDTs and leader elections.

First, thanks for the patient and kind advice. This is so rare I didn't even know you were allowed to talk like that online.

I think you should try to let go of the idea of "evolving the license" of an open source project that invites contributions.

Yes, this is becoming clearer now, but wasn't for a long time, since choosing any license on a complicated tool like this one, which few people fully understand, inevitably sparks discussions on the license, because if you got to say something, it is easier to discuss than commutativity, pushouts (hard-core theory), Sanakirja, endianness (hard-core practice).

This is quite a neat trick to pull off, although I fear it was unintentional.

;-)

Yeah I saw that PR

There were many: a single one wouldn't have changed my mind. And many more on what projects/topics/software I should be interested in.

we have `pijul diff -sU`, which is similar to `git status`. Also, feel free to contribute!

By not being a CI tool, nor claiming to solve such Turing-complete problems.

Pijul has a theory of textual changes, but indeed doesn't care at all about what you write in your files: that's your problem!

Oops, don't look at what we do! These repos have been used for dogfooding and bootstrapping extensively, they have the worst structures and aren't good examples of nice, clean workflows. The tool isn't really "experimental" anymore, its repos are still used for heavy experiments.

But its not clear at all how those "version identifiers" can be used or indeed if they are even implemented at all yet? At least based on the manual, no commands seem to take "version identifier" as argument, nor can I see them anywhere in Nest web UI.

`pijul log --state` does that, and various commands (tags, fork) do it as well. Tags are due for a redesign, since they can be made much more efficient with a really cool new design, and make Pijul a perfect hybrid between patches and snapshots.

The key insight of Pijul is to be the smallest generalisation of a file that is a CRDT with insertions and deletions of bytes as its two operations, where "smallest" and "file" are meant in a specific sense.

The main thing that makes it all work is the extreme performance of its storage backend, which allows to manipulate a graph datastructure directly on disk, and avoid as many IO operations as possible while doing that. This works well, as all operations in Pijul (with some caveats) work in a time logarithmic in the size of history. And yet, it is slower than Git for some operations.

Therefore, what you're suggesting is linear in the size of history (importing), i.e. exponentially slower than Pijul, for every single merge!

This problem was never reported a single time on our Zulip or by email, but I'm glad you asked, because this choice wasn't obvious:

- The main reason the CLA is there is that earlier versions of Libpijul saw lots of online arguments about its very reasonable GPL2 license, which made me doubt of the choice of GPL2. These arguments were often followed by "@me was there"-style contributions to Libpijul, like applying linter fixes without understanding any of the code. I got scared of having to ask all past contributors for permission to release it under (say) a BSD license (which I have even recently discussed with the FreeBSD maintainers during FOSDEM 2024).

- Also, the fact that the Pijul binary is GPL2 (without a CLA) and dependent on Libpijul forces me, in the case of an evolution of the license, to choose something compatible with GPL2. Deal breaker, really?

- Another goal of the CLA was to experiment with something cool and meta, at the interactions between version control and licensing: I wanted to test the idea that the CLA was a single patch (or a sequence of patches, strictly ordered by dependency), and contributors would be required to add a dependency on the CLA patch. That would make them state in their patches which version of the CLA the agreed with when they recorded. And as Pijul does a lot of dogfooding…

- Other than that, the Pijul plumbing is indeed mostly meant to become just a static algorithm after a while, not many features are missing. The goal of Pijul is to have essentially three core functions, create a patch, apply a patch, unapply a patch.

When the contents has a conflict, git and pijul behave similarly.

Not really: Pijul can record a conflict resolution as a patch, and apply it in a different context. Also, the conflict doesn't "come back", so you don't need extra hacks like rerere/jujutsu.

Pijul just removes the manual work when there is no conflict in the contents but the history is different.

This is true, but could be confusing as our definition of conflicts isn't based on contents, but on operations, which is very different from Git (Git doesn't detect all conflicts).

the notion of commutativity here is very weak and counterintuitive; it only seems to cover the applicability of an auto-merge heuristic

This is completely false: in Pijul, any patches that could have been produced independently can be applied in any order without changing the result. There are 0 heuristics in Pijul, unlike in Git where even random line reshuffling can happen (there are examples in the "Why Pijul" section of the Pijul manual).

Obviously, deciding whether a merge has the correct semantic is Turing-complete, and Pijul doesn't try to do any of that.

As probably the person in the entire world who has been wanting this for the longest, thank you! Please share on Pijul's Zulip, and ask for any help you may need.

`pijul fork`. There you go, I wrote an entire key-value store just to get that to work. It turned out to be faster than all others, but that wasn't intentional.