HN user
Developer of Mutagen, Docker Captain, Recovering Physicist
https://mutagen.io/ https://xenoscopic.com/
[ my public key: https://keybase.io/xenoscopic; my proof: https://keybase.io/xenoscopic/sigs/bm3iD1SNiri3kSj45MvbV_ORylBHBAmq8e9Z3BAnHwM ]
Thanks! I totally get it — I think a lot of time with acquisitions in general there's also a lot of good faith intentions that never manifest for whatever practical reason(s), so I can totally understand why the cynicism arises. I appreciate the positive wishes and I will do my best to convert those into positive outcomes!
I'd point out that most of the foundational components of Docker's technology stack are FOSS (e.g. [0] [1]). Mutagen also has (and had, prior to acquisition) closed-source components and components licensed under non-OSI licenses (e.g. SSPL), so evaluate my response in that context (and also please note that these are my responses, and not representative of Docker's opinions/positions). Also, as mentioned in the acquisition FAQ [2], we don't have any plans to change the open-source licensing structure, and we're still open to contributions.
It's a delicate balance to strike, and almost more delicate to discuss. At the end of the day, an open source business is still a business, and you have to make money to eat, shelter, and continue writing code. I can certainly appreciate that there are different approaches to balancing that with open-source (e.g. consulting rather than close-sourcing), but so long as you're setting natural (non-contrived) boundaries between open-source and proprietary, then I don't think you're doing anything wrong. In fact, several companies were embedding and making money from Mutagen before I was, but that never really bothered me — that was the freedom I was affording them as users.
I have nothing but the utmost respect for user freedom and I've always strived to offer that with Mutagen (e.g. allowing people to disable components that might not fit the OSI definition of FOSS). I also have nothing but the utmost respect for the authors of open-source software; Mutagen stands on the shoulders of many different dependencies and I've always strived to ensure that we're acknowledging those in a manner that is compliant with their licensing requirements (or going above and beyond that).
In the end, it's a tough but fair question. I don't think my views have changed and I don't think there's any real incompatibility. It's a balance I'll necessarily continue to assess on a daily basis, both at Docker and in any other FOSS I write.
[2] https://www.docker.com/blog/mutagen-acquisition/#mutagen-faq
I'm really happy with this outcome! I've been a Docker Captain for almost 3 years now, so being an employee at Docker was a very easy switch to flip (at least mentally). As I said in another comment, this really felt like the logical step given that almost all of Mutagen's users are Docker users, and I think this is the best possible outcome for the project too!
Mutagen's core users are primarily Docker users, and I honestly think they'll be much better served by the tighter integrations that we'll be able to offer now. If I didn't think that were the case, and I didn't think there were a net win (for Docker, Mutagen, and their users), then I wouldn't have joined.
Well, that's one of those things I won't answer, but Docker is actually really great about listing nominal salary ranges for their positions on their Docker Careers page [1] - I'd encourage anyone to check out the openings there!
I'm the creator of Mutagen, happy to answer any questions that I can about the project, acquisition, or anything else!
If anyone has any questions about Mutagen (or integrating it into their Docker-based workflows), I'm happy to help.
Just one clarification on the article: Mutagen offers Docker Compose integration, not Composer integration (Composer is a PHP package manager). However, as mentioned, DDEV is a great option if you're looking to do containerized PHP development while using Mutagen for performance improvements.
I can't predict ultra far into the future (who can these days... :|), but Mutagen has been under active development for about 6 years now[0]. At the moment I have enough funding to work on it full-time until at least the middle of next year, though I also do Mutagen-related contracting and consulting work to support the project. Mutagen's Docker Desktop extension is going to be a freemium product designed to support the project more directly, which will hopefully allow development to continue indefinitely.
[0] https://github.com/mutagen-io/mutagen/graphs/contributors
There's no conflict resolution UI at the moment (either graphical or command-line based). Mutagen's conflict resolution is primarily performed via the specification of its synchronization mode (which can automate the resolution of most conflicts) and (in the case where conflicts can't be resolved automatically) by manually deleting the file that should "lose" the conflict (because Mutagen will synchronize a modification over a deletion).
Mutagen provides fairly detailed reporting of synchronization status, file staging progress, and change application problems via its "mutagen sync monitor" and "mutagen sync list" commands. These also support JSON output (and Go-template-style formatting), so you can pipe this information into other tooling. If you need REALLY detailed information, you can look at the debug or trace-level logs from the Mutagen daemon, but that's typically only for debugging during development of Mutagen itself.
Mutagen is somewhere in between Unison and Syncthing on the topology front, but closer to Unison. It still only supports two endpoints per synchronization session, but unlike Unison it doesn't require that one is local (i.e. you can do remote-to-remote sync using your local system as a proxy). But with both Mutagen (and Unison), you can set up a hub-and-spoke topology if you want to sync with multiple nodes.
One option I often recommend is setting up multiple synchronization sessions targeting different parts of a particular codebase with different synchronization configurations (kept mutually exclusive via ignore specifications). Quite often people want unidirectional replicas for a certain folder (e.g. a build directory) with more standard bidirectional synchronization for the rest of the code. It can be a little more complex to orchestrate, but you can use Mutagen's project functionality or a shell script to automate the setup a bit.
In many ways Mutagen and VSCode's remote extensions are the same idea, with trade-offs in terms of flexibility vs. integration.
Shared systems with multiple non-admin users was one of the original motivating use cases for tighter default permissions.
I don't think there's any scenario where one can perform truly secure development work on an untrusted system. You could certainly store encrypted code in an untrusted location, but there's not much you could do with it on that system (without a hypothetical compiler or tool that maybe supported some sort of homomorphic-encryption compilation operations?). Even decryption on-the-fly for processing by regular tools wouldn't be secure on an untrusted system. And running any code there would be equally insecure.
I'd imagine that for any seriously sensitive work, one would only want to work in highly controlled, trusted, and firewalled environments. If there's a scenario I'm missing though, definitely let me know.
Here's a bit of a comparison that I wrote the last time Mutagen was posted:
The primary benefits:
- Mutagen performs bidirectional synchronization (though it can also operate unidirectionally); rsync is unidirectional
- Mutagen uses recursive filesystem watching to avoid full filesystem rescans (whereas rsync always does a full filesystem rescan). This allows Mutagen to provide a more "real time" sync.
- Mutagen has an active synchronization loop that doesn't require manual invocation.
- Mutagen has more idiomatic Windows support.
- Mutagen doesn't require that it be pre-installed on both endpoints.
Both use differential transfers (i.e. the "rsync algorithm") for transferring individual files.
There are other differences, of course, as well as similarities. Mutagen's design is tuned for development work, rsync's design is tuned for replication. I still use rsync for archival operations on a daily basis - it's great!
Conceptually speaking, Mutagen and Unison are very similar (and actually I mentioned Benjamin Pierce's work in another comment here asking about the sync algorithm - fantastic stuff!). I tend to avoid direct comparisons because they always come across one-sided, but some cursory differences:
- Mutagen tries to integrate recursive filesystem watching very tightly into its synchronization loop to drive synchronization and allow for near-instant filesystem rescans
- Mutagen automatically copies an "agent" binary to remote systems to support synchronization, so no remote install is required
- Mutagen uses Protocol Buffers for its data storage, so synchronization sessions created with older versions continue to work with newer versions
- Mutagen written in Go, Unison in OCaml (which allows Mutagen broader platform support "for free")
- Mutagen tries to treat Windows as a first-class citizen
- Mutagen uses race-free traversal (e.g. openat, fstatat, unlinkat, etc.) to perform operations
Obviously the internal implementations are different, but both use differential (rsync-style) file transfers, both use the same reconciliation concepts, etc.
Mutagen has the advantage of Go, recursive filesystem watching, and modern POSIX/Windows APIs that didn't exist when Unison was originally written, though some of that functionality has been brought into Unison.
For a comparison with Syncthing (and to some extent Unison), check out this comment[0].
It should work fine. Many users use Mutagen on multi-GB codebases. If we're talking something larger (say 10s of GBs or TB-sized monorepos), then there are some tweaks you can do to make life with Mutagen a little easier. Feel free to reach out to jacob[-at-]mutagen.io if you have a specific use case, or pop over to the Mutagen Community Slack Workspace[0] to chat.
The general philosophy with Mutagen is to (a) delegate encryption to other tools and (b) use secure defaults (especially for permissions).
So, for example, Mutagen doesn't implement any encryption, instead relying on transports like OpenSSH to provide the underlying transport encryption. In the Docker case, Mutagen does rely on the user securing the Docker transport if using TCP, but works to make this clear in the docs, and Mutagen is generally using the Docker Unix Domain Socket transport anyway. When communicating with itself, Mutagen also only uses secure Unix Domain Sockets and Windows Named Pipes.
When it comes to permissions, Mutagen doesn't do a blanket transfer of file ownership and permissions. Ownership defaults to the user under which the mutagen-agent binary is operating and permissions default to 0700/0600. The only permission bits that Mutagen transfers are executability bits, and only to entities with a corresponding read bit set. The idea is that synchronizing files to a remote, multi-user system shouldn't automatically expose your files to everyone on that system. These settings can be tweaked, of course, and in certain cases (specifically the Docker Desktop extension), broader permissions are used by default to emulate the behavior of the existing virtual filesystems that Mutagen is replacing.
Also, while Mutagen's exact implementation is novel in a number of ways, I would be remiss to not point out that huge amount of academic work in this field was done by Benjamin Pierce[0] and later implemented in Unison[1].
[0] https://www.cis.upenn.edu/~bcpierce/papers/index.shtml#Synch... [1]: https://www.cis.upenn.edu/~bcpierce/unison/
The synchronization uses a repeated three-way merge algorithm, very similar to Git's merge when merging branches. It is triggered by recursive filesystem watching, which is also used to accelerate filesystem rescans. It maintains a virtual most-recent-ancestor and uses the two synchronization endpoints as the "branches" being merged. Much like Git has "-X ours" and "-X theirs" options, Mutagen also has automated conflict resolution[0] modes that can be specified. You can find the reconciliation algorithm here[1] (and there are an exhaustive set of test cases in the corresponding _test.go file).
To avoid a large class of race conditions (at least to the extent possible allowed by POSIX and Windows), Mutagen will use `*at` style system calls for all filesystem traversal on POSIX systems, with a similar strategy on Windows.
Also, to avoid race conditions due to filesystem changes between scan time and change-application time, Mutagen will perform just-in-time checks that filesystem contents haven't changed from what was fed into the reconciliation algorithm.
[0] https://mutagen.io/documentation/synchronization#modes [1]: https://github.com/mutagen-io/mutagen/blob/master/pkg/synchr...
The benchmarks will likely be highly dependent on your use case, but SSHFS-style virtual filesystems (specifically those backed by FUSE) typically have significantly lower performance than something like an APFS/ext4/NTFS filesystem that Mutagen could target with synchronization.
All of your readdir()/stat()/open()/read()-style calls will suffer significantly on virtual filesystems, and unfortunately these get hit a lot by things like IDEs (e.g. when indexing code), compilers, and dynamic language runtimes (especially PHP).
No tool is at fault in this chain, of course, it's a hard problem. Mutagen is able to offer better performance by being a little less dynamic and creating "real" copies of all the files on a more persistent filesystem.
On the synchronization front, Mutagen's only goal is to facilitate the synchronization of files (albeit with a focus on development-related settings and low-latency for a "real time" feel). It doesn't attempt to integrate with any higher-level tooling (except in the cases of Docker Desktop and Compose, which is facilitated via external projects). That sort of tooling, language, and framework-specific integration is a bit outside the project's target scope (and something that becomes very domain-specific).
Mutagen will, however, happily operate between different operating systems and architectures, so things like working with a remote amd64-based Docker engine from your local arm64-based laptop are totally possible.
Also, several external projects (such as DDEV[0] and Garden[1]) do use Mutagen as a low-level component in their stack to provide synchronization that does "know" a bit more about the framework that you're using.
[0] https://ddev.com/ [1]: https://garden.io/
Mutagen author here — happy to answer any questions about Mutagen[0], its Docker Desktop extension[1], its Compose integration[2], or anything else!
[0] https://mutagen.io/ [1]: https://mutagen.io/documentation/docker-desktop-extension [2]: https://mutagen.io/documentation/orchestration/compose
It should be possible, especially since Mutagen is already built for almost all of Go's supported architectures. The biggest issue would just be implementing the race-free filesystem traversal that's used on Windows and POSIX (or potentially living without it given that WASM would probably provide sufficient sandboxing). In any case, most of the work would take place in Mutagen's `filesystem` package, where those syscall equivalents would have to be figured out. The rest of Mutagen should compile without modification. You'd also need to define a transport for Mutagen to use, which would depend on the exact mode of operation you're looking at, but that's the easier problem to solve. Mutagen v0.15 is going to be focused on extensibility (including custom transports), so something like this may become a reality soon.
Sure, that's a great question. I'll preface my response by saying that I'm a huge fan of Syncthing (and that Mutagen's use cases form a Venn diagram with those of Syncthing (as well as tools like rsync)). Technologically, all three of these tools are very similar in terms of using the rsync differential transfer algorithm, but their architectures and primary use cases differ. I think the core differentiators with Mutagen are:
Development-oriented: Mutagen's sync configuration is primarily focused on development, so it adds more granular controls for things like uni-/bi-directionality, conflict resolution, ignores, symbolic link handling, etc. It also has a permission propagation model that's focused on things like cross-platform executability propagation and preservation (i.e. between Windows and POSIX), as well as operating in multi-service environments where many different process UIDs/GIDs might be in play. It also handles weird filesystem quirks (like macOS Unicode decomposition).
Low-latency: Mutagen's goal is to reduce the latency of sync cycles (i.e. time from local edit to change reflection on the remote) to an imperceptible level. It uses a lot of tricks to try to do this, but the goal is really to use the absolute best filesystem watching mechanism for each platform and to integrate that tightly with the sync loop. On Linux, for example, Mutagen is now starting to experiment with the recently revamped fanotify[0] API to get highly scalable but low-latency watching (as opposed to the emulated and janky recursive watching emulation with inotify that most tools use). It also uses tricks like rsync-diffing of the metadata snapshots that it transfers to get latency as low as possible. The eventual goal is to reach sub-100ms sync cycles for multi-GB codebases, and I think that's pretty close.
Git-like sync: Conceptually, Mutagen's sync algorithm is like a filesystem watcher + repetitive three-way Git merge (with the difference being that file transfers are deltified and the merging (potentially) affects both endpoints). This means it tracks content in a manner very similar to Git's CAS and branches, which is a little different than the way that Syncthing does it. This affords (in my opinion) more precise identification of conflicts.
Distrust: Mutagen takes a more aggressive approach to mutual distrust between endpoints, working hard to ensure that a malicious endpoint can't read outside the synchronization root on the other endpoint via symbolic links or maliciously crafted paths. It does this by using POSIX *at functions to traverse the filesystem and perform operations. This avoids issues like CVE-2017-1000420. You can harden this further by using unidirectional sync and other configuration options. This makes it well-suited to cases where multiple users might be syncing to file storage on a shared system (say on a SaaS platform) (though, at least in that case, you can protect yourself and users with the filesystem namespacing afforded by containers).
One-sided installation and flexible topology: Mutagen's primary M.O. is injecting small "agent" binaries to remote systems via a copy mechanism (such as `scp` or `docker cp`), so you don't have to manually install it on both endpoints. This is less important to "full stack" cases like Okteto, where your tooling can handle the setup of Syncthing on the remote, but it makes working directly over SSH or in ephemeral containers significantly more convenient. And Mutagen's architecture is also really flexible, allowing it to sync files and forward traffic between any combination of local and remote endpoints (including remote-to-remote, proxied via the local Mutagen daemon).
Command-based transports: Mutagen uses the standard I/O streams of commands like `ssh` and `docker exec` as its transport (similar to tools like Git or rsync), making it easier to target remote environments with your existing tooling and configuration. Again, this is less of an issue for a case like Okteto's, but is useful in the standalone case.
Network forwarding: This is outside the scope of sync, but Mutagen offers OpenSSH-style TCP/UDS forwarding (with the difference from OpenSSH being that Mutagen's forwarding is persistent and managed by a background daemon). This offers support for doing things like forwarding a local socket to a remote Docker daemon over SSH, and then forwarding web application traffic over that underlying forwarding by using Mutagen over `docker exec` (or reverse forwarding, or forwarding between two remotes and bridging them via your laptop, or loads of other crazy shenanigans).
I hope that clarifies things a bit. Ping me via email if you want an expanded comparison.
One way you can do this with Mutagen is to use a hub-and-spoke topology, where one copy of the files (say the one on the laptop where you're coding) is the "hub," and then you have multiple two-way (or one-way) synchronization sessions going out to the various "spoke" endpoints. In the two-way case, changes on a spoke endpoint will be propagated out to the other endpoints by first propagating to the hub. Obviously, you can have contention in this case if you have ultra-high-frequency updates to the same sets of of files, though the conflicting operations would have to occur on a smaller timescale than that of a single Mutagen synchronization cycle (which is typically somewhere around 100ms for an average codebase). Also, in the case of contention, Mutagen will simply display the conflicting files to you and you can delete the "losing" copy. You can also set auto-resolution behavior with Mutagen's `two-way-resolved` mode to let the hub win in conflicts, or use `one-way-replica` mode to make all the spoke endpoints replicas of the hub, or some combination of all these things, etc.
To be totally transparent: funding is a bit of an open question, although there's no risk of Mutagen disappearing in the short term.
Mutagen was actually part of YC's S19 batch, and it still has a significant amount of runway from that, but also some revenue from contracting work.
I really want to keep as much of Mutagen as FOSS as possible, ideally MIT licensed. I did just add a small portion of code under the SSPL, which I might experiment with dual licensing to SaaS embedders of Mutagen (since it's really only useful in cases where Mutagen is being embedded in other tooling), but even that I wanted to keep open source for other FOSS projects embedding Mutagen.
In the near term, I have some ideas about plugins and tooling that I want to build on top of Mutagen that will probably be closed source, but those will be separate entities from Mutagen itself and the aim will be to avoid compromising on any functionality that belongs in the core of Mutagen.
Feel free to reach out on Slack or GitHub issues or by email if there's anything I can help with.
I think framework-specific stuff like that is better handled by higher-level tooling, typically something that's just embedding Mutagen. DDEV, for example, adds that type of functionality for PHP-based projects, while embedding Mutagen underneath.
The goal with Mutagen itself is (ideally) to be invisible - only providing synchronization and forwarding. Even Mutagen Compose (being just tweaked version of Docker Compose) probably falls too low in the stack to know anything about the specific frameworks being used atop it.
That being said, I think this is something that Compose wants to address, and that Mutagen Compose could inherit once it's added to the Compose Spec and implemented into Docker Compose. Things like profile specifications in Compose files are designed to encapsulate these types of operations (among other things).
Glad to hear that it works for you. Don't hesitate to reach out on the Mutagen issue tracker or Slack channel if you run into any issues or questions - I'm always happy to help.
Yeah, Randy's done a great job with the integration - it seems to work seamlessly for most users as far as I can tell. I'm really excited to get Mutagen v0.14 and beyond into DDEV, because support for fanotify[0] is being added and it's going to make Mutagen's Linux filesystem watching unbelievably faster for container-based development.
The goal with Mutagen is real-time remote development, not source control or complex networking. It's designed to facilitate making changes to code locally and then having those immediately propagated to a remote system for compilation, execution, etc. Basically, it's designed for the work you do between Git commits, saving you the need to do a full commit, push, pull, evaluate cycle to test your code on a remote system.
Basically, it's like VS Code's remote development extensions, but editor-agnostic and more flexible.
It can tolerate disconnection very reliably, and it will automatically reconnect to synchronization and forwarding endpoints as soon as possible.
Its synchronization algorithm in particular (which is essentially a (repeated) three-way merge with rsync-style differential file transfers) is designed for safety and robust tracking of the changes that it has propagated. It's also capable of resuming file transfers once it reconnects.