Collaborative Software That’s Wary of the Cloud

https://www.wired.com/story/collaborative-software-wary-cloud/

by mpweiher • 7 years ago

70 11 7 years ago

digb• 7 years ago

Didn’t Google Wave first use CRDT’s? Well before 2011?

dangoor• 7 years ago

I thought Google Wave was based on operational transformation? [stackoverflow about the difference](https://stackoverflow.com/questions/26694359/differences-bet...)

jakelazaroff• 7 years ago

Here’s the link to the Ink & Switch paper: https://www.inkandswitch.com/local-first.html

They also have a bunch of other articles that are great reads. The one on end-user programming is especially good, imo: https://www.inkandswitch.com/end-user-programming.html

SkyMarshal• 7 years ago

Thanks, that was so much easier to read than some long-winded narrative about it.

sansnomme• 7 years ago

Just to clear something up: CRDTs, op transforms etc. only work well when your collaboration is near real-time. The oft-cited "work off-grid for two weeks and magically sync your work with everyone else" doesn't really work out in real life. You will need merge and diff tools. The equivalent of CRDT's behavior would be to merge git conflicts by time — whoever commits first gets merged.

adamfeldman• 7 years ago

Ink & Switch are behind Automerge: "A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically."

https://github.com/automerge/automerge/

myu701• 7 years ago

RE: storing user data in a central server or on device, this feeling is what it must be like to feel old.

I remember developing applications for the pocket PC in the .NET Compact Framework 3.5 era. The advice from the Larry Roofs and Dan Ferguses of the world was simple: you can't assume you have network connections all the time, so get your data locally on the device, and sync it when the network returns.

All these years later, while the devices are much more powerful, the network is still not reliable enough to treat it as interchangeable with local storage. Why stream my music over Google Play Music when I know that the hand-off from one cell tower to another will consistently cause a few seconds delay in re-establishing a connection and thus my VPN will have to reconnect, and other knock-on effects?

Yes, use the cloud where appropriate, but if your app is useless without a data connection, it (IMHO) likely means that it could be replaced with a mobile website which at least operates under different user expectations.

I'm excited about CRDTs, once someone makes a robust-enough library for 'good enough' CRUD style abstractions for them (automerge may be this, but I'm a no-JS snob), that library will likely take off in usage, IMO.

z3t4• 7 years ago

It depends on what transformations are allowed. So you cant have a general solution and apply it to many use cases.

russnewcomer• 7 years ago

Really interested in this idea for a significant personal reason - I've been working on a Medical Records system for a friend in the developing world, to replace an existing system that is 'carry around binders of data wherever we go and dig through them to find patient records.' The problem with this, of course, is that if you don't have the right binder with you (and they have multiple health professionals and cultural reasons related to how they see patients meaning that they often might not have the right binder), you have no access to that patient's records.

The solution I've been working on, because of their context, is a C# Windows Forms app (the plan right now is to deploy on MS Surface Go tablets [why WinForms? Because it can be run by any old Windows machine and should be able to for a long time, and if push came to shove it should run under WINE or something similar with minimal problems, hopefully lengthening the life of the software) that stores data in JSON files, with a data-reconciliation mechanism that works by the app zipping up the data and then they can transfer that file to another peer machine and integrate the data together ( Naive reconciliation model of newest data wins ), with all peers able to have the latest data at any time. There is no server component because of the context where this project works means that they cannot rely on internet access being available for up to days at a time, but they can share files via USB drives/local wifi/etc. Plus, for lots of reasons I don't want them to have to rely on me for a working system. If they have the app, they can share data.

I have looked at CRDTs, but doubtful I want to adopt them for several reasons - 1) No good open-source implementation in C# that I know of, 2) The current app is hopefully fairly simple, code-wise, and just does 'newest record wins' data reconciliation, which is easy to explain and easy for a user with minimal technical knowledge to internalize, 3) Medical data really should be repeatable and deterministic, and I have not seen good answers on how to record and display the decisions that the CRDT made.

P.S. if this project sounds interesting to anyone with C# experience who would be interested doing code review of this app, please comment and I can add you to the GitHub repo (it's private right now, as it's not quite finished and I want to have some documentation around it so that others could use it, before turning it to public). I'm not a C# dev (VB6/Mobile/web), and the app would benefit from someone with C# experience pointing out all my dumb mistakes.

cxr• 7 years ago

This sounds well-intentioned, but here's a suggestion that I really hope you consider: do the simplest possible thing and make your app spit out a dumb HTML file that represents the patient's dead tree file (where the output is either in a fixed form, or it's marked up with something like microformats). Your app should operate on that, and not JSON. Now the rule becomes, "Even if they don't have the app, they can share the data".

jefurii• 7 years ago

Sounds like an interesting system! I'm working on a digital archiving system that is conceptually similar except that I also have to manage binary files (and also doesn't have to store critical patient data). The system stores metadata as JSON files in Git repositories, with binaries managed by git-annex.

Collaborative Software That’s Wary of the Cloud

Related Stories