DocNode: A TypeScript OT library for local-first apps

by GermanJablo • 8 months ago

12 10 8 months ago

GermanJablo5• 8 months ago

Hi everyone! After two years of development, I’m excited to announce DocNode: a type-safe, fast, ID-based Operational Transformation (OT) framework for conflict-free collaborative editing. CRDT mode is in progress.

Along the way, I learned a ton. I rewrote the library several times. Sometimes because I was obsessed with the API, other times for technical reasons. I moved from CvRDT to CmRDT, and finally to ID-based OT.

Happy to answer questions!

pscanf2• 8 months ago

Hey German. Congrats on shipping the project!

Reading the "Why another CRDT / OT library?" I like that you seem to have taken a "Pareto approach": going for a simpler solution, even if not theoretically perfect. In the past few months I've been building a local-first app, and I've been a bit overwhelmed by the complexity of CRDTs.

The goal that I have with my app is to allow syncing between devices via Dropbox / Google Drive / iCloud or any another file-syncing server that the user is already using. I don't want to host a sync server for my users, and I don't want my users to need to self-host either.

Do you think it would be possible to use Dropbox as the sync "transport" for DocNode documents? I'm thinking: since a server is needed, one device could be designated as the server, and the others as clients. (Assuming a trusted environment with no rogue clients.)

GermanJablo5• 8 months ago

Thanks! The answer depends on what you want:

1. Do you care about resolving concurrent conflicts? That is, if two users modify the same document simultaneously (or while one is offline), is it acceptable if one of their changes is lost? If that’s not a problem, then using Dropbox or something similar is totally fine, and you don't need a CRDT or OT like DocNode at all. Technologies like Dropbox aren't designed for conflict resolution and can't be integrated with CRDT or OT protocols.

2. If you do want to resolve conflicts, you have two options. (a) Use a CRDT, which doesn’t require a server. One downside is that clients must be connected simultaneously to synchronize changes. Personally, I don’t think most people want software that behaves like that, and that’s one of the reasons I didn’t focus on building a CRDT. If you’re going to end up needing a server anyway, what’s the point? (b) Use a server, either hosted by you or by your users. The good news is that it’s extremely simple. With DocNode Sync, you can literally set it up with one line of code on the server.

That doesn't apply if you're using a CRDT with "a server as an always-present client". But in that scenario, DocNode will be more efficient.

pscanf2• 8 months ago

Thanks for the detailed answer!

I think I do want to solve conflicts. My use case is for a personal database, which simplifies things a bit: sync is between devices of a single person, so it's unlikely for concurrent offline changes to occur.

What I have in mind is a setup like the one from this experiment: https://tonsky.me/blog/crdt-filesync/ . I don't know if it's at all possible in my use case though, or–in case it is possible–if it ends up being practical. As you said, the resulting user experience might be so strange that it's not something users want.

Anyway, thanks again for the info and good luck with DocNode. :)

jv222223• 8 months ago

Hey German. I think you should re-submit this as a "Show HN: DocNode, A TypeScript OT library for local-first apps" in a few days to a week. I think it will get more of a chance titled that way.

GermanJablo5• 8 months ago

Thanks for the advice. I'll try it when the post is a week old. I hope it doesn't get rejected.

jv222223• 8 months ago

I don't think it will. I know a few folks who have done the same. It' is part of the T&C's that it is ok to re-submit just not super fast.

DocNode: A TypeScript OT library for local-first apps

Related Stories