Automerge: a new foundation for collaboration software [video]

https://www.youtube.com/watch?v=Qytg0Ibet2E

by mr_tyzik • 5 years ago

68 29 5 years ago

youtube.com

thunderbong• 5 years ago

Relevant github link:

https://github.com/automerge/automerge

aardvark179• 5 years ago

Thanks for that. I wish they were a little more honest at the top when they claim there are no merge conflicts. Yes, they can automatically merge, but some cases will inevitably produce conflict objects and you are going to have to handle them in some way, or indicate to users that some of their edits have been lost.

The downside of anything like this is that for it to work well everything change must go through this framework because the fine grained change information is needed to effectively replay changes. That can make it hard to use existing domain libraries which might require a round trip data conversion as it may not be possible to differentiate between an insertion plus a deletion versus a complex modification.

yuchi• 5 years ago

I really love Automerge, had some fun in the last months with building some small experiment. Some major problem I see currently is the lacking ecosystem.

I had to roll my own persistence implementation on top of IndexedDB and I still haven’t touched the backend synchronization part.

I would love to see Y.js or Automerge more throughly supported by a wider ecosystem of elements that make building a production app easier.

jamil7• 5 years ago

I agree but they do seem quite focused and have made a lot of headway over the last year or so when I first started following the project. I think switching to a rust backend was the right move and will open the project up to a lot more platforms than other implementations.

adeptima5• 5 years ago

Looks like both Y.js or Automerge communities are busy with benchmarks

mlajtos2• 5 years ago

Y.js has pretty good integrations with editors. What would you wanted to see in particular?

BTW Rust port of Y.js might be start of something rather wild. :)

outside12342• 5 years ago

Are you looking for https://github.com/rust-crdt/rust-crdt and https://github.com/libp2p/rust-libp2p?

adeptima5• 5 years ago

How Yjs works from the inside out https://www.youtube.com/watch?v=0l5XgnQ6rB4

adeptima5• 5 years ago

Main takeaways from toying with both Yjs and Automerge:

1. Extremely difficult to build backend in other programming languages than Nodejs

Rust implementations https://github.com/automerge/automerge-rs https://github.com/yjs/y-crdt

You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs

C-binding, FFI, etc

2. Both communities are great. Before committing recommend to go through issues and discussions

Yjs forum https://discuss.yjs.dev/

Automerge Slack https://join.slack.com/t/automerge/shared_invite/zt-e4p3760n...

3. Watch out implementations of underline libraries. Trace lib0 libraries usage and internals in Yjs for example

https://github.com/yjs/yjs/blob/7bd764fba73fecb8bacd1eb82a4b... https://github.com/dmonad/lib0

JavaScript engines use UTF-16 encoding. Golang (my main backend language) is using UTF-8 ... reimplementing Yjs code in Golang with algorithms and optimization and futher scaling might become impossible for small startups.

Build parallels with Automerge

4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines

BitPhinix seems making a progress on updated example https://github.com/BitPhinix/slate-yjs/tree/next

However there is a surprise waiting for you on backend https://github.com/BitPhinix/slate-yjs/blob/next/examples/ba...

See "@hocuspocus/server" https://tiptap.dev/hocuspocus/

5. There's ProseMirror editor for collaborative editing. However you might not like its internals compare to Slatejs (which is far from perfect and lot of people rely on hard forks)

Feel free to ping me on Telegram @reactima if you are committed to build something similar to Notion.so or thinking to add collaborating editing with Slatejs and Reactjs. I'm really curious how other people plans to live with Yjs and Automerge impressive foundation.

hanspagel• 5 years ago

I’m a bit biased, but I think 2022 will be the year of Y.js.

What we’ve done or set up for the next year:

* The Y-Collective to fund more and more related projects, like slate-yjs: https://opencollective.com/y-collective

* The Hocuspocus backend you've discovered, which will become public in April or so

* A cloud offering we’re bootstrapping to hopefully get even more money into the ecosystem

* Y.js core support for the Tiptap editor (900k downloads/month), with a lot more advanced features coming next year

* More and more frontend libraries, like SyncedStore which was on the frontpage a few days ago: https://syncedstore.org/docs/

* The Y.js Rust port: https://github.com/yjs/y-crdt

Exciting times!

mlajtos2• 5 years ago

CRDTs (and Y.js especially) will deliver what Google Wave promised. Not as a single unifying UI for communication, rather all existing apps can get collaborative power where it makes sense. And that is a game changer for me – as a user and developer. Exciting times indeed!

melony• 5 years ago

How do the performance and features of Yjs compare to Automerge?

samwillis2• 5 years ago

I agree with Hans, we are going to see an explosion of collaborative tools next year. And Yjs is best placed to enable them.

The ecosystem around Yjs with the Y-collective is so impressive.

adeptima5• 5 years ago

I’m biased into Yjs too and bet on it. Yjs Rust port and other simply not ready for the prime and won’t be for a while. Bindings and unsafe Rust is the road to hell

bschne• 5 years ago

4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines

They (Ink & Switch) are actually working on that too, see https://www.inkandswitch.com/peritext/

outside12342• 5 years ago

Wow - you weren't kidding - the rust port is total trash. What is even going on there?

Horusiath• 5 years ago

I'm one of the maintainers of this project: we're happy to hear some constructive feedback and proposals for improvements, so please don't hesitate to speak your mind at github issues: https://github.com/yjs/y-crdt/issues .

The internals are still very hot and in a state of flux, as we 1st decided to go with porting the Yjs, then leave cleaning and optimizations for 2nd step after we have something, that's compatible with existing Yjs behavior.

caladin• 5 years ago

You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs

Am I missing something? You linked a file that is inherently going to be "un-Rusty" because it's meant to be an FFI shim for _non-Rust_ languages to use through a common C API that this file exposes.

aogaili4• 5 years ago

MeteorJS solved this 12 years ago...but good to see more thoughts being put into this.

ImprobableTruth2• 5 years ago

The key feature of automerge is that it's decentralized like git. Two people could work asynchronously and only occasionally merge their results.

jpgvm2• 5 years ago

It did not at all. It solved the problem of how to reflect a source of truth (the server) in somewhat real-time on the client through subscriptions. Also made some attempt to have edits that were persisted to client-side mini-mongo pushed to the server but this was best effort at most.

CRDTs are on a completely different level. They provide deterministic conflict resolution strategies through either/or causal and physical timestamps to ensure it's always safe to persist a write locally and later sync it with the server or not even have a server and sync directly which other peers.

aogaili4• 5 years ago

I understand the difference.

However, the local mini-mongo is acting a local-state and the JSON/JS all the way is solving the marshalling/ORM issue. I understand it is a centralized way of dealing with this, however, it was operating on the same problem space. Furthermore, automerge can be used as the server, which bring it closer to what Meteor did. I was hinting at the overlap in the problem space, but I should have phrased it better. Anyway, I'm keep to try automerge.

aogaili4• 5 years ago

I get it, it is different Thanks for correction.

ImprobableTruth2• 5 years ago

One sort of issue is that this only directly supports documents/DAGs, so representing many-to-many relationships is a bit iffy. This is similar to how Rust can't directly represent cyclic data structures.

It's not a huge deal, because working with indices works well enough, but it is a bit annoying.

pictur• 5 years ago

what problem does this solve?

Automerge: a new foundation for collaboration software [video]

Related Stories