The building blocks of offline support

by pketh • 3 years ago

83 30 3 years ago

tmikaeld• 2 years ago

This is really impressive and I like the app, it's fast and powerful.

However, I wouldn't consider using it without better undo, if i merge a set of cards by mistake it cannot be undone. Maybe store a larger undo history in IndexedDB?

pketh7• 2 years ago

While there is undo, you're right, it's pretty rough right now. That might be a good way for me to dip my toes into indexedDB

tussa3• 2 years ago

It conveniently doesn't mention the hardest part: conflicts.

If you just drop a conflicting edit then it's a stretch to call your app "offline". Yes, it "works" but who wants to use an application to at drops you edits?

halfcat2• 2 years ago

Conflicts themselves are not hard: Keep a directed acyclic graph of immutable records. Changes to a record point to the parent/prior record. Two users update the same record, now you have a tree.

The challenge is interpreting what that tree structure should mean.

If you can, let a user decide how to resolve the conflict.

- User logs in, they have a “conflict inbox” of things that need to be resolved.

- Two coworkers make conflicting edits, maybe the manager gets a notification in their conflict inbox and they decide

tussa3• 2 years ago

If we want to discuss terminology, I agree I should have said "handling conflicts" instead of just "conflicts".

What you are describing is just the beginning of conflict handling. The consequences for bad handling are dire: data loss. If conflict handling and resolving was easy (hint: it's not), the article would have mentioned it.

matharmin• 2 years ago

Conceptually that's not hard, but in practice an approach like that can significantly increase the complexity of the app:

1. Do you store the tree structure for every table in your app? If you have 20 tables that could be edited offline, do you re-implement it for each table, or try to have a generic implementation? 2. Do you design all your tables around the tree structure, or do you just store it in addition to your "normal" tables? 3. Every piece of code that modifies one of these tables need to do it via the tree structure - if you update your tables directly from any place it could effectively cause conflicts. 4. Do you build separate UI to resolve conflicts for every table? 5. Do you query and cache the tree structure on the device, or does it have to be online to resolve conflicts? 6. Do you expose the tree structure via external APIs, or keep it internal?

I find that "last-write-wins" is sufficient for a large percentage of cases, and much simpler to implement. Or in some cases, just doing conflict detection is sufficient (notify the user that the data has changed between loading and saving, and they need to re-apply their edits).

If you do need conflict resolution on a large scale (many different tables), I'd recommend using data structures designed for that. CRDTs is one example - while it is typically used for automatic conflict resolution, it often stores enough data to allow manual resolution if desired.

gerdemb2• 2 years ago

I created an experimental Swift package that implemented this architecture using SQLite.

https://github.com/gerdemb/SQLiteChangesetSync

As you noted, detecting conflicts is easy, but handling them is not.

pketh7• 2 years ago

The scenarios where people are simultaneously editing the same space online and offline are very rare. When both users are online, I use the UI to communicate who's editing what to avoid conflicts. I would like to improve this in the future, but for now it's a utilitarian scope/resource issue

JamesSwift6• 2 years ago

Conflict resolution and the inability to come to a business decision on what to do has stopped offline-first support in a couple apps I've done. Its a really messy conversation to have, especially with people that aren't used to sweating the details. You can't just handwave away the complexity!

tussa3• 2 years ago

I've seen business software, where money can be lost, just handwave and silently dropping conflicts (because "it doesn't really happen that often!")

pketh7• 2 years ago

perfect is the enemy of the good, and perfect can be iterated towards as demand scenarios increase

emacsen• 2 years ago

I used to think this a lot, but then I find products that do exactly this and work reliably most of the time by simply asking you "Local Version" or "Server Version". This is what Steam does, what Nextcloud does, what PS5 does, and probably others. It's a naive approach but it seems to work well enough.

Beyond that, you get into complex territory but maybe we've all been overthinking the problem space.

JamesSwift6• 2 years ago

It only works for certain classes of products. A game has decidedly few copies (will just be your own saves across multiple devices), and is already serializing in a singular high-level format. When you have business software that is used my multiple people, and has multiple different 'kinds' of things that are being reconciled (at varying levels of granularity), it gets really messy and really hard to hit the sweet spot.

woodrowbarlow• 2 years ago

if you can encode your state in CRDTs, there are no conflicts to resolve.

JamesSwift6• 2 years ago

Thats only solving the conflicts at a technical level. The hard problem to solve is what conflict-resolution means at a business level. The even harder problem to solve is auditing and coming up with an appropriate answer for _every single entity in the system_.

For most products, there isnt a one-size-fits-all answer to how to handle conflicts.

jiehong• 2 years ago

That first load time was pretty long, though (like 4 seconds).

Felt as if it checked online or loaded too many things.

pketh7• 2 years ago

that's def not ideal and not normal, loading up the page in incognito takes me less than a second. It does make a lot of requests for assets and to the API but those are mostly async. Can you repro the long load time everytime? Do you still see a long load time for a space url like https://kinopio.club/fonts-6n5memHYV0K1qmeUcX2Fs ? If the issue persists? What browser/OS are you using? Where in the world are you? (the client should be serving over a CDN)

EDIT: in case you were referring to the loading state in the vid in the post, I suspect that's a bug with the iOS loading screen that I didn't have time to fix before this post (it's just idling for most of that). will update later

JamesSwift6• 2 years ago

As someone who has built offline-first, I noticed that as well : D

All the offline-first apps I've built have felt magically fast to load. It really is an incredible experience. I think its likely something to do with parsing/loading the initial view state that is taking a bit. Thats a pretty complicated widget so likely has to load libraries to handle it.

pketh7• 2 years ago

it's an issue with the ios app only that i'll be investigating (the loading screen is mostly idling during that time). If you load the kinopio website in safari while offline (the app is just a thin wrapper over the site), then it is indeed magically fast

jamestanderson2• 2 years ago

It's really cool to see offline support done well. It can be very frustrating when it's done poorly, or not offered at all.

One of my biggest gripes with the Spotify app is the poor offline support, at least in my experience on Android. I have the bulk of my library downloaded for offline listening, so when I have a spotty network connection like when I'm on the Subway, I'd expect that I can still easily access at least my downloaded songs. Not the case. Spotify, it seems, won't use its local cache until it's thoroughly convinced you're offline, which may take several minutes of waiting for requests to time out. Once Spotify is convinced I'm offline, my downloaded songs will then finally load normally.

My guess is that instead of doing it the way the Kinopio does - by reading from the local cache before fetching the remote data - Spotify does it the other way around.

Anyway, nicely done!

paxys• 2 years ago

Yeah, the only way to get Spotify to work well offline is to manually set your phone to airplane mode. Otherwise it will endlessly try and fetch everything over the internet despite it being fully cached.

JamesSwift6• 2 years ago

I ran into this all the time when trying to listen while mowing my lawn and going into/out-of wifi range. Very frustrating UX.

latentcall• 2 years ago

I thought Spotify used to have an “offline mode” toggle in the settings a long time ago. Is that no longer the case?

jamestanderson2• 2 years ago

Huh, I never knew about this! It looks like it's still around. I'll try that next time. Thanks for the tip!

The building blocks of offline support

Related Stories