LocalStorage or WebSQL unexpectedly cleared

https://bugs.chromium.org/p/chromium/issues/detail?id=1033655

by ssddanbrown • 7 years ago

147 74 7 years ago

bugs.chromium.org

McDev• 7 years ago

Ouch! A recent Reddit post immediately came to mind:

https://www.reddit.com/r/androiddev/comments/ea86jt/users_re...

Lex-2008• 7 years ago

looks like a big f_ckup...

From comment #26:

why it was missed: test apps that don't happen to use local storage will not have a file created :(

C14L2• 7 years ago

Shouldn't test apps test _all_ possible files?

Sounds like a case of "who's testing the tests".

Thorrez2• 7 years ago

I think in this case the test coverage metric would have been handy to find things that aren't being tested.

low_key3• 7 years ago

I was bit by this in a cordova app on a past update. After that I backed up api keys, etc to native storage using a cordova plugin and added a wrapper that will fall back to it if localstorage lookups fail.

I would advise others to do the something similar since unless you are packaging the browser too (like crosswalk did), then that piece is really out of your control and you shouldn't expect it to be retained.

A similar issue can be seen on iOS under low storage conditions where the operating system will clear the browser cache to reclaim space.

webappguy5• 7 years ago

Which plugin?

martin_bech• 7 years ago

If would guess the cordova sqlite plugin

low_key3• 7 years ago

This plugin has worked reliably on both Android and iOS and has a simple api:

https://github.com/TheCocoaProject/cordova-plugin-nativestor...

traspler• 7 years ago

I can only recommend you switch fully to SQLite or native storage. localStorage has always been unreliable and could be cleared any time on iOS.

the_gipsy4• 7 years ago

a wrapper that will fall back to it if localstorage lookups fail.

If I understand correctly, the scenario is that saved data in localStorage is lost, so that won't help.

makeramen• 7 years ago

“It” in this case is native storage, not localStorage

dmitriid4• 7 years ago

This also servers as a warning for PWAs: despite all the hype and virtues (real or perceived), your are at the hands of very fickle gods

webappguy5• 7 years ago

We all are at the fikle hands of Apple and Google sadly.

cromwellian• 7 years ago

Don't you have the same problems with native? It's like like no OS update ever broke apps, or that Apple doesn't capriciously remove APIs, or reject you from the app store sometimes for unexplained reasons.

Hell, people get broken by OSS ecosystem libraries a lot.

webappguy5• 7 years ago

Wish Apple and Google would put more browser collaboration time in, for the benefit of all..constant one upmanship hurts both and millions of man hours in bugs, pollyfills, API repairs, and on and on. Webview, WKWebview teams need more rigorous standards and safety checks clearly for consistency and reliability. Progressive development benefits both. Web-apps are a critical, large and important part of the app developer ecosystem.

dmitriid4• 7 years ago

Yup. In the end you are at the mercy of whoever controls the platform and its development.

tsimionescu2• 7 years ago

True to some extent, though I don't think there have been many OS updates which deleted all storage for all apps (well, moved to an inaccessible place)... This really is a massive failure of QA and dev.

tehlike• 7 years ago

That's cynical. You must have your own bugs every now and then. This is no different.

PS: google employee, used to work with torne every now and then, but no stake in the conversation anymore.

dmitriid4• 7 years ago

Cynical? Yes. Is that the state of the world? Yes.

Browsers are platforms that are as complex as OSes these days. And any bugs or changes in how they work will affect you app if your app is PWA.

- It could be Chrome breaking audio: https://danshumway.com/blog/chrome-autoplay/

- It could be Chrome breaking LocalStorage: this discussion

- It could be Chrome removing support for a web standard that was implemented and shipped: https://www.chromestatus.com/feature/4642138092470272

I'm linking Chrome issues here because Chrome is the most visible of all browsers with the largest marketshare, but other browsers have their issues, too.

C14L2• 7 years ago

That's not just true for PWAs.

That's why offsite data backups are a really good thing.

anticristi• 7 years ago

Indeed, I don't mean to downplay the frustration of the developers and their customers. However, valuable data should really not be kept on a single mobile device, which is prone to damage, loss or stealing.

the_gipsy4• 7 years ago

Wrong. Relying on being allowed on some appstore is much more like being "at the hands of very fickle gods". Pray that apple/google don't suddenly change their mind and lock you out, because it competes or goes against whatever new trend they adopt this year.

swiley2• 7 years ago

On sane OSes native is nice.

On most consumer OSes I honestly would trust the chrome developers to be more well behaved.

WA4• 7 years ago

Chrome had a similar fuckup in April 2018. They changed the minimum granted quota for IndexedDb. Suddenly, all Android devices with low free storage space (< 1GB roughly) had a granted quota of 0. This of course led to all Android WebView apps relying on IndexedDb to have no access to existing stored items and no way to save new data.

It took them 6 weeks to push a fix. I used the opportunity to switch to Sqlite and can only recommend others the same.

webappguy5• 7 years ago

Ftw SQLite primary with Indexed DB and LokiJS wrapper for in-mem.

EGreg2• 7 years ago

How did sqlite help in this case? We are talking about the Web.

WA4• 7 years ago

We are talking about WebView apps, i. e. Cordova-based with SQLite plugin.

apatheticonion2• 7 years ago

This actually hit me really hard. I made a PWA I used to track certain stats but the data wasn't so important that it needed to be stored in a database.

Instead I had an import/export for manual backups and assumed that the data inside indexedDb wouldn't be dropped.

Bad assumption.

WA4• 7 years ago

Yes, same for me. Also a bad assumption I made: People probably have enough storage space. The number of people running around with older devices that have 16 GB of internal storage is still like 5% or so. Add 32 GB devices, probably 10%. And most of them are constantly full.

Btw. the same is true of older iOS devices. They are constantly almost out of space and there is not enough free space to update to the most recent iOS version. It'd probably work with iTunes, but many people don't install iTunes. So they're stuck with iOS 10, although their device would still be capable of running iOS 13, simply because they don't have enough free space.

jsnell• 7 years ago

Just how fast are Chrome rollouts on Android? Given how destructive this appears to have been, it seems amazing that they got to a 50% rollout before the first bug report.

webappguy5• 7 years ago

Blaming people for not using beta version for six weeks to test and inform them prior of the issue. Rich. Millions of apps affected.

swsieber• 7 years ago

That seems like an uncharitbale reading. A manager let people know that they'll be able to tell if a new chrome release will affect them by testing with beta. The overwhelming sentiment was "we're sorry".

ken• 7 years ago

When Google introduced Chrome in 2008 (via comic!), they claimed that within 20-30 minutes of a beta build, it was automatically tested on tens of thousands of the most popular webpages. Is that no longer true? Does that not include any which use local storage?

orf• 7 years ago

Not to downplay the issue, but what a ridiculous bug report:

It causes such a big problem for our products!!!! Who is in charge?

I'd have expected more from a tech focused company like pocketpos.

beart• 7 years ago

This could be an exhausted Customer Service manager. They tend to be a lot less technical.

oauea• 7 years ago

In general, app developers are of surprisingly low quality

To any app developers reading this, caring enough to read HN or similar puts you leagues above the kind of people I'm talking about.

kbutler• 7 years ago

Apple taught this lesson a few years ago, when iOS 5.1 changed the local storage location: https://phonegap.com/blog/2012/04/18/ios-5-1-and-the-embedde...

But this behavior is within the range of spec compliance: "User agents may, possibly in a manner configured by the user, automatically delete stored data after a period of time." https://www.w3.org/TR/webdatabase/#privacy https://www.w3.org/TR/webstorage/#user-tracking

So these browser storage mechanisms must be considered cache, not primary storage of important data.

finaliteration• 7 years ago

So these browser storage mechanisms must be considered cache, not primary storage of important data.

Maybe I’m an exception, but I’ve always viewed these storage mechanisms as unreliable. I primarily use them for storing things like user filter settings on a list view. It never even occurred to me to use them for anything critical.

r721• 7 years ago

Real life consequences (from comment #63):

I heard from a company that uses localstorage for offline no-connection available that had local record of animals getting vaccionation. The update "erased" all the data. They don't know which animals got vaccine and can't repeat on all of them. Serious stuff.

https://bugs.chromium.org/p/chromium/issues/detail?id=103365...

tcd• 7 years ago

You should apply a rule similar to 3-2-1:

3 copies of the data

2 copies on different storage mediums

1 backup in a separate location.

Ideally, a backup solution on the OS would target the browser profile itself at regular intervals, and you could allow the user to export the data into a file that can be downloaded by the browser (or copy/pasted).

It should be obvious that any form of browser storage is unreliable, and MUST be treated as such, and it should be EXPECTED to be deleted at a moments notice - setting such expectations encourages good practice at backing up your data incase of a failure.

Don't rely on others to ensure business critical data is stored and accessible - that's partially on the users/application to offer what they can.

the_gipsy4• 7 years ago

Sounds like it's their fault for not backing up such important information.

jessaustin3• 7 years ago

"offline no-connection available"

Probably the veterinary work is not taking place at Starbucks...

megous• 7 years ago

Who uses localStorage for more than a local cache, or a short lived log of records to be synced somewhere into some serious database asap?

That people are storing/losing:

- years of financial data

- animal vaccination records

- etc.

because of this is so weird.

tracker1• 7 years ago

Even then, it can still be used for offline data... you collect/perform tasks without an internet connection and it syncs later. Chrome updates before data sync, boom, all your data is gone.

Now, as for years of data, I mostly agree, this should absolutely be backed up and/or synced more regularly.

curryst• 7 years ago

That still seems like a poor design. This would also happen if you cleared localStorage before the data is synced. If, for example, when you came back online and were trying to do something on an unrelated site, hit a bug and tech support asked you to clear your browser.

For something as critical as financial records or vaccine records, the sensible thing might be to not work in offline mode, or to persist to native storage.

I think it's fine as a cache for offline data for a lot of use cases (i.e. task trackers, note apps, shopping lists, etc). But it creates an unacceptable risk for business critical data, imo.

Exuma• 7 years ago

Storing financial data in localstorage. Lol how stupid can people be

swiley2• 7 years ago

Developer writes POC/Demo/quick hack without the necessary warnings.

Much later a user (or less aware developer) finds the thing and starts using it.

Maybe software needs GHS labels like chemicals have.

kmlx6• 7 years ago

i still don't understand the core issue: why is anyone saving essential data to localstorage? isn't that supposed to be a state-related, somewhat temporary (due to its nature) storage in-between the UI and the API/database? don't you lose all your data when you use a different pc or smartphone?

i see a lot of mentions of cordova apps. are people publishing apps without a backend (and save everything in localstorage)?

i've worked on pwa's in the past but server sync was always a priority in my algorithms.

does anyone have any realistic use cases where localstorage is so essential that clearing it would somehow lose tons of data?

capableweb• 7 years ago

Depends on the context. I've built Cordova/Phonegap apps in the past where you want things to work offline and sync once the user goes online. If the time from user going offline to online again is too long / device storage too little, that data that would eventually need to be synced, would be lost.

Always ways of working around it, like saving to files or whatever, but localStorage is a nice interface that works both for Cordova and client-side web apps.

kmlx6• 7 years ago

agree. for the use case that you mentioned there would be a problem.

but as you said, it could be overcome by using native apis that store state. it's even imperative in case of secrets, like access tokens and the like.

for client side web apps there's a bigger problem with localstorage. from my experience end users switch browsers and platforms. and they expect consistency from their web sites/apps. this expectation is much more intense than for native apps. so, while practical for developers, i'm not sure it's that welcomed in web land except for always having a local backup or as a temporary storage to alleviate network issues.

zmzrr2• 7 years ago

i see a lot of mentions of cordova apps. are people publishing apps without a backend (and save everything in localstorage)?

Yes. Why not? You can write a Cordova app that doesn't even have a backend.

kmlx6• 7 years ago

of course you can.

but since every device is as persistent as its weakest point (e.g. physical damage, loss/theft, etc) shouldn't apps that need to save essential/important data basically give up the argument of persistence because of using localstorage in the first place? your data is one breakdown/loss away from permanent destruction when using localstorage exclusively.

that is why, from my experience, you always rely on the server and only use localstorage as a backup in case the server is down, there's no internet, etc.

claiming massive loss of data because localstorage got emptied is basically saying "our app wasn't designed correctly".

what am i missing?

apatheticonion2• 7 years ago

I wrote a PWA which functioned as a calories counter and exercise tracker.

I stored the data on the client using indexedDB, offering an export/import option for manual backups.

I wasn't interested in the data and it's simpler to just store it on the client rather than setting up the infrastructure required to centrally persist this information.

It's really unfortunate that it's not a reliable storage medium.

gnaritas• 7 years ago

It's really unfortunate that it's not a reliable storage medium.

That's not what it's for, you're just misusing the tool for something it isn't meant to be. It's not a database, it's a cache.

londons_explore• 7 years ago

In today's GDPR world, a lot of companies want to see as little of the users private data as possible.

If their app can avoid sending any data to a server and run fully in a web browser offline, they'll do that.

erikrothoff• 7 years ago

This has been happening in Chrome extensions for years now. I’ve tried reporting it so many times. In our case it happens randomly so it’s impossible to reproduce, which inevitably leads to ”issue closed - couldn’t reproduce”.

LocalStorage or WebSQL unexpectedly cleared

Related Stories