What It Takes to Truly Delete Data

https://fivethirtyeight.com/features/what-it-takes-to-truly-delete-data/

by spaceboy • 9 years ago

48 32 9 years ago

Pfhreak• 9 years ago

It's difficult to teach this lesson to people.

It was difficult for me to learn this lesson personally. When I understood it, I deleted my Facebook account (best decision, btw), but I assume that even though my account was deleted there are still references to it all over. IDs in log lines, references not cleaned up -- enough to paint something of a picture of who I was and what I did with the site.

Now I'm more judicious in what I share, how I share it, which applications I give permissions. But I'm still leaking metadata everywhere. It's frustrating and terrifying that the cost of participating in our society is either a HUGE cognitive load in keeping an eye on all the data you emit or accepting that basically everyone will have full access to your (meta)data over time.

conductr3• 9 years ago

The society part is tough and frustrating. I get so tired of arguing with clerks that want my email, zip code, phone #, etc. I don't want to give everyone that, and it really slows down the checkout process in general. Just take my damn money.

A few weeks ago, I walked away from a purchase because the store wanted to take my photo at POS. This was my first encounter with the store, I probably would not be a repeat customer, I just wanted to do a single purchase. Just take my damn money.

The stupid part is, I typically use the same credit card for every purchase. I know they could use that to query info product and know everything about me - and, I don't really care about that (I've given in to it). It's the constant asking for info by companies that I have no ongoing relationship with that I really hate.

Edit: Also, it makes me feel like some weirdo freak when I don't want to participate. It starts to cause a scene. Other customers are staring. My wife is embarrassed. Everyone is thinking, "just let them take the photo" or "the clerk is just doing their job".

x1798DE• 9 years ago

WTF store wants to take your photo just to sell you something? I can't possibly imagine submitting to that unless it was for something amazing or something where the photography was obviously necessary to use the product.

enigmango• 9 years ago

Wow, this is the first I've heard of photos being taken at a register. Do you mind sharing what store this was, or if it was triggered by the purchase of a specific item?

I realize this request to reveal where you shopped, or what item you bought, is in response to your objection to being tracked, so I understand if you'd rather not say :)

billmalarky• 9 years ago

Everyone is thinking, "just let them take the photo" or "the clerk is just doing their job".

Maybe, or maybe you made them second guess their own personal policy of what they are comfortable sharing. So long as you are confident I don't think a reasonable person would look down on you for protecting your privacy. I usually just say something like "I don't do that" when asked to share email/phone/zip with stores, never had a clerk get weird about it.

gowld3• 9 years ago

Vendors are prohibited by contract from using purchase information tied to your credit card. That's why they have store-specific loyalty cards.

lend000• 9 years ago

It's impossible to be "relevant" without giving away your privacy these days. Doesn't help that there are dozens, if not hundreds of parties with supposedly private information like your SS number, and the odds that all of them (or even a significant proportion of them) have strong data security measures are next to nil.

TeMPOraL3• 9 years ago

There are also some other things I find people have trouble understanding.

First thing that one needs to understand: you always leak data. It's a feature of reality itself. At the very least, every second of your life, the record of where you are and what you do radiates away from you at the speed of light. Of course interacting with matter, you leave stronger traces everywhere. Mechanical, thermal, accoustic. Most of the data isn't easily accessible in any meaningful way, but it doesn't change the fact it's always there.

Two: progress of technology increases our ability to access this data. There's no way to ensure that data is denied to all people other than stopping the development of new technologies. There's no meaningful way of preventing anyone from accessing the data that's available today other than rolling back the Industrial Revolution and going back to living in caves. Science and technology lets us see the physical world in more details, faster and cheaper, and it also lets us correlate information faster and cheaper. There's no way around that.

So that's about "metadata". As for "personally identifiable information", I may avoid giving some of it to random strangers, but I do assume that PII is public information. Because, again, there's no meaningful way to contain their spread. So pretending to yourself that any PII is private would be stupid.

Retric2• 9 years ago

There is a contradiction in your line of thinking between:

data isn't easily accessible in any meaningful way

and

no meaningful way to contain their spread

If A is not readable then there is no need to contain it.

stinkytaco• 9 years ago

I think that the issue is not that we leak data, but that in the last 10 years, it's exponentially more accessible. There was a time when finding out who I interacted with would have required many man hours and probably some court orders. Now a carelessly adding a friend on Facebook can do so. Even if I choose not to do business with large companies, so many other people can leak my data (pictures on Facebook, emails on Google servers, etc). And, of course, the ability to algorithmicly correlate it. The only answer here is strong legal protections across borders, and I'm afraid I don't see that on the horizon.

bko• 9 years ago

That computer, armed with the company’s proprietary wiping software, uses a special algorithm to programmatically access each bit on the drive, stored as either a ‘0’ or ‘1,’ and write a new digit to it. The process must be repeated at least three times and with different numbers to be up to DoD standards.

Maybe a stupid question, but why don't they just write every bit to either 1 or 0 regardless of its current content? Are they trying to preserve some data and not others? Why do you need to do it multiple times if the bit being written is independent of the actual contents?

gizmo686• 9 years ago

Why do you need to do it multiple times if the bit being written is independent of the actual contents?

There is a concern that the write process is not perfect, so the old value of the bit might still be visible under a microscope.

cr0sh• 9 years ago

It's not because "the bit might still be visible under a microscope" - it's because of something called (IIRC) "intra-track data".

Basically when a hard drive stores the data on the physical platter, it writes it as a series of concentric tracks. These tracks are spaced very close together. When data is written to the drive, the magnetic regions that are written on the track can "bleed over" to the intra-track regions, leaving a faint trace that with the proper equipment ($$$$) can be read, even if the tracks are all zeroed. Think of the magnetic regions as not being (say) "spikes", but rather "hills", where the greatest magnetic intensity is at the peak, tapering off as it reaches the edge/base of the hill. That energy might be small, but it is there.

By overwriting each track multiple times with varying data patterns, these leftover tale-tell magnetic leftovers are essentially pushed into the noise of the rest of the data, and are effectively "wiped". But you have to write the pattern multiple times, over all the tracks.

This effect has "recently" been used to an advantage (depending on how you look at it) for something called an "SMR Hard Drive" - or "Shingled Magnetic Recording":

https://en.wikipedia.org/wiki/Shingled_magnetic_recording

In this kind of hard drive, the tracks are purposefully placed very close together so that the magnetic regions purposefully overlap, allowing for a higher density on fewer platters (so, larger drives in a smaller space for a lower cost). However, they tend to be slower - and not recommended for purposes like RAID NAS or anything where you want speed and proper data retention (my understanding, at least).

adrianN2• 9 years ago

Drives store analog information and only present it in digital form to you. So a 1 might be a 0.8 and a 0 might be a 0.2. Writing all ones might turn bits that already were "1" to 0.9, while leaving bits that used to be "0" at 0.8.

0x0• 9 years ago

It is generally accepted that it is impossible to recover data from a single wipe on modern hard drives. There is some mention of it here https://en.wikipedia.org/wiki/Data_erasure#Number_of_overwri... and also I believe I read somewhere that there's unclaimed prizes available for anyone who can prove otherwise (but I do not have a source available for that now).

What can bite you is the drive firmware silently remapping sectors, causing your write request to be mapped to a different sector altogether.

yaur• 9 years ago

There was a (possibly mistaken) belief that some data could be recovered with an electron microscope. AFAIK it is definitively not true with modern spinning drives and its the "AFAIK" portion of that statement that keeps people doing multiple wipes.

anfractuosity• 9 years ago

I think you might mean something like a Magnetic Force Microscopy.

Edit: It looks like there is something called lorentz microscopy that uses an electron beam.

gowld3• 9 years ago

To make sure every bit is overwritten, they need to change all the 0s to 1s and all the 1s to 0s ;-)

I'm also curious what's the third number they found in {0,1}

Casseres• 9 years ago

new ≠ different

rads3• 9 years ago

If @realDonaldTrump deletes a tweet, it's against the Twitter ToS to republish it using a bot.

When it does happen I've seen people post screenshots of the old tweet. This doesn't prove anything more than trivial knowledge of the Chrome Web Inspector.

I've thought about distributing a program so people can keep their own local archives of important tweets. There are plenty of ways to achieve this already, but that's besides my point.

How can we make it obvious when an important figure deletes a tweet without breaking Twitter's ToS?

throwaway2016a• 9 years ago

doesn't prove anything more than trivial knowledge of the Chrome Web Inspector

That might be generous. They could have used Photoshop as well and had no technical knowledge at all.

grenoire• 9 years ago

Inspector makes it practically impossible to prove wrong. You literally cannot get the font, positioning, kerning, or spacing wrong with the replaced text.

adrianN2• 9 years ago

Don't use Twitter to publish that information?

rads3• 9 years ago

The issue is with consuming the tweets. Publishing the data somewhere else has the same problems as publishing on Twitter.

1. The Twitter ToS prevents scraping the content in section 4.iii.

https://twitter.com/tos?lang=en

2. If you use the API instead, they have this rule:

Only surface Twitter activity as it surfaced on Twitter. For example, your Service should execute the unlike and delete actions by removing all relevant Content, not by publicly displaying to other users that the Tweet is no longer liked or has been deleted.

https://dev.twitter.com/overview/terms/agreement-and-policy#...

gowld3• 9 years ago

Twitter can falsify tweets too. At the bottom of it all, all anyone has is their own memories and records (both can be tampered!) and whatever trust they place in other people's records. That's the history of "escrow".

rads3• 9 years ago

I've found the answer to my own question:

https://projects.propublica.org/politwoops/user/realDonaldTr...

Back story:

https://techcrunch.com/2016/02/09/u-s-version-of-politwoops-...

zokier• 9 years ago

While its still no silver bullet, encryption can make the problem more manageable. For example for hard drives with full-disk encryption you should be able to just throw away the key and the data should become inaccessible. Same applies to for cloud backups; you don't know what the cloud provider might do with the data you upload, but they should not be able to access the content if it is properly encrypted, and as such the issues raised in the article become lesser.

Of course this does not apply to data that is actually being used/processed in cloud instead of just being passively stored.

What It Takes to Truly Delete Data

Related Stories