Facebook’s Blu-Ray Powered Cold Storage Data Center

http://datacenterfrontier.com/inside-facebooks-blu-ray-cold-storage-data-center/

by 1SockChuck • 11 years ago

40 44 11 years ago

datacenterfrontier.com

y0ghur7_xxx• 11 years ago

All our chat history is there

CodingGuy• 11 years ago

And our cat images! :)

chinathrow4• 11 years ago

I wonder if they had the balls to store deleted conversations as well.

hsshah2• 11 years ago

I am glad companies like Facebook is using Blue-Ray. Hopefully this will trigger progress in increasing density of optical discs and/or decreasing cost of high capacity discs and writers for consumer market.

Eager to archive my yearly ever increasing personal media (thanks to having a kid) onto a high capacity disc and just store it in the safety deposit box.

harryf• 11 years ago

Awesome that all our dick pics are being stored forever. Future generations should not miss out.

bitJericho• 11 years ago

I don't know what the life of blu-rays are but burned optical cds and dvds have an estimated lifespan of only 10 years. Make sure you continue to keep up your backups! How I do it is I update my backups every few years but I also keep everything on functional and "hot" hard drives at all times (so I can see it's all still there).

There's also http://www.mdisc.com/what-is-mdisc/ but I've never used/seen it.

dvdkhlng• 11 years ago

The common Blu-Ray media (BD-R, not BD-R LTH) discs use a non-organic dye and should be more resilient than DVD-R or CD-R media. According to this source [1] 100-150 years data retention.

Edit: note also, BD-R writers immediately read back and check written data, re-writing damaged sectors to a spare area. This in-hardware defect management should make BD-R more resilient against data corruption than DVD-R or CD-R [2].

[1] https://superuser.com/questions/251369/what-is-the-lifespan-...

[2] http://fy.chalmers.se/~appro/linux/DVD+RW/Blu-ray/

gambiting• 11 years ago

CDs and DVDs have horrible shelf life because they all use organic dyes. Some Blu-Rays do too, but you can still buy Blu-Rays with metallic dyes which should work for decades(panasonic BD-Rs are guaranteed for 50 years).

In fact, do you remember the very first, early writable CDs? They all were blue on the bottom - that's because they used a metallic dye as well, I imagine 99% of those discs will be perfectly readable today.

slantyyz2• 11 years ago

The main downside is that MDiscs have a max capacity is 25GB on the Blu-ray.

I have some myself, but I can't attest to the life span, since I've only had my BDR for about a year now. How does a normal person prove the lifespan claims of an MDiscs other than waiting?

I still find myself burning most backups to 50GB disks, and making redundant copies of only my most important stuff to the 25GB MDiscs.

Where I live, a 3 pack of 25GB MDiscs runs around $20 vs $70 for a spindle of 25 RiData 50GB discs. That's ~6.66/25GB MDisc vs ~2.80/50GB BDR. Not a huge premium for piece of mind, I guess.

grinich2• 11 years ago

A few months ago, Frank Frankovsky left Facebook to start a startup based on this tech! He was previously "VP of Hardware Design & Supply Chain Optimization" and has also been super involved in the Open Compute Project.

I'm pretty excited to see what they produce, and hope they can ship before someone like Amazon tries to acquire them. This level of cold storage is going to be critical as the world generates exponentially more data every year. He also mentioned this is an open source venture, which is doubly awesome. :)

https://www.facebook.com/frankovsky/posts/10203257723264639?...

x5n12• 11 years ago

There is a reason that deletion was invented. Everything is not meant to be stored. This is a stupid idea whose time I guess has come.

grhmc• 11 years ago

Go delete all your photos then.

lucd• 11 years ago

FTA: Last month that startup, Optical Archive, was acquired by Sony. http://www.datacenterknowledge.com/archives/2015/05/27/sony-...

grinich2• 11 years ago

...sigh :(

sschueller• 11 years ago

Isn't the life span of a burned disc quite bad?

rogeryu• 11 years ago

Look at mdisc - it should last a lifetime and longer.

kalleboo• 11 years ago

Isn't the life span of a magnetic disc quite bad?

slantyyz2• 11 years ago

I've gone through over a half dozen Seagate 2.5 drives (spindle and hybrid) and one 3.5 external in the past five years, and I'm lucky if they last 2 years. This includes drives that are mostly offline.

I've switched to WD, but I haven't had them long enough to say that they're only better. The WD Green in my Time Capsule has lasted a few years though.

All these drive failures led me to buy a BD-R burner, which has worked out fairly well, although I would say it's better to buy burning software than using the free software (on Windows at least) if you want to avoid making a >= $3 coaster.

I find the 50GB capacity to be fairly easy to work with, since I'm much less likely to have to span files/folders across discs than the 25GB ones.

sp332• 11 years ago

Rewritable discs are bad, but write-once discs aren't. Especially if you keep them cool and dark.

jacquesm• 11 years ago

I wonder how Facebook is able to implement the EU privacy directives with this technology as their long term storage solution. If a customer requests deletion of their data you're supposed to comply with that, even on back-ups.

Cthulhu_• 11 years ago

Yet I don't think anyone will ever check that. How are you going to find one picture that should've been deleted in a cold storage system with exabytes of data (trillions(?) of photos)?

de_dave• 11 years ago

Assuming each item is scrambled with a small, unique key (kept on hot storage), just throw away the key?

junto2• 11 years ago

In all seriousness, it is more than likely a flag:

  User.IsDeleted = true

  User.Status = Status.Deleted (or Status.Deactivated)

Facebook allow users to deactivate their accounts, which can then be reactivated. Facebook users can also delete their accounts. Facebook say that this is permanent:

  When you delete an account, it is permanently deleted from Facebook. 
  You should only delete your account if you are certain you never want 
  to reactivate it.

Permanently deleted still doesn't mean deleted. It just means that you can't recover it. Facebook have never publicly stated what their deletion process is.

Is it logical or physical?

Since Facebook keep anonymous profiles, I wouldn't be surprised if deleted accounts just get migrated to anonymous profiles which they can still profit from. That's what I'd do if I was Facebook.

xyzzyz• 11 years ago

That kind of flag wouldn't be enough to satisfy EU privacy directives, the comment you're replying to is all about.

evgen• 11 years ago

"Permanently deleted" means exactly what it says. Deactivated means that the account is inactive but the data is still there. When you delete your account the data is gone within 14 days, no options for recovery; it happens in bulk and with a bit of a grace period for 'buyers remorse' so it is somewhere in the tail end of that 14 day period. And yes, Facebook has publicly stated their deletion process on multiple occasions and has explanations of how it works in online help docs and FAQs that you just choose to ignore.

paulsutter• 11 years ago

Delete by forgetting an encryption key.

If you encrypt each users data, in every backup, using a unique key, you need only erase the key in a central database to invalidate all possible backups.

chkuendig2• 11 years ago

Easiest way would be by just using another (slightly more expensive) cold storage medium for EU residents. From the article it sounds like they havent rolled this out to all data (yet)

bloat• 11 years ago

I used to have a job as one of those robotic arms.

The datacenter were I worked had lots of shelves full of tapes, and the computer would beep and display a tape number and a drive number. You'd have to jump up, find the tape and put it in the drive.

During the day you were running about constantly trying to keep the thing happy. On the night shift you could sit a watch movies and only get up once or twice an hour.

chinathrow4• 11 years ago

One of the real robots dropped a tape within a swiss data center on December 24th 2010 and subsequent error states stopped all debit card transaction as a result...

chkuendig2• 11 years ago

on the day everybody was scrambling to get their christmas shopping done.

i still remember the chaos (I also had to get some presents...)

edit: it was actually in 2000: http://www.boerse-express.com/mobile#/mobile/pages/96409 (german)

therealmarv• 11 years ago

I wonder how long it takes to put data from that storage online. What happens e.g. if I want to see my 5 old images fullscreen?

phire• 11 years ago

I think they still keep a copy online. But they only have to keep one copy as the cold storage is used as a backup instead.

Fuxy• 11 years ago

Can I have one of those Blu-Ray storage units searching all my discs would be way more convenient if i didn't have to do it manually :)

giancarlostoro• 11 years ago

I wonder how projects like Archive.org could benefit from something like this. I know it has it's up and downs, but if you have enough backups it shouldn't be too bad. I'm sure there's tons of untouched content that could easily be archived (an archive inside the archive), but other sites might as well, such as wikipedia. Of course they would need a bit of funding for this somewhat new endeavour. Maybe facebook can 'donate' the implementation to such sites.

mark_l_watson• 11 years ago

I really like how open Facebook is about their infrastructure, hardware designs, etc.

I would have thought that they would be trying something with higher capacity, but off the shelf == lower costs.

Interesting that FB needs to handle 900 million photo uploads a day. With higher resolution cameras on smartphones, their storage requirements will keep increasing. My Note 4 takes 4K video and very high resolution photos - FB, Google, etc. can store lower res versions only, but the pressure will be there for higher resolution support, especially with 4K TVs, and other high resolution devices.

samch• 11 years ago

Just to help add some clarity, this isn't Blu-Ray in the common consumer sense. I've seen some comments on here about the stability of the dyes and capacity. This technology, as mentioned at the end of the article, is Optical Archive storage [1]. The media are in the form of cartridges with 12 discs in each cartridge, and Sony claims that the media is "50 year rated". Sony currently manufactures cartridges with up to 1.5TB of storage[2]. They also make a 7U, 10-cartridge library [3]. So, in a standard rack, you could fit a maximum of 90TB of Optical Archive storage. That's admittedly not a lot given the storage density of hard drives, so I'm guessing that Facebook's decision to invest in this is partly driven by energy savings or other considerations.

[1] https://en.wikipedia.org/wiki/Archival_Disc

[2] http://pro.sony.com/bbsc/ssr/cat-recmedia/cat-oda/

[3] http://pro.sony.com/bbsc/ssr/cat-datastorage/cat-opticaldisc...

cc438• 11 years ago

So how does this work with the way Facebook delivers content? I understand the advantages of disc storage for a service like Amazon Glacier as the customer expectations are set for significant time and bandwidth constraints but Facebook is supposed to deliver personal data as fast as it can be served up. They've created a "customer" expectation of instant access that can't be provided by an offline disc.

It makes for a good solution as a 2nd or 3rd level backup but you'd need some form of RAID to prevent users from experience a lengthy delay every single time a disk fails. That's a lot of money to invest in a novel solution that creates a lengthy interruption in service should it ever be needed. Maybe the power savings are worth it when weighed against the rarity of such use and expected level of revenue lost to user frustration but that's an equation made fragile by so many variables.

I could also see it used in conjunction with some amazing use-prediction algorithm to store data that will most likely never be requested. If Facebook truly keeps everything then they have millions of photos, videos, hidden posts, and other digital detritus tied up in dead, inactive, or orphaned repositories. I know half the groups I joined back in high school are now impossible to find but the data they contained is still important for the purpose of linking people and interests together. I also have several friends who have passed away, making their account a kind of digital mausoleum. Their private data will never be accessed again but it must remain in FB's datacenter until someone is given access to the account to delete it. The number of Facebook accounts tied to the deceased is only increasing so these racks of Blu Rays might be considered the first digital cemetery. I don't know how many resources are allocated to this kind of data but this is where cold storage makes the most sense.

Facebook’s Blu-Ray Powered Cold Storage Data Center

Related Stories