An Australian researcher has worked out how to store 1000TB on a CD

http://www.sciencealert.com.au/news/20140309-26116.html

by notdarkyet • 12 years ago

179 128 12 years ago

sciencealert.com.au

droessmj• 12 years ago

Brilliant solution to a known natural limitation.

EGreg• 12 years ago

OK A) I think in this case the patents are justified :)

AND B) Are CDs gonna make a comeback? Can we have cool mini-CDs?

duaneb• 12 years ago

The 1000TB figure was actually for a DVD. That said, I certainly hope so :)

sp332• 12 years ago

I'm pretty sure "DVD" isn't any more accurate a term for this tech. The bits are way smaller than the pits on a CD or DVD. Also, the laser used for writing is purple, while for DVDs it's red, and CDs use infrared.

mark-r• 12 years ago

By the time this is ready for production, flash or disk will have already caught up. I've seen it time and time again - the old tech continues to improve exponentially while the new tech takes forever to turn into something practical.

peter303• 12 years ago

The CD is twenty feet wide!

jtchang• 12 years ago

That is pretty awesome. The solution seem so simple in retrospect. Just mask out the larger 500nm light you don't want to hit the surface.

Certainly even 1TB CDs would be insanely awesome. It would open up a whole realm of mass storage. I'm just feeling how easy backups will be :)

frandroid• 12 years ago

A young Victorian researcher

Steampunk storage FTW.

DigitalJack• 12 years ago

This (interference) is how the feature size of semiconductors has gotten so small.

hristov• 12 years ago

That will warm the hearts of the Ubuntu Unity team. Can you imagine the amount of bloat they can put on a single installation disk now?

codereflection2• 12 years ago

Title says CD, but it's actually a DVD. Still extremely impressive.

ctdonath2• 12 years ago

It's not a DVD. Point is it's a 5.25" thin polymer disc of some hole-based storage media. "CD" and "DVD" are used just to make muggles get the idea.

codereflection2• 12 years ago

Link? Every article I can find says they used a DVD.

jpmattia• 12 years ago

I see where they mention writing the data, but does anyone see how/if they are reading the data?

From his other pubs [1], it looks like a two-photon process is involved as part of writing in deep-subdiffraction limit for lithography. Will be fun to try and reverse that.

[1] http://www.swinburne.edu.au/engineering/cmp/profile.php?memb...

Edit: Paper that this PR refers to is here: http://www.nature.com/ncomms/2013/130619/ncomms3061/full/nco...

wil421• 12 years ago

The real breakthrough would be a way for regular CD/DVD drives to at least be able to read the new format. But I doubt that this will be practical for consumers. Most consumers dont really need a CD drive anymore, companies like Apple arent even including them in laptops any longer.

bitJericho2• 12 years ago

I think CDs would make a comeback if you're talking about being able to hold an entire series of movies all in 4K and in 3D and not break a sweat.

Digital downloads is all the rage right now, but people will still want to archive their downloads.

ChuckMcM• 12 years ago

PDF link: http://www.nature.com/ncomms/2013/130619/ncomms3061/pdf/ncom...

This is pretty cool stuff. Reading through the paper now.

golemotron• 12 years ago

The print collection of the Library of Congress is estimated at 10TB. 990TB would be left on the DVD. Amazing.

toomuchtodo6• 12 years ago

Futurama called it:

http://bookriotcom.c.presscdn.com/wp-content/uploads/2013/09...

jacob019• 12 years ago

Can't wait to back up my 12TB RAID on a single disc.

benjaminva• 12 years ago

The idea to break the diffraction limit by using the this doughnut-shaped excitation spot comes from the STED microscopy (http://en.wikipedia.org/wiki/STED_microscopy) developed by Stefan Hell (Max Planck Institute for Biophysical Chemistry in Goettingen, Germany). He actually showed that an additional intensity-dependent term can be added to Abbe's diffraction limit allowing sub 100um resolution with light. I think this idea is awesome and it sure won't be the last invention made possible by this trick.

penguat• 12 years ago

I'm sure this or a similar technique is already in use in commercial photolithography.

krrrh• 12 years ago

If this is real and can be commercialized it could lead to some pretty devastating fallout for the entertainment industry. You've got to know that there are a few obsessive archivists out there with pretty comprehensive collections of films spread across rooms of RAID arrays. When they can start backing those collections up for friends onto a single disc, ie a single disc that has more films than you could possibly watch in your lifetime the entertainment industry is going to pine for the days when torrents eve the big problem.

The same goes for rogue librarians, or Google Books employees dumping entire libraries onto discs and leaking them out.

waqf2• 12 years ago

Books are small enough that this has pretty much already happened (search for "library genesis").

toomuchtodo6• 12 years ago

Thank you for that.

fdej• 12 years ago

Library genesis is the best thing that happened to research since Google (and a huge number of researchers use it, secretly). Being able to access any obscure technical book in the world in a few seconds is literally orders of magnitude more efficient than waiting a week for an interlibrary loan. I dread the day it inevitably gets shut down due to pressure from the copyright mafia. It's ridiculous that such a service isn't 1) legal, and 2) publicly funded (with due compensation to the authors).

laichzeit0• 12 years ago

Could you explain more exactly what "library genesis" is? Is it a collection of basically every book ever published or only modern things? Did someone scan everything, like Google books?

IkmoIkmo• 12 years ago

Holy shit... Just tried some obscure stuff I'd been trying to find for ages. This is amazing

Someone1234• 12 years ago

more films than you could possibly watch in your lifetime

I was curious if this statement was true...

So assuming, 4.4 GB her HD film & 2 hrs long each, you can fit roughly 232 films per TB or 232,000 films total (for 1000 TB), that's 464,000 hours of HD content or 53 years (!) back to back.

So, yes, that statement is true. Now the next logical question: Has the human race produced 464,000 hours of video content?

Let's assume there are around 100 cable channels showing content in HD video 24/7 (with no overlap, there are more but a lot of duplication). That's 2,400 hours of content a day, 876,000 hours a year or 101 years back-to-back worth of watchable content shown each year.

But realistically that will be at least 70% duplication (particularly looking at it over several years). So even if we just look at 24/7 news and shopping channels which always produce original content, you're still talking about easily 200,000+ hours/year.

atrus• 12 years ago

http://www.imdb.com/stats

Seems to be rather likely.

Someone3• 12 years ago

https://www.youtube.com/yt/press/statistics.html claims "100 hours of video are uploaded to YouTube every minute"

So, that would be 4,640 minutes or less than a week. Lower quality, and probably with (near) duplicates, but I guess it is safe to say we have that amount of video.

Another way to look at it: I expect we produce more content in unique wedding videos every year (at an hour each, that takes half a million weddings or a million people marrying. At an average of one marriage per life and a life expectancy of 75 year, that takes a population of 75 million people)

userbinator• 12 years ago

Has the human race produced 464,000 hours of video content?

Far more than that... YouTube alone claims "100 hours of video are uploaded to YouTube every minute":

https://www.youtube.com/yt/press/statistics.html

...which comes out to 144kh/day. In other words, one of these discs would be enough to store ~3 days worth of contributions to YouTube. So a box of these discs, let's say 1000 of them - which isn't all that big (e.g. http://www.amazon.co.uk/Neo-Aluminium-Storage-1000-sleeves/d... ), could hold an archive of every single video that has ever been uploaded to YouTube.

I am now reminded of this: http://www.youtube.com/watch?v=Y_UmWdcTrrc

Edit: got beaten to mentioning YouTube, but there's many other video sharing sites on the Internet, and of course there are probably countless hours of one of the types of video the Internet is well known for: porn.

morcheeba• 12 years ago

Let's assume there are around 100 cable channels showing content in HD video 24/7 (with no overlap, there are more but a lot of duplication). That's 2,400 hours of content a day, 876,000 hours a year or 101 years back-to-back worth of watchable content shown each year.

I'm thinking that 100 channels provide exactly 100 years worth of content per year instead of 101. Unless you rewind the good parts :-p

vidarh2• 12 years ago

And there are of course vastly more than a 100 cable channels. BSkyB alone broadcasts more than 600 channels for the UK market (more than a thousand if you include radio). In 2010 Ofcom passed 1000 licensed UK channels.

The US is supposedly the only other country with similarly high number of channels, but I don't think it's unreasonable to assume that there are more than 10,000 channels worldwide.

jlaswell• 12 years ago

more films than you could possibly watch in your lifetime
> Let's assume there are around 100 cable channels showing content in HD video 24/7

Note that this is comparing HD content to films, though I still think you are fine in saying we easily have enough content. :)

NoMoreNicksLeft2• 12 years ago

So assuming, 4.4 GB her HD film & 2 hrs long each

People bitch constantly, but a typical feature film encoded by YIFY at 720p will clock in at somewhere between 650-900 megs, leaning towards the lower end of that range.

Not that it matters, the rest of your math makes it look ridiculous. Without too many tradeoffs, you could certainly put every American (Hollywood and indie both) film on such a disc.

Now the next logical question: Has the human race produced 464,000 hours of video content?

Somewhere Google has a gauge that shows how many hours of videos they store... but I can't find it.

So even if we just look at 24/7 news and shopping channels which always produce original content, you're still talking about easily 200,000+ hours/year.

Agreed. But I don't think that they have archival policies in place where everything is kept permanently. Even the news channels may have been in the habit of dumping everything but stuff deemed important, well into the 1990s.

chimeracoder2• 12 years ago

As of a couple of years ago, the entire content of The Pirate Bay[0] was under 100MB[1].

Think about that for a moment - it's the index for one of the largest collections of publicly available content in the history of humanity, and it fits in my wallet 80 times over[2].

[0] Not the P2P content that people download - just the main site itself, containing all of the magnet links, etc. TPB's 'content' is literally just an index (it's not even a tracker).

[1] I don't want to link to TPB from HN, but just search for "pirate bay archive". There's a large archive that contains all of the site data (db backups), but the magnet links are all you really need content-wise.

[2] http://www.amazon.com/Compact-Credit-Style-Flash-Drive/dp/B0...

sauere• 12 years ago

Why would you want that? The magnet links alone are rather useless. The whole site can be crawled within 2-3 days easily. There are ~1.2 million torrents indexed, lets just assume that a single magnet-link plus some metadata is 0,5-1kB... so that would be an total of ~1GB data.

dmn001• 12 years ago

the entire content of The Pirate Bay Not the P2P content that people download

I think you're confusing different definitions of 'content' here. Also, there are many more sites that have publicly available indexes of other links much greater than the pirate bay, google.com for example.

Furthermore, 8GB is relatively small for a flash drive nowadays, e.g. here is an 128GB micro sd card: http://www.amazon.com/dp/B00M562LF4

toomuchtodo6• 12 years ago

Looking forward to being able to carry the Internet Archive around in my backpack.

NKCSS• 12 years ago

You are forgetting that the speed at which optical drives can be written is bound by the rotation speed. If you have a single laser of this type and mount it in a traditional drive as we know, it would take over a year to write a single disk :) Still lots to do :)

throwawayaway• 12 years ago

is anyone else really bothered that the article says CD but it's actually about DVD?

harwoodr2• 12 years ago

Really? Assuming that the rotation speed is the same for a normal disk and this denser format, wouldn't said laser be moving over 250,000 times as much data in the same time period?

At 1X speed, a DVD's data rate is about 11Mbit/s - so at 250,000 times the density, isn't it safe to assume a theoretical data rate of 2.75Tbit/s? Of course, pushing that much data at that rate is another problem...

If my assumptions are incorrect, please let me know - I'm just trying to understand.

thinkling• 12 years ago

Hmmm. Maybe you could use a prism/mirror array (like in digital projectors) to use a single laser to write in multiple places at once? (Or rather, in sequence but moving faster than the disc rotates?)

shock2• 12 years ago

I'm wondering if the entertainment industry is the reason for which Hyper-CDROM[0] is not commercially available. If so, something is very wrong with the way society incentivises development and commercialization of new technologies.

[0] http://en.wikipedia.org/wiki/Hyper_CD-ROM

polack• 12 years ago

Will this really be a larger "problem" for the entertainment industry than online file sharing? Is there many people out there that would want such a disc that already doesn't have fast speed Internet and can download any movie they want within minutes?

woodymcpecks• 12 years ago

Seems a little past it's prime. Why would you want this when you have the internet?

gfodor• 12 years ago

sometimes the lack of imagination on Hacker News is a sight to behold

nnnnni• 12 years ago

That's because it's often really "startup news" =-(

ctdonath2• 12 years ago

Bandwidth.

What if I could hand you the totality of Netflix, iTunes, etc.? Not wait for it to download, but transfer large chunks of the Internet in seconds? and no longer have to hope that, say, Netflix won't lose vast sections of their library for contractual reasons (as it has done of late)?

Backups: permanent archiving of everything you have all on one disc.

It makes vast swaths of data yours, in your hand, under your control, in a ridiculously compact & cheap media. No more hoping that you can get X in time, or that it will be there some time hence.

I've seen profound transformations in computing when cheap fast storage increased by an order of magnitude of orders of magnitude. This will bring that about again, to similarly disruptive and amazing results.

toomuchtodo6• 12 years ago

Netflix would put the entire catalog in your home, encrypted, and the only data they would have to transfer would be for the control plane (billing, recommendations, etc) and encryption keys to auth you.

Totally side-steps last mile monopolies as well. Don't want to work with us? We'll drop an LTE chipset in as well for the admin stuff. Redbox In Your Home.

seanflyon• 12 years ago

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." —Tanenbaum, Andrew S.

oconnore• 12 years ago

If you mail a single disk (nevermind a box full of discs) to your friend, and it arrives in 3 days, you have achieved about 31.6 Gbps. If you mail a USPS flat rate box with 100 discs, you have achieved 3160 Gbps.

You cannot buy a three thousand Gbps internet connection.

tlarkworthy• 12 years ago

you have to read the data off the disk though, and that won't be 3160 Gbps

krapp2• 12 years ago

"the internet" is a more fragile medium than a disk, that's why. High-density, cheap, persistent data storage is critical to make sure that most of the cultural expressions of the last few generations don't vanish into a black hole sooner than later. It's our version of ancient Babylon's cuneiforms etched in wet clay. Just imagine how much crap Jason Scott could fit into a storage container with these. A lot of crap. A loooot of crap.

kedean2• 12 years ago

I'm skeptical that this won't be extremely fragile, physically. With details that small, the slightest scratch could erase large swaths of data.

mjackson• 12 years ago

With the $18,000 fellowship, Gan will collaborate with industry and researchers around the world to work on new breakthroughs for data storage devices, and also see how his existing research can be used on a larger scale to rapidly improve the capacity of optics-based information technologies.

$18K seems like a drop in the bucket for powering this kind of research. Dr. Gan needs someone to introduce him to Kickstarter.

cgio• 12 years ago

Given Australia salary scales, this may be a typo, 180,000 would sound normal.

vacri• 12 years ago

Nope, it's $18k, but it's not a wage. It is low (Australia doesn't have the money that US universities are awash in), but it's for things like international travel to conferences and the like.

http://www.swinburne.edu.au/media-centre/news/2014/08/victor...

miahi• 12 years ago

I guess this is because the technology only solves one of the problems (getting a very narrow writing beam/point) and not the others, like: making the disk so stable that it doesn't wobble for more than a few nanometers during rotation; keeping the laser beam focused on the disk; figuring what kind of rotational and seek speeds (=read/write speeds) you can handle with these limitations; making everything small and ready for market.

trhway• 12 years ago

laser heterodyne :)

jcrubino• 12 years ago

Have those guys sign up for Y Combinator!!

happycube• 12 years ago

This could be a DVD - it would be an easy enough thing to perform tests on, and they say they're using a red laser for the actual writing. If the dye can handle the very small writes, that is. (And I wouldn't trust Memorex for this ;) )

Error correction for these is going to be interesting, even a speck of dust could obscure many megabytes of data.

Fando• 12 years ago

Awesome tech. I wonder how data will be written and read from such a disk? How long will it take? The transfer of 1000TB will be an interesting problem indeed. At current theoretical blu ray writing speed of 12x or 400Mbps [1] the time to read/write such a disk would be:

1000 terabytes = 8,000,000,000 megabits 8,000,000,000 / 400 = 20,000,000 seconds (7.6053 months)

[1] http://www.blu-ray.com/faq/#bluray_speed

rasz_pl• 12 years ago

Could probably be written all at once using immersion lithography. You could threat them as big read only memory 'crystals'. Buy a set of 100 containing entire YT archive etc

Someone3• 12 years ago

A big problem with spinning disks is that storage grows inversely with the area of the dots, while reading and writing speed grow inversely with the diameter of the dots.

If you store 100 times as much data, all else being equal, time to write a full disk will be ten times as large.

Ways around that are increasing rotation speed (been there, done that. There is little to gain here without making the disk a lot stronger = heavier, if that is really possible at all) and using multiple heads (harder to do, but may eventually be the better solution, certainly if one can completely do away with head movements)

Retric• 12 years ago

A 50GB Blueray disk can be written in 12 Minutes with a home burner but a few seconds in mass production. Scaling that up this would be (1,000,000 / 50) = 20,000 times the space but take sqrt(20,000) * 12min = 28 hours to write at home.

However, assuming you can R/W at the same rate that's 10GB / second speed which might be tough for a home PC to pull off anytime soon.

PS: I suspect if this where ever out into production they would change the form factor to enable even faster R/W speeds and a significantly reduced capacity.

delecti• 12 years ago

Bandwidth increasing proportional with the square root of the storage growth is a pretty great trend. If anything that sounds like a plus.

tgb• 12 years ago

Looks like it gives 2.9 gigabits per second for a 5cm disk radius at 5000 rpm.

http://www.wolframalpha.com/input/?i=5000rpm+*+(2*pi*5cm)%2F...

ridgeguy• 12 years ago

Stronger ≠ heavier.

The parameter of interest is specific strength, which is the material's (ultimate tensile strength/density).

I forget what the plastic in DVDs is, but it could probably be reinforced with carbon fibers (chopped, nanotubes, etc.) to give a useful increase in rotation speed.

Someone3• 12 years ago

You are right and I am not an structural engineer, but I understand that pure strength isn't the only limiting factor for disk; they start wobbling before they tear apart (http://superuser.com/questions/554717/will-there-ever-be-fas...). That, I think, is more a matter of thickness than of pure tensile strength. Barring the construction of hollow disks, that means heavier ones.

Also, now I think a bit more about it: in a 50-speed CD player or a 16-speed DVD player, the edge of the disk moves at about half the speed of sound. Doubling rotation speed would push it over the sound barrier. I guess it would take quite a bit of research to make a CD/DVD/BlueRay player that will work fine at those speeds (a way around that is to make the disk smaller. That would be an option for a disk with the storage density discussed here, but it wouldn't win you more than a factor of four or so, at most)

egamble• 12 years ago

One hopes that the write speed of these disks will be proportionately faster than DVD write speed. A DVD writing at 16X transfers about 22MB/s. At that rate it would take 1.44 years to fill up this disk writing to it continuously. Even at 1GB/s (roughly the write speed of an SSD) it would take almost 12 days to fill up one of these disks.

vlunkr• 12 years ago

Wow this seems huge If it's practical to implement everywhere. You could keep every picture/song/movie/document/game from your whole life on a disk, and hey, make a backup too.

polskibus2• 12 years ago

Expect Amazon Glacier price to drop dramatically in a couple of years :)

polskibus2• 12 years ago

Why downvoted ? Allegedly, blue-ray recordable discs are used by Amazon for Glacier backup. Surely they will be the ones very interesting of further development of this technology.

stronglikedan• 12 years ago

Would this reduce the storage lifespan of the disc? Would this technique make it physically more brittle, if the current form factor is used?

lukifer2• 12 years ago

I would love a potential use case as a backup that is redundant dozens or hundreds of times over. I'm not sure the exact mechanism by which DVDs start to lose data over 10-20 years, but this tech might be a way to mitigate that risk (as well as scratches and other physical damage).

jorjordandan• 12 years ago

He has announced that for his next challenge, he will store 4k video on betamax tapes. Impressive.

motoboi• 12 years ago

The Internet Archive would benefit very much from this! Does anyone remember the blueray based long term archive rack from facebook?

dghughes• 12 years ago

Anyone remember the company from about 2000/2001 called Constellation 3D? I do I lost my $3,000 investment when they went belly-up, doh!

They made a fluorescent multilayer DVD and a credit card shaped ClearCard the DVD which I believe the goal was to store several hundred TB of data.

And this was about 14 years ago!

craigds• 12 years ago

Every day, humans are producing more data than ever before - around 90% of the world’s data was generated in the past two years alone - and there will come a point when our data storage centres and the cloud can no longer keep up.

Is this true? Can anyone point to a some kind of study showing convergence of storage capacity and data production over time? I was under the impression that we've got far more storage than we'll need, at least in the near future.

eru4• 12 years ago

People find use for empty space.

vidarh2• 12 years ago

Given that we can today fit tens of TB in a fistful of SD-cards (512GB per card), no I don't think it's true at all that we'd get to a point where our data storage can't hold up. Our ability to filter and identify gems outside of the most popular stuff is a far greater challenge.

Vanayad• 12 years ago

I know I've seen this like a year or more ago here or on slashdot... :/

shock2• 12 years ago

Could Amazon Glacier use some pre-consumer form of this technology?

perlgeek• 12 years ago

I doubt it. There's a long, long way from the theoretical finding to the practical device.

Writing and reading speed has been mentioned. Another point is that as the dots become smaller, the mechanical parts need to be much more precise, moving the device from consumer parts to really expensive parts and assembly.

sfmelton• 12 years ago

It's hard to believe they are only giving this guy $18k.

saeguaiga2• 12 years ago

Oh great, another 1024-bit daisy-chained medium with a 20MB UeFI boot partition. We have watched this way too long and it was dragging us down...(cough)

Oops, accidentally implemented as a medium-hash CAMFS in disused packaging-and-optronics 45nm fab. Please write test, mind 5W limit if retaining cyano/pyridene dye in media.

elwell• 12 years ago

This is technically old news. Here's one from a year ago with the same picture: http://gizmodo.com/researchers-have-found-a-way-to-cram-1-00...

An Australian researcher has worked out how to store 1000TB on a CD

Related Stories