5D ‘Superman memory’ crystal could lead to unlimited lifetime data storage

http://www.southampton.ac.uk/mediacentre/news/2013/jul/13_131.shtml?1

by swombat • 13 years ago

188 111 13 years ago

southampton.ac.uk

maggit• 13 years ago

I initially interpreted "5D" in the title as five spatial dimensions, which would be incredible, but the reality is entirely credible:

The information encoding is realised in five dimensions: the size and orientation in addition to the three dimensional position of these nanostructures.

quchen2• 13 years ago

I wonder how orientation is just a single degree of freedom though; somehow one of the polarization directions of the light must get lost along the way.

vlasev• 13 years ago

Perhaps it is because this is a crystal and there is a relationship or redundancy between the different axes of orientation.

EDIT: Actually, this [1] paper says in the abstract that they use wavelength, polarization and the three spatial dimensions. So it is wavelength and not direction!

[1]http://www.nature.com/nature/journal/v459/n7245/full/nature0...

rch2• 13 years ago

Quasicrystals were the first thing I thought of...

https://en.wikipedia.org/wiki/Quasicrystal

e3pi• 13 years ago

Quasicrystals were the first thing I thought of...

I searched your link, I did not find quasicrystals suggesting anything related to memory storage. Why quasicrystals?

kirualex• 13 years ago

The NSA is probably looking into it right now...

swombat• 13 years ago

For all you know they've built (or something else equally advanced) already. Which would make claims that they're storing everything pretty believable.

vergiliu• 13 years ago

"[...]At the moment companies have to back up their archives every five to ten years because hard-drive memory has a relatively short lifespan,” says Jingyu.

Since when hard drives die after 5-10 years? Are there some new physics laws that I'm not aware of, or what?

eksith3• 13 years ago

I'm guessing this is with MTFB rates given by manufacturers. With continuous use, it's not that uncommon to for hard drives to reach EOL fairly quickly.

sambeau• 13 years ago

It's a bit off but in the ballpark:

  Google's 2007 study found, based on a large field sample 
  of drives, that actual annualized failure rates (AFRs) for
  individual drives ranged from 1.7% for first year drives to
  over 8.6% for three-year old drives

http://en.wikipedia.org/wiki/Hard_disk_drive_failure

Memory, of course, has other problems:

http://www.ewh.ieee.org/r6/scv/rl/articles/ser-050323-talk-r...

lifeformed3• 13 years ago

Haven't you experienced it? My HDDs all wear out eventually... there's only so many times a thing can spin 7200rpm just nanometers off the ground before something gives.

JonFish85• 13 years ago

This stuff is really fascinating to me. Will we ever hit a point in data storage where you can fit all the data about a volume in space, into a smaller volume of data storage? I'm sure the Uncertainty Principle plays into it, but theoretically could it be done--store 1 cubic meter's worth of data into a 1 cubic micrometer of storage?

gardarh2• 13 years ago

You can't really, by definition. By the same logic you should be able to store 1 cubic meter's worth of data storage cube into a 1 cubic micrometer of data storage cube, which in turn could be stored in a smaller cube ad infinitum.

Going by the same logic you could store all entropy of the universe in an infinitely small cube which is not possible unless the universe had zero entropy.

stingraycharles• 13 years ago

But assuming the universe has an entropy smaller than one, couldn't the data be compressed ?

venomsnake• 13 years ago

Hmm ... go back 13.5 billion years in time and that is exactly what was the universe back than.

sp3322• 13 years ago

Sort of, according to the holographic principle. Since there is a limit on information density in space, you should be able to reduce a 3-dimentional volume to a 2-dimentional digital data structure. https://en.wikipedia.org/wiki/Holographic_principle

DanBC2• 13 years ago

How much entropy does the "data about a volume in space" have?

And is this lossless or lossy compression?

Here's a stack exchange question with a good answer (http://stackoverflow.com/questions/4716116/what-is-the-best-...)

If file sizes could be specified accurate to the bit, for any file size N, there would be precisely 2^(N+1)-1 possible files of N bits or smaller. In order for a file of size X to be mapped to some smaller size Y, some file of size Y or smaller must be mapped to a file of size X or larger. The only way lossless compression can work is if some possible files can be identified as being more probable than others; in that scenario, the likely files will be shrunk and the unlikely ones will grow.

As a simple example, suppose that one wishes to store losslessly a file in which the bits are random and independent, but instead of 50% of the bits being set, only 33% are. One could compress such a file by taking each pair of bits and writing "0" if both bits were clear, "10" if the first bit was set and the second one not, "110" if the second was set and the first not, or "111" if both bits were set. The effect would be that each pair of bits would become one bit 44% of the time, two bits 22% of the time, and three bits 33% of the time. While some strings of data would grow, others would shrink; the ones that shrank would--if the probability distribution was as expected--outnumber those that grow (4/9 files would shrink by a bit, 2/9 would stay the same, and 3/9 would grow).

yk• 13 years ago

No, since you could fill a cubic meter with the storage media. And then you need to record the data on the storage media as well as the structure of the storage media.

hughes• 13 years ago

I am not a physicist, but isn't that already possible, depending on the contents of the volume? I can store all the data about a 1km^3 vacuum containing a single atom at the centre in a volume much smaller than 1km^3.

jbri• 13 years ago

Sure, but you can't store the contents of an arbitrary volume, which is what people generally mean when they talk about storage capacity.

pessimizer2• 13 years ago

More depending on how much detail you want to use to describe the atom.

lifeformed3• 13 years ago

Yes you could - it would be the object itself.

eli_gottlieb2• 13 years ago

If you mean all information, enough data to recreate the volume particle-by-particle, then the answer is by definition HELL NO.

If you mean "all the data" as in "all the important data we actually care about", that's been feasible for decades.

So basically, any new or exciting sense of the question leads strongly to the answer, "No".

Dylan16807• 13 years ago

It doesn't even matter in the first place if it could store 'all' the information about a space, because it's physically impossible to measure that data.

But when you talk about recreating a volume, in a manner that is at least vaguely possible: you could store a record of every atom at angstrom-level resolution in much less space than the original.

GotAnyMegadeth• 13 years ago

They will have to change the name before they market it. I can't imagine many people understanding anything with 5D in the name, regardless of what it means.

wikwocket2• 13 years ago

Since when do people have to understand a name for it to take off? Look at the soup of technologies grouped under the label "4G." All consumers know is that 4G sounds better than 3G.

I'm sure if the technology is commercialized, someone will come up with a cute nonsensical name for it, but I can absolutely see a subhead somewhere saying "Advanced Five-dimensional storage technology!", just because it sounds cool.

anigbrowl• 13 years ago

Go tell Canon. The 5d has been one of their most successful cameras for years.

TerraHertz• 13 years ago

Finally.

Though I too suspect the NSA must be using some form of dense, long-lifetime data-in-solid storage technology that we haven't been told of. I can believe them storing 'all comms' in real time using hard disks, but I find it hard to believe they'd be anticipating refreshing the entire archive onto new media every 10 years of so. Which they'd have to do with hard disks or DVD equivalents. And I flatly refuse to believe they'd be prepared to just let it go.

There was an announcement back in 1999 from Keele University in UK, of having succeeded with using NMR to store and retrieve data in solids. The lead researcher was Prof. Ted Williams, who led development of the nuclear magnetic resonance scanner.

Then suddenly... nothing more was heard of that.

There's just something about this idea that sounds extremely likely. That not only is the NSA spying on everyone, but also maybe keeping to themselves a revolutionary storage technology that would change everything. So far I haven't heard anything at all about _how_ the NSA is storing all that data, have you?

We should be building eternal, public archives of all the cultural and scientific knowledge of humankind. Instead we're building vast secret archives of everyone's tweets, lunch appointments and credit card transactions.

swamp403• 13 years ago

I'm just not buying that.

It's well known that several hundred patents a year get swept up under a "secrecy order" and disappear. But most of these are from a) defense contractors working on military systems and b) nuclear research.

Occasionally (maybe 20 per year), an inventor unaffiliated with the government will attempt to get a patent, and have it get swept up under a secrecy order.

But a new type of data storage? I would surely think the government would let this get developed and refined in the private arena.

Cutting edge development is a sinkhole for massive amounts of money. Just think how many hard drives a billion dollars could buy? I don't think it makes sense economically, when there is a commonly available alternative.

What worries me more is the possibility that the oil industry has spies with control over this "secrecy order". The fabled "200 mile per gallon carburetor" type stuff.

It is quite easy for me to believe that a trillion dollar industry would attempt to protect itself by any means possible.

VladRussian2• 13 years ago

What worries me more is the possibility that the oil industry has spies with control over this "secrecy order". The fabled "200 mile per gallon carburetor" type stuff.
It is quite easy for me to believe that a trillion dollar industry would attempt to protect itself by any means possible.

all the engineering calculations for a car you can make yourself starting from the first principles, like

http://en.wikipedia.org/wiki/Carnot_cycle http://en.wikipedia.org/wiki/Fluid_dynamics

etc...

It would be very interesting if you can come up for 200mpg goal with better shape or lighter design or better in some other way than that

http://green.autoblog.com/2013/04/01/vw-hints-hyper-efficien...

sp3322• 13 years ago

They probably filter out most of the data. They're building a huge new datacenter; why would they do that if they have the capacity to store all that data already?

akavel• 13 years ago

Some other works resulting from their research announced on http://www.femtoprint.eu/ also sound quite interesting, e.g. "a first transparent actuator fabricated using the femtoprint process", sized in millimeters.

DanBC2• 13 years ago

This is really exciting. I hope it does come to market. There's a need for stable, huge, slow, storage which isn't really covered by drives at the moment.

A company called InPhase has had various forms of holographic storage "ready any time now!" for years. They eventually released a product. It's specialist, and expensive. That company eventually folded. (I remember when they were using concept art using a credit-card sized thing for media. That was ten years ago, and 1 TB was exciting.)

(http://gizmodo.com/5466793/inphase-closes-holographic-storag...)

fireismyflag• 13 years ago

Unlimited lifetime data storage is how I though of my 1GB HDD in 1996

cs702• 13 years ago

If commercially viable, this looks very disruptive to me: if everyone can get virtually unlimited, ultra-fast, lasts-a-million-years local storage in all their Internet-connected devices, the planet's online needs would change significantly in unpredictable ways. For example, third-party cloud-storage providers would face intense price pressure as more customers find it cheaper to store data locally and pay for more bandwidth.

PS. Does anyone here have sufficient knowledge of or experience with Photonics and nanostructured glass to comment on the near-term feasibility and commercial viability of this technology? Here's the paper: http://www.orc.soton.ac.uk/fileadmin/downloads/5D_Data_Stora...

quchen2• 13 years ago

Keep in mind that a positive research result is not necessarily anywhere close to any of usability, commercial viability, or even possibility outside the lab. Nuclear fusion is a prominent example of this, and I'm still waiting for MRAM, which has been "around the corner" for quite some time now. Until memory glass may be available, who knows what else is.

swamp403• 13 years ago

I'm STILL waiting on flexible LCD's.

5000 trade show demos, press releases and tech articles - and you STILL can't buy one to put in your product.

swamp403• 13 years ago

TI just released an FRAM memory (instead of FLASH) version of the MSP430.

It has several features which make it better suited to ultra-low power operation.

I know FRAM was popular 10 years or so ago, but never really made it big. I'd always assumed someone had a patent locked up that made other companies avoid it. Maybe that's changed recently?

kordless• 13 years ago

I disagree with your irrelevant comparison to fusion technologies. It's not like they have to create crazy powerful magnetic confinement fields here. It's lasers and glass. The CD went from an initiative at Phillips in 1974 to release in 1983. 9 years to fully develop and do a production release.

I think it's inspiring, and I'm thankful I'm alive to see all of it unfold before us.

TylerE• 13 years ago

Not even that....what about the 3D "holographic" optical storage (aka 1TB dvds) we've heard about for years?

dragontamer4• 13 years ago

MRAM is actually commercially available right NOW: http://www.everspin.com/PDF/ST-MRAM_Presentation.pdf

Buy it here: http://www.digikey.com/catalog/en/partgroup/parallel-mram/14...

Everspin needs to market it better... but yeah, if you're a Computer Engineer, you can build MRAM systems. The issue now is to build computers that actually take advantage of MRAM.

johnchristopher• 13 years ago

Keep in mind that a positive research result is not necessarily anywhere close to any of usability, commercial viability, or even possibility outside the lab. Nuclear fusion is a prominent example of this, and I'm still waiting for MRAM

I, for one, am still waiting for those jelly fish based CPU :(

rorrr2• 13 years ago

Storage is already pretty cheap, but it's a pain in the ass to manage, because you need to handle backups and make sure they are not in one spot, so a fire won't destroy 50 years of your photos. That's why cloud services and online backup is so successful recently. People want somebody else to secure their data, and they are willing to pay for it.

Our future is small computers + high speed internet available everywhere + online storage and services. Not the other way around.

CapitalistCartr• 13 years ago

"Our future is small computers + high speed internet available everywhere + online storage and services. Not the other way around."

Assuming our current technology, maybe. If I have limitless storage, and you do, too, I can be your backup, and you mine. Trickling in the background, always backing up. Or we could all have a mesh network backup doing similar to Dropbox crossed with Napster, without the centralization of either. Depending on bandwidth and storage growth, of course. Maybe the future is small computers, high-speed Internet available everywhere, and mass mesh storage/backup. Hopefully with encryption, however it goes, please God.

pessimizer2• 13 years ago

Storage is already pretty cheap, but it's a pain in the ass to manage

Management is part of the cost of storage. If storage is cheap except for the cost of managing it, which is expensive, then storage is expensive.

eli_gottlieb2• 13 years ago

Our future is small computers + high speed internet available everywhere + online storage and services. Not the other way around.

There's a massive backlash against this now that we find it's a recipe for Big Brother.

podperson• 13 years ago

Storage isn't so cheap when you consider that it doesn't last very long, so we're constantly having to move stuff to newer (thankfully usually bigger) storage devices.

dinkumthinkum• 13 years ago

"Our future is small computers + high speed internet available everywhere + online storage and services. Not the other way around."

That's nice. If you perform a cursory examination of computer science history, you will find many examples of "what's new is old." Any new paradigm, usage scenario, software type, or what have you could invert that. Even the one you like so much ... I think they had these things called dumb terminals and mainframes at one point ...

lifeformed3• 13 years ago

ultra-fast

Does it say how fast the read/write speed is?

nrivadeneira• 13 years ago

I believe the article mentioned femtosecond laser writing. I don't know how much data gets written in each pulse though.

peter3032• 13 years ago

A "balanced" computer system used to follow the "one second" heuristic. (Is this named after somebody?) That is you should be able to offload temporary memory (core) to permanent storage in one second. And it should be able to execute a computation on every element of memory in one second too. I know this has held up through the gigaword generation. Leading edge computers, like a supercomputer, may be unbalanced where one of these factors lags.

vortmeester• 13 years ago

They are getting 12 kbit/s right now, and claim they could get Mbit/s to Gbit/s using very fast light modulators (which themselves are research-lab projects, to my knowledge).[1]

The phrase "ultra-fast" laser is a term of art, which really means "laser with ultra-short pulses". They are using a laser with pretty short, 280 fs, pulses, but only a 200 kHz repetition rate.[2] It evidently takes several pulses to write one bit. Due to the physics of femtosecond lasers, you can't easily increase the rep rate without compromising the pulse length and intensity, which both need to be very good in this application.

The readout "should" be faster than "conventional" methods, but no details are given.[1] I don't see evidence that they have shown even moderate speed readout. It looks like they took micrographs and then inspected the images to determine the values of the bits.

[1] Gizmag article referenced in another comment, http://www.gizmag.com/superman-memory-crystal/28231/?utm_sou...

[2] Authors' original paper, http://www.orc.soton.ac.uk/fileadmin/downloads/5D_Data_Stora...

peter3032• 13 years ago

The new Utah NSA facility is planning for zetabyte (million petabyte) bandwidths. We dont use this much yet, but may during the lifetime of that facility. Humankind has an insatiable appetite for digital storage.

vmarsy• 13 years ago

In the article they don't say if it's a re-writable device.

Supposing it's not, it would be useful only for big archives : you want to use the full 360 Tb available, because the crystal can be expensive. An average user don't have 360 Tb to write everyday.

If the crystal was cheap, or if you could build smaller crystals that store less Tb , it would be different.

As lifeformed said, nothing is said about the reading speed.

ScottBurson• 13 years ago

The Gizmag article [0] claims that it is rewritable.

0. http://www.gizmag.com/superman-memory-crystal/28231/?utm_sou...

mortenjorck• 13 years ago

Or you create a novel filesystem that allows you to use an exotically large capacity, write-once volume as a less exotically large, write-many volume with built-in, block-level history.

njharman• 13 years ago

Available data will expand to exceed capacity. The amount of data created (or desired to be stored) will always exceed capacity to store (and process) it.

It would be disruptive, same as every other couple of orders magnitude storage in the past and in the future.

jotm2• 13 years ago

Not really - it would just continue the trend of offline local storage and maybe make the Internet more distributed. Pretty much only cloud hosting providers would be affected.

ISPs would probably be pressured into finally going big or going home with their bandwidth, too, which is a good thing.

bjcubsfan• 13 years ago

I worked in optics in grad school. I think that near-term commercial viability is not good. The precision optics and the ultrafast laser would have to be heavily adapted to work outside the lab, and would be needed for both read and write.

kragen4• 13 years ago

It's not disruptive unless it's not only commercially viable but actually cheaper, and I know that without even having read Christensen's book. Also I think they said they were writing data at some small multiple of 200 kilobits per second, which hardly qualifies as "ultra-fast".

6ren• 13 years ago

They don't actually claim it's fast or give any measurements of speed (apart from "ultrafast laser"). It seems reasonable to be on the order of other laser-based optical storage systems (CD, DVD, blueray). EDIT see also https://news.ycombinator.com/item?id=6034075

Instead, longevity and data density is emphasised, and application for long-term storage. Sounds like a replacement for tape.

tensei• 13 years ago

If the read/write/access speed is going to be good enough then perhaps it might replace HDD and depend on SSD as another cache level. The SSD cache would need to be page-file-based (just like OS caches file pages in RAM). block-based caching would be horribly inefficient in this case.

This also got me thinking about a filesystem for the 'superman memory'. It could be an extremely simple logging filesystem without garbage collection. Since the storage is supposed to be enourmous this could work for more than just a an incremental backup.

JulianMorrison• 13 years ago

"third-party cloud-storage providers would face intense price pressure"

…or it could go the other way and they would be able to offer "eternal cumulative storage" for a low fee. This tech will be big and expensive before it's small and ubiquitous. There may be a first mover advantage to being able to afford the early models.

nkoren• 13 years ago

Really intriguing. My father was in the data storage industry (magnetic tape), so I've always watched innovations in this sector with great interest. Nine times out of ten, the storage-density breakthroughs you read about end up being commercially meaningless because they aren't thermally or chemically stable. Doesn't matter how many TB/cm^2 you're writing, if the material degrades after a couple of weeks or months. This announcement is dramatically different, which strikes me as really interesting.

Manufacturability is the other obstacle which data storage inventions traditionally founder upon, on the road to commercialisation. I don't have a sense of whether or not that'll be an issue here. If it isn't a problem... then yes, this could be a genuine game-changer.

wikwocket2• 13 years ago

Excellent! Seeing as how Superman released in 1978, I expect scientists to have figured out the Arc Reactor and Repulsor technology sometime in late 2048.

heroic• 13 years ago

How awesome would it be if this was then used by NSA to provide agents with backups of all snooped data, available in their pockets, in a piece of glass!

dm2• 13 years ago

Not very awesome because you lose the ability to restrict access.

The internet makes that unnecessary.

chrisweekly• 13 years ago

So it really will all be over when the fat lady sings, huh. ;)

tocomment2• 13 years ago

Along a simliar vein I've heard it proposed you could store information in a diamond using different isotopes of carbon as binary 1 and 0's. That would last a tremendously long time and I believe have a huge storage capacity.

pronoiac• 13 years ago

It's called "memory diamond." I'm aware of it thanks to Charlie Stross[1], who credits the idea to Eric Drexler.

[1] http://www.antipope.org/charlie/blog-static/2007/05/shaping_...

tocomment2• 13 years ago

Interesting read, thanks. I wonder why he suggests two atoms per bit? Seems wasteful?

sixdimensional• 13 years ago

"Three dimensions should be enough for anybody" :)

egypturnash• 13 years ago

I can't quite tell from any of the articles about this if it's write-once-read-many or read-write. Given that it's about "fused quartz" I am suspecting write-once.

jotm2• 13 years ago

Funny how they use crystals in many movies for data storage and I always found it a bit stupid... But in reality, it looks like they're the future of storage - I remember they experimented with writing data using lasers to cubes about a decade ago...

samatman• 13 years ago

With a background in chemistry, it seems inevitable, not stupid.

codemonkeymike• 13 years ago

Data always expands to fit a technology

milhous• 13 years ago

Amazing. Isolinear Chips from Star Trek!

ChuckMcM• 13 years ago

This is a fascinating result. The equipment needed to reproduce is a bit extreme (femptosecond multi-watt lasers are not 'garden variety' by any means) and one wonders if the 'read' set-up could be made cost effectively than the 'write' set-up. I would hope that Google would invest in a system to copy all of the books they've scanned into such crystals.

That said, I could imagine offering something like a web 'snapshot' (think Internet Archive meets the DVD-R :-) which one might subscribe to. That would be pretty cool.

jl6• 13 years ago

Really, how can they get away with the word "unlimited" in the title there? The limit is mentioned in the first paragraph: 360TB/disc (whatever a disc is... though I assume it is something which does not exist in unlimited quantities).

Still, nice work! Data storage capacity is a great metric of our digital quality of life.

Coincoin• 13 years ago

Why does this same magic crystal storage article keep poping up every 5 years? They always claim they could be very close to commercialisation and that it could change the world forever. Yet 25 years later, I fail to see any progress.

EDIT: I don't pretend I can understand the technical details, does anyone know if this is actually anything different from that old holographic stuff?

danpalmer• 13 years ago

Nice to see Southampton in the news. This is part of the department I study Computer Science in, and it's this research focus that drew me to Southampton. Awesome place.

xmr• 13 years ago

Glad to see it's still going well - it had a great atmosphere a decade ago when I was there. So much goes on there that isn't widely reported.

abrichr• 13 years ago

There seems to be a lot of misunderstanding of the title.

"Unlimited lifetime storage" means storage whose lifetime is unlimited.

As it says in the article, the actual data capacity is "360 TB/disc".

5D ‘Superman memory’ crystal could lead to unlimited lifetime data storage

Related Stories