Web server offers true random numbers via fluctuations of the quantum vacuum

http://www.jwz.org/blog/2012/04/random-numbers-via-fluctuations-of-the-quantum-vacuum/

by rickyconnolly • 14 years ago

73 50 14 years ago

jwz.org

noodly• 14 years ago

Here's the original source: http://150.203.48.55/index.php

mapleoin• 14 years ago

And here's a pretty stream of random colors: http://150.203.48.55/RainCol.php

moontear• 14 years ago

I wanted to post this too. Makes me feel special knowing that these are truly random.

acqq3• 14 years ago

What's wrong with the one in jwz's post? http://photonics.anu.edu.au/qoptics/Research/qrng.php

mike-cardwell6• 14 years ago

If you have a modernish machine running Linux, find out if it has a TPM (Trusted Platform Module) built in. If it does, you can use an application called "Trousers" to take advantage of its built in hardware random number generator. It will directly feed random data into /dev/random

If you don't have a TPM, you can get one of these usb sticks: http://www.entropykey.co.uk/ and it will also feed /dev/random with large amounts of real random data.

skrebbel• 14 years ago

How can a TPM provide as "real" random values as a quantum vacuum? (or cosmic rays, or anything else very much random in nature)

JoachimSchipper2• 14 years ago

According to http://www.outflux.net/blog/archives/2009/10/22/tpm-as-rng/#..., TPM may even be using PRNGs. So you're right that the quality of output need not be better than what a modern OS already provides. (I am not aware of any case where a properly-used /dev/random failed to produce sufficient entropy. Note that doing something sensible at boot is part of "properly-used" here.)

It is possible to provide true randomness from a chip, but it's probably cheaper not to.

mike-cardwell6• 14 years ago

When did I say that it does? The randomness of the data is entirely down to the implementation.

Personally I'd trust the random data supplied by a third party web service less than the pseudo-random data that the software on my local machine produces. Especially if you mix in data from a TPM, especially if you mix in data from an Entropy Key as well.

teamnyble• 14 years ago

Can anyone explain how this works? The entropy key site says "The Entropy Key contains two high-quality noise generators, and an ARM Cortex CPU that actively measures, checks and confirms all generated random numbers, before encrypting them and sending them to the server"... but why can't we just write high-quality noise generators in software? How is the hardware better?

ajuc• 14 years ago

Because you can't get really random number from deterministic hardware (and computers are deterministic, more or less :) ). No matter what you do, running the same algorithm on the same input gives you the same data on regular computer. And the randomness (measured in Kolmogorov complexity [1] of the resulting string) is by definition smaller than the length of the generator program.

And truly random string is such string, that there's no possible program producing this string as output, that is shorter than the string.

On deterministic computers you can only produce pseudorandom numbers, with varying quality of randomness, depending on generator program you use, but it's sometimes not enough (esp. in cryptography).

[1] http://en.wikipedia.org/wiki/Kolmogorov_complexity

silvestrov• 14 years ago

Pure software-based random generators are deterministic and thus not random.

When something is deterministic, it is reproducible and therefore not good for creating crypto keys.

Most /dev/random implementations use data from the ethernet driver, the keyboard, mouse etc to get some input which isn't easy to reproduce.

mzl• 14 years ago

Software is deterministic, so it is not really possible to generate noise. There are non-deterministic parts of software (for example, thread interaction in concurrent systems), but this is just observable behavior of hardware effects.

The typical thing people do to get large amounts of randomness is to first generate some "real" random noise from some source (hardware thingamajigs, network timings, user interactions) and extrapolate these into longer sequences of random-looking numbers. The extrapolation can be fast and simple (in which case the randomness is not so good), or it can be slow and complex (to get decent looking randomness).

DanBC2• 14 years ago

Everyone has mentioned how software is deterministic.

Hardware RNGs sometimes rely on thermal noise (which is really random) but there are sometimes flaws with how that noise is sampled and de-skewed. Also, they need to be monitored to cope with hardware failure. Be aware, especially if you're using them for cryptography, that they might be a poor fit for your purpose.

Testing Hardware RNGs

(http://www.robertnz.net/true_rng.html)

EDIT:

Descriptions of various forms of noise:

(http://www.eie.polyu.edu.hk/~ensurya/lect_notes/commun_cir/C...)

And surely everyone on HN knows that part of the "snow" noise displayed on an untuned TV is cosmic background radiation, ie "afterglow" of the big bang. I still find that amazing.

defrost• 14 years ago

In theory, as others that replied have noted, hardware is better ... in practice there are issues other than "software is deterministic".

Many naive hardware based random number generators suffer from being not as random as one might think - thanks to quantising levels in A/D converters, to supposedly random physical processes having "spectra" where measurable signal occurs more in some bands than others, and to a multitude of other odd little effects just making a hardware based RNG is as simple as it seems.

The entropy key cited above has multiple noise sources and PRNG processes that mix them up and running checks to see that things are working as expected. That level of paranoid checking is more or less the minimum level required in a RNG to be confident about it.

mike-cardwell6• 14 years ago

Others have explained why it's not possible to create a good software random number generator. Hardware RNGs don't just provide a much better quality stream of random data, they also provide a much higher quantity of it. I just generate a GnuPG 2048 bit keypair by running "gpg --gen-key" on my laptop. It took over 11 minutes to gather enough entropy to build the key. I then plugged my Entropy Key into the laptop and repeated the process. This time it took less than 2 seconds.

Florin_Andrei• 14 years ago

> If you have a modernish machine running Linux, find out if it has a TPM (Trusted Platform Module) built in.

How do you determine that on Linux?

fleitz• 14 years ago

https://wiki.archlinux.org/index.php/TPM

Core-TX2• 14 years ago

oh noes. Not this poo again. Save money, and do the following.

- Get a high res webcam - Open it up and remove the filters. - Hook up the device. - From time to time you'll see random white dots appear.

There you have your ultra cheap random number generator. Works best in high radiation environments.

mike-cardwell6• 14 years ago

Do you know of any guides to do this? Are you aware of any free software to convert the image data into a usable stream of random data?

rickyconnolly• 14 years ago

"oh noes. Not this poo again. Save money, and do the following. - Get a high res webcam - Open it up and remove the filters. - Hook up the device. - From time to time you'll see random white dots appear. There you have your ultra cheap random number generator."

It would seem that this method would be muchmore expensive and time consuming than a simple web query. How precisely would this save me money?

Arjuna• 14 years ago

"Do you know of any guides to do this? Are you aware of any free software to convert the image data into a usable stream of random data?"

Look into LavaRnd [1][2][3][4].

[1] What is LavaRnd? http://www.lavarnd.org/what/index.html

[2] LavaRnd Process in Detail: http://www.lavarnd.org/what/process.html

[3] Construction of a LavaCan: http://www.lavarnd.org/what/can-details.html

[4] Source code (open source): http://www.lavarnd.org/download/index.html

xerula• 14 years ago

If you don't want to build your own generator, random.org provides a service like the ANU Quantum Random Number Server, i.e. provision of unique sets of true random numbers by http request, but generated from atmospheric noise. There's a convenience R package "random" I often use for this purpose. The articles on the website are quite edifying regarding the art and science of random numbers generally. Here's a link to the interface documentation: http://www.random.org/clients/http/

sp332• 14 years ago

That's random, but you still need to analyze the output for a while to normalize it.

andrewcooke• 14 years ago

so is the paper that describes this available for free anywhere? the trail of links from that page ends at http://apl.aip.org/resource/1/applab/v98/i23/p231103_s1?isAu...

update - similar article (free) from same people http://www.opticsinfobase.org/view_article.cfm?gotourl=http%... (source http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-21-2... )

[it seems like this is actually a big deal - they are getting huge throughput]

tete• 14 years ago

Hasn't Via done something similar?

Since we have millions of transistors in our systems couldn't they (their collectors) be used to generate "truly" random numbers?

mtrimpe• 14 years ago

I've always wondered how many probabilistic algorithms would have been discovered which outperform deterministic ones if a high-bandwidth hardware RNG was available on every machine.

patrickyeon• 14 years ago

I understood that probabilistic algos only need something that "looks random enough" and are fine using PRNGs. True randomness doesn't buy you any improvement, as the algorithm is only counting on some of the properties of a random source.

mbq• 14 years ago

The game is about speed -- probabilistic algorithm use PRNGs so heavily that their internal CPU demand may easily become the limiting factor.

eli_gottlieb• 14 years ago

I dunno, it could be entangled...

joblessjunkie2• 14 years ago

FermiLab has been offering HotBits for a very long time:

http://www.fourmilab.ch/hotbits/

The site contains some interesting discussion around randomness and how their service works.

ScottBurson• 14 years ago

Yes, but, uh ... Fourmilab is not Fermilab :-)

joblessjunkie2• 14 years ago

Ouch, that's a bit embarrassing...

acqq3• 14 years ago

People, if you need random numbers for some application of yours, you shouldn't get them from their server, period. See alternatives in the posts here. The most modern example, apparently the following hardware instruction will be available in all next Intel processors: http://en.wikipedia.org/wiki/RdRand

The main article is just to appreciate the pure awesomeness of us being able to actually see something that's a result of quantum fluctuations! In vacuum!

EGreg• 14 years ago

Perhaps they are only truly random until they are observed. After which they become deterministic. :)

jerf• 14 years ago

That might sort of be funny if it weren't simply true, and not in any sort of funny-true way, either. Random numbers generated in the past are deterministic. Of course they are. What would non-deterministic-yet-already-generated numbers look like?

marshray3• 14 years ago

I see a few people mentioning applications of this in cryptography and I feel that this is harmful.

You do not need exotic hardware to collect entropy to generate crypto keys. You need a correctly designed cryptographically secure PRNG which has had time to accumulate 100 or so bits of entropy (i.e., unknowability to the attacker). It's nice if it can be freshened every now and then with more entropy, but that really only should make a difference if you lose a backup tape and by some miracle the private key wasn't on there already. After it's warmed up, it should never need to block due to 'depleted entropy'. No one has ever broken such a CSPRNG. This is how the OpenSSL, OpenBSD, and FreeBSD /dev/randoms work.

On the other hand, what does tend to get broken regularly are overcomplicated and overengineered RNGs like this beast.

The one on the new Intel chips is handy, but it's largely overkill. We only need 100 bits now and then, not a steady rate at Gb/s (which can shut off abruptly when an attacker sharing your cloud hardware node decides to starve you of them). It's also not something that we can review for backdoors, so we should be reluctant to use it as the sole source of entropy.

A quantum vacuum random server is great like the way an internet-connected coffee pot is great. Fun, but not useful in production.

tptacek3• 14 years ago

One wouldn't want to give the impression that CSPRNG design is a solved problem, but I think you're right: exotic "true random" entropy sources definitely don't target the real challenges.

marshray3• 14 years ago

I believe it is largely a solved problem. Or at least the unsolved parts are reduced to a single problem: entropy input estimation.

JoachimSchipper2• 14 years ago

Are you aware of any instances of the OS /dev/Xrandom failing in an exploitable way, assuming proper initialization? I'm sure that people still mess up, but there are a lot of problems that are solved in that sense.

acqq3• 14 years ago

On Linux, the need for hardware sources became more visible because of this:

"/dev/random vs /dev/urandom" http://www.onkarjoshi.com/blog/191/device-dev-random-vs-uran...

/dev/random is hardware entropy dependent, and if application users use it too much (and we all know that many application developers don't do what's minimal stress for hardware or what's optimal "I just make it fast, it works for me"), every such application can block too often. Especially servers are problematic because you can't accumulate the entropy by measuring keyboard or whatever. That's why Intel made RdRand. And their competitors made some similar solutions even before that.

Good detailed discussion also on: http://en.wikipedia.org/wiki//dev/random

In short, hardware RNG in the CPU is a very good thing.

Of course, whoever doesn't need cryptographically secure randomness should anyway just use plain old pure software pseudorandom number generator:

http://en.wikipedia.org/wiki/Pseudorandom_number_generator

marshray3• 14 years ago

ln -sf /dev/urandom /dev/random

OpenSSL, OpenBSD, and FreeBSD agree with me.

Web server offers true random numbers via fluctuations of the quantum vacuum

Related Stories