Show HN: WordPass – password generator giving over 90 bits of entropy

http://hackage.haskell.org/package/wordpass

by miga • 12 years ago

41 51 12 years ago

hackage.haskell.org

dasil003• 12 years ago

I don't understand how 4 random words from a 50k dictionary gives 90 bits of entropy.

    50k ^ 4 => 6.25e+18

    2 ^ 90  => 1.24e+27

Am I missing something?

saurik• 12 years ago

The words are separated (and I guess terminated) by a random symbol (of which there are apparently 32 possibilities) or (randomly) a random two-digit number. log(50000^4*(100+32)^4)/log(2) = 90.6

ghshephard3• 12 years ago

Doesn't that defeat the "Trivial to remember" part of the xkcd concept of "Trivial to remember, hard to guess?"

atmosx• 12 years ago

Can you offer a link on how do you do these measurements and what exactly do they mean?

Reading the wikipedia article about entropy in (computing) I stumbled across this[1]:

" Around 2011, two of the random devices were dropped and linked into a single source as it could produce hundreds of megabytes per second of high quality random data on an average system. "

What does it mean high quality random data in this context?. I mean 'random' is a precise definition, something 'is random' (e.g. distance of next prime :-P, kill Riemann) or 'not random' (distance of next prime if Riemann's ζ(s) is solved).

[1] https://news.ycombinator.com/user?id=dasil003

tinco2• 12 years ago

That's random in maths, we're not talking maths here. In the real world, random is a value that's hard to predict. For example a dice throw has a high quality of random in normal circumstances.

A low quality random source would depend on some variables that make it easier to predict the next random number (perhaps just statistically). The worst quality random source would be pseudo-random numbers, where the value of each next random number directly follows from the previous value and perhaps some secret second value.

It might be distributed like a true random number, but eventually the secret could be guessed and the numbers would become predictable.

darkmighty• 12 years ago

A simple explanation is that "randomness" here is related to (logarithm of) the minimum average time [1] you would take to guess the password correctly (H=sum(pi * log(1/pi)). Since the random number generator supposedly gives an uniform distribution over a set of words, this simplifies to H=n(1/n log(n))=log(n).

[1] Your average time is at least t* =1/2 * 2^H, so log( t* )>H-1.

ef42• 12 years ago

"Distance of next prime" is not random, it's entirely deterministic. It's a well-defined function. Just because we don't have an analytical solution for it doesn't make it random.

As for "high quality", some random variables are less predictable than others. A weighted coin that comes up heads 51% of the time is still random (not deterministic), but more predictable than a fair coin.

eliteraspberrie• 12 years ago

The information theory (mathematical) definition of randomness is explained nicely in Chapter V of An Introduction to Information Theory:

http://www.amazon.com/Introduction-Information-Theory-Symbol...

In the particular case of a series of n equally-probably, independent events, the entropy is given as H = - log 1/n, measured in bits. For example, the entropy of a fair die is - log 1/6 = 2.58 bits per throw.

In this case, the random event is words chosen from a word list. Four words are chosen from fifty thousand, with each word having equal probability of being chosen. So the entropy (measure of randomness) is - log 1/50,000 = 15.6 bits per word, or 62.4 bits per four-word combination. (The script also adds random numbers or symbols, to add up to 90 bits.)

CHY872• 12 years ago

Ugh, the other replies aren't quite getting at the crux of the matter. A function f : {0, 1}^k -> {0, 1}^n -> {0, 1}^m is considered to be pseudo-random if (when provided with a key K) there is no way of distinguishing between its output and the output of a random function F : {0, 1}^n -> {0, 1}^m in polynomial time (it's a little more complicated than that, but that's the gist).

In modern computers, the basic principle is that one generates a random seed (through heat noise, keyboard movements, mouse movements, user interactions with the computer etc) and then uses it as the key for a pseudo-random function such that the next output of the function cannot be predicted given all of the previous. One can simply iterate if one so chose (first n bits output = f_K(0), next f_K(1), ...).

In this case, he's simply worked by the fact that if one knows that their target is using this key scheme, one knows that at most there are 50^4 passwords the target could be using. If there are 90 bits of entropy, that means that there are 2^90 passwords that the target could be using. What seems to be the difference here is that this program outputs a special character between each word, or a 2-digit number, meaning that we have 50000^4*(100+32)^4 > 2^90. It's kinda messy, and probably not that easy to remember.

In general, with passwords, one wants about 2^80 bits of entropy - it's high enough to be incredibly difficult to crack even when hashed insecurely. Once we start using proper algorithms like bcrypt it's even harder.

To give a comparison, a password output by this is roughly as hard to crack as a 15 character random password encoded in ASCII, or if unicode is allowed, a 6 character password in that encoding.

Ruud-v-A• 12 years ago

I wrote a post about this some time ago: http://ruudvanasseldonk.com/2012/07/07/passphrase-entropy

ghshephard3• 12 years ago

  import random, re

  lines=[re.sub('[-!.,;:]',' ',x).split() for x in open('in').readlines()]

  w=set()
  for l in lines:
      w=w | set(l)

  random.sample(w,4)

Hrm, for romeo and juliet I get some interesting combinations...

  ['unseen', 'iron', 'Catling', 'baked']
  ['grove', 'press', 'Aurora', 'garish']
  ['agate', 'She', 'drybeat', 'rather']
  ['flowed', 'sails', 'wed', 'masks']
  ['spilt', 'cage', 'Remembering', 'stiff']
  ['heartsick', 'shame', 'enjoin', 'weeping']

wyager4• 12 years ago

import random, re

You'll want to replace random with a cryptographically secure RNG.

peteretep• 12 years ago

Could you describe a practical attack against this password generator based on how it actually works?

Specifically what information you would need to be in possession of in order to exploit it, and then a description of how you would - again, in practice - exploit that, inside a practical timeframe. Thanks.

Jach• 12 years ago

    from random import SystemRandom
    random = SystemRandom()

abecedarius• 12 years ago

Here is what I happen to have sitting around. (I'm not recommending it; there are much more secure ones. But it's short, and uses a crypto-random generator.)

    import math
    from random import SystemRandom

    cryptogen = SystemRandom()

    alphabet = open('/path/to/a/custom/wordlist').read().splitlines()
    assert len(set(alphabet)) == len(alphabet)

    def gen_token(length):
        return ','.join(cryptogen.sample(alphabet, length))

    nbits = 80
    length = int(math.ceil(nbits * math.log(2, len(alphabet))))
    print gen_token(length)

tinco2• 12 years ago

I made one in Ruby a few weeks ago. Please don't do what ghshephard does below and blindly import random. It's actually quite important that you use a secure random generator, or your entropy will be drastically reduced.

My solution is here, don't use it unless smart people have confirmed that it works:

https://github.com/d-snp/wachtwoord/blob/master/wachtwoord.r...

Also smart people: please let me know if I've done anything stupid here, I actually use passwords generated by this tool.

For people who don't enjoy reading Ruby: My tool reads in a dictionary file, drops any too small words because they don't have enough absolute entropy and too long words because they're harder to remember (for Dutch anyway, your mileage might vary in your own language).

You can vary the amount of words you want to have in your passphrase, and the tool will print how much entropy the password has if the attacker is aware that you're using this scheme and this dictionary file, but not aware of the random values spitted out by the random generator to pick the words.

It will also spit out a cheesy ballpark estimate as to how long it might take an attacker to crack this password with reasonable effort, you can tweak what reasonable effort means for your context. What currently is a reasonable attack depends heavily on the hashing scheme you employ.

keithpeter2• 12 years ago

https://www.schneier.com/essay-246.html

Oulipo rules. Latin squares. Nursery texts. Maps.

Anyone working on a generator based on this kind of approach?

miga3• 12 years ago

Could you elaborate and give pointers to what "Oulipo rules" and others are?

keithpeter2• 12 years ago

Oulipo was a silly 'organisation' invented by some writers in Paris after the second world war. The members included mathematics lecturers and writers interested in producing texts according to routines or with constraints. George Perec walked the streets of Paris systematically for some months according to a routine derived from a double latin square, and then wrote about what was happening at each location at a specified time.

I was alluding to the idea easily remembered rules for selecting characters from a text known to the person.

arh682• 12 years ago

This inspired me to write a rudimentary version in bash/C:

    ----pwgen.sh
    #!/usr/bin/env bash
    DICT=/usr/share/dict/words
    NL=$(wc -l $DICT | sed 's_[ \t]*\([0-9]*\).*_\1_')
    PW=""
    for i in {1..4}; do
    RAND=$(./arc4random)
    LINE=$[$RAND % $NL]
    WORD=$(sed -n $LINE"p" < $DICT)
    PW=$WORD" "$PW
    done
    echo $PW
 
    ----arc4random.c
    #include <stdio.h>
    #include <stdlib.h>
    int main() {
        printf("%u\n", arc4random());
        return 0;
    }

They're not always easy to remember, though, at least for me:

    > ./pwgen.sh 
    interdepartmental anesthyl intrabranchial physiolatrous

cynwoody• 12 years ago

Your little script performs amazingly well on my Mac, coming in at about 165 ms, quite a bit faster than I expected. If your script is too fast for you, you could consider reading the dictionary into a bash array instead of using sed. My /usr/share/dict/words contains 234,936 words.

As a point of comparison, here is a Python version that clocks in around 107 ms:

    #!/usr/bin/env python
    from random import SystemRandom as r
    
    with open('/usr/share/dict/words', 'rb') as f:
        lines = f.readlines()
    
    print ' '.join(r().choice(lines).rstrip() for x in xrange(4))

However, I don't know how well SystemRandom compares to arc4random in terms of crypto.

jzwinck• 12 years ago

If you're set on using arc4random it's no big deal to use ctypes to invoke it from Python. Bash is quick to bang out but it's funny how your parent needs to compile a C program then invoke it N times per bash script run. Using Python for this is an obvious choice.

euank3• 12 years ago

We need to just get away from remembering passwords. Something like Mozilla Persona would have been great, but since it didn't catch on we already have KeePass (and for mac/linux, keepassx2 alpha works quite well).

These cutesy little passwords might be fairly strong and easy to remember, but it hardly matters because having to remember them results in reuse and an upper bound on the possible entropy. The human mind has more important things to do than remembering thousands of bits of entropy (spread among all your passwords, you should have at least that much)... we made computers to remember this sort of stuff for us.

If these passwords aren't meant to be remembered, but put in keepass, then there's no point in not just upping the entropy by randomizing at the character granularity.

KeePass makes it much easier; you have it generate you 300 bits of entropy passwords (or whatever you feel is enough), and then you can focus all your security efforts on that one database file. Having a "single point of failure/password" doesn't have to be bad because it lets you guard more closely against that single point.

tomberek• 12 years ago

A similar scheme used to save, retrieve, and publish elm code: http://tomberek.insomnia247.nl:5055/xkcd/flowerbed/too/socio...

Using 'Publish' will produce http://tomberek.insomnia247.nl:5055/xkcd/2797191408452820889 which is just the output page without access to an editor.

Edit: go to http://tomberek.insomnia247.nl:5055/try to make your own

kolev2• 12 years ago

How do you install stuff like this? I've never touched Haskell, I tried "cabal install wordpass", but it seems it doesn't exist.

samstokes• 12 years ago

Assuming you mean wordpass doesn't exist: try "cabal update" first.

If you mean cabal doesn't exist, you'll need to install the Haskell Platform first. There may be a package for your distro (or Homebrew).

kolev2• 12 years ago

Thanks, "cabal update" did it for me. I guess I have no more excuses to dive into Haskell!

pjaspers• 12 years ago

I made something similar [0] but based on Diceware [1] and a book you pass on to the app. It generates passphrases by using words from the book.

[0] https://github.com/pjaspers/frasier

[1] http://world.std.com/~reinhold/diceware.html

robgering• 12 years ago

Quick and dirty Bash one-liner:

  shuf -n4 /usr/share/dict/words | sed "s/'s//g" | tr -d "\\n"; echo;

csirac22• 12 years ago

My favourite tool is simply gpw(1). It tries to generate semi-pronounceable words that aren't necessarily dictionary words:

  $ gpw
    rpreence
    rethersi
    atencend
    rostrass
    rtschers
    pocrevoy
    umblowfl
    disalsit
    rmsturnu
    tinfethe

I then have a perl script which can give me slightly more interesting capitalizations and numbers:

  nessinea 258 rusness
  redoodr rInGLe 893
  orinGsT 87 iNdaPeRv
  3423 RisEv screar bullys

I've become quite good at remembering these. I can usually remember them even if I go a month or two between usages. It's hard to explain, but it seems to require a different memorization effort that somehow sticks better into my long-term memory than with random dictionary words.

(1) http://manpages.ubuntu.com/manpages/hardy/man1/gpw.1.html

miga3• 12 years ago

Yes, it is critical constraint - if you find it easy to remember.

What's the size of random password space there?

csirac22• 12 years ago

You just prompted me to read the source code :-) gpw.c has this comment at the top:

  /* GPW - Generate pronounceable passwords
   This program uses statistics on the frequency of three-letter sequences
   in your dictionary to generate passwords.  The statistics are in trigram.h,
   generated there by the program loadtris.  Use different dictionaries
   and you'll get different statistics.

   This program can generate every word in the dictionary, and a lot of
   non-words.  It won't generate a bunch of other non-words, call them the
   unpronounceable ones, containing letter combinations found nowhere in
   the dictionary.  My rough estimate is that if there are 10^6 words,
   then there are about 10^9 pronounceables, out of a total population
   of 10^11 8-character strings.  I base this on running the program a lot
   and looking for real words in its output.. they are very rare, on the
   order of one in a thousand.

   ...

I'm not really in a position to properly figure this out... for a start, it's commonly claimed that there are 1M words used in English however my local /usr/share/dict/words has only 100K entries (and a even lot of those seem redundant).

Assuming my gpw binary was "trained" on a 1M wordlist, and that my algorithm averages three 8-char strings per passphrase... and we take the comments at face-value, i.e. 10^9 possibilities per 8-char string generated.

I'm going to pretend I can competently use Mlog2(N) to estimate that three such strings might make up 3log2(10^9)=89 bits of entropy. Throw in a 0-9999 number, let's pretend that's worth 13-ish bits, let's call it 100 bits?

I haven't factored in the capitalizations yet, and I'm not sure how much the random placement of the numbers and the random lengths of each "word" are worth.

IANACG (I am not a crypto guy) but that seems like a huge number of bits and so this is likely completely wrong (I'd start with going back to trying to properly analyze how many combinations my gpw binary can actually produces in an n-char string - that 10^9 combinations from a 10^6 wordlist needs testing).

Show HN: WordPass – password generator giving over 90 bits of entropy

Related Stories