Randomly-generated passwords still have to be legal strings

https://blogs.msdn.microsoft.com/oldnewthing/20160316-00/?p=93163

by ikeboy • 10 years ago

69 88 10 years ago

blogs.msdn.microsoft.com

alyandon• 10 years ago

Is there really any need to generate non-alphanumeric passwords as long as they are of sufficient length? Sticking with plain old A-Za-z0-9 seems like it avoids a lot of interop issues.

Cthulhu_• 10 years ago

True, but I'm also pretty sure that there's no brute force password guess-o-matic that tries every possible UTF-16 character; they too will stick to the default keyboard-entered combinations because there's very few passwords outside of those.

jfoutz4• 10 years ago

Every extra supported character makes the solution space much bigger. with 2 character passwords, your set is 3844 combinations. If you can manage to add in "!@#$%^&*()", that goes up to 5184, a 16% increase in options leads to a 34% increase in search space.

No, it's not necessary, unbounded length is probably better. But it's pretty nice to have a lot more characters, and ought to be an easy win.

Retric5• 10 years ago

Every required charter type makes the search space dramatically smaller. https://xkcd.com/936/

The goal should be long easy to type and remember passwords, adding an extra few digit is worth far more than forcing passwords to look like: 2K#@1j*

Compare say a 20 digit numeric password vs an 9 digit [0-9] [a-z] [A-Z] and special.

ghayes• 10 years ago

Easy is relative to the experience. If we're talking (26+26+10)^100 versus (26+26+10+10)^100, the difference is moot in terms of brute-forcing the password. However, the inability to double click select, to paste in to chat rooms, etc may change based on which characters are allowed. These properties may be desired (or not) for your organization.

lmm• 10 years ago

It's not an easy win compared to just adding more length. Yes adding those characters to the set would increase the search space by 34%, but going from 2 characters to 3 (with the original set) would increase it by 6100%.

ajuc• 10 years ago

A-Za-z0-9 != alphanumeric.

My alphabet last letter is "ż" not "z".

\w = alphanumeric

Some people don't even have latin letters on their keyboards by default. And "latin letters" aren't well defined anyway. For one thing technicaly "w" isn't a latin letter.

I'm of the opinion that all restrictions on character set in password are unnesessary and harmful. You shouldn't store plain passwords anyway. You shouldn't even pass the plain password from client to server. Why do you care what character set your customer uses?

IMHO the main reason to restrict passwords' character set is to annoy users by forcing them to make another variation of their favorite throwaway password for your criticaly important blog comment section.

falsedan• 10 years ago

  > For one thing technicaly "w" isn't a latin letter.

Don't be silly. Yes, I know how Romans pronounced 'ave'.

sugarfactory• 10 years ago

I quite agree. However I have seen a great many registration forms that only accept a short password. If passwords are handled properly there's no reason to limit password length because a password will get converted to a fixed-length string due to hashing before storing it to a database. So I suspect many people prefer storing passwords in plaintext.

JoeAltmaier2• 10 years ago

Another solution is to treat every password as binary. Text works. UUIDs work. Everything works. But folks went down the text-password road a long time ago, so we're stuck with this for now.

dchest• 10 years ago

Passwords-based systems assume input, and most UI frameworks use text strings for inputs.

Binary password is usually not called a password, it's a key.

smsm422• 10 years ago

Exactly, so if they are not meant to be input, as the article claims, why are they passwords and not keys? Why they are even checked to be valid unicode strings? I know of no encryption algorithm that cares about key being a valid unicode string. Looks like just laziness in distinguishing which data should be binary and which should be Unicode string.

eterm2• 10 years ago

I second the "hex encode it" solution, 256 bit of generated hex looks something like this:

232e2252b8d5a294dd8564020f0cf609f511b3d196ed83030ce9615c6737f578

That's readable, typable, copy-pastable. Not very memorable but 256 bits is rarely memorable.

It's still short enough to send around by normal messaging.

This is specifically about randomly generated passwords, this is not about dealing with human generated "must have capitals, not start or end with a number" etc rules.

delinka• 10 years ago

I argue with 'typable.' If I'm reading this from my smartphone and typing it on my computer (or vice versa), I'm going to lose my place. Maybe formatting could fix that, but it's still a pretty long chunk of text to get lost in.

Gracana• 10 years ago

Two bytes at a time is not so bad, especially with some friendly newlines:

    232e 2252 b8d5 a294
    dd85 6402 0f0c f609
    f511 b3d1 96ed 8303
    0ce9 615c 6737 f578

Of course, it's still a lot of characters.

Spooky232• 10 years ago

Remember on windows you are effectively stuck with a maximum password length of 15 characters in many scenarios.

And before you Google and say "No! It changed to 127 in Windows XP!", remember:

- Somewhere on your network, NTLM is being used.

- Office 365 supports 16 max.

- Lots of random apps won't work.

monochromatic• 10 years ago

I've been using 20+ character Windows passwords for years and have literally never had a single problem with it.

kyberias• 10 years ago

I think you're wrong.

I login to Office 365 with a password that has over 16 characters. I have similar length passwords in Windows networks that use NTLM. Clearly it works just fine.

Godel_unicode4• 10 years ago

Dangerously incorrect (also, the two magic #s are 14 for ntlm and 16 for o365; where did you get 15 from?)

1 - if you have ntlm (non-v2) on your network, you're fucked anyway and this debate is academic. 2 - even if you do, having a password of 15 or more characters will cause your legacy hash to be set to "AAD3B435B51404EEAAD3B435B51404EE" and will protect you against having your hash cracked if it's stolen from the non-v2 service. 3 - the office 365 limits can be worked around with dirsync or federation (source: I've done both)(note: outlook does weird things with leading/trailing spaces) 4 - those apps should be kerberized or similar

jsprogrammer• 10 years ago

Why not base64? Even shorter, readable, typable, copy-pastable (why are you copy-pastung passwords?), and pehaps even memorable.

falcolas4• 10 years ago

Indeed.

    $ dd if=/dev/random bs=1 count=32 | base64
    ca2ktYYvppr1/HwwGYrg01udQX1cP4Ek5VmyzQydT+8=

sp332• 10 years ago

I copy-paste passwords from a password manager. Usually it puts it right in the clipboard for me though, so I don't have to highlight it manually.

matt_wulfeck2• 10 years ago

I always loved hex encoding because you can always click it twice to highlight the entire string. It's a simple thing, but always saves time and reduces errors.

Different systems have different characters that will be included/discluded from "twice click highlighting" (I have no idea what the term is called). Hex works on virtually all systems.

The article does mention it's not very space efficient.

adzm• 10 years ago

Ascii85 is a great compromise between binary and base64, which works well for passwords.

13of40• 10 years ago

Throw in the complexity requirements a Windows domain imposes on the password, and this turns into even more of a headache. Of course you can just append A1! to an otherwise sufficiently complex password to meet the requirements, but it will make people freak out in the code review.

teraflop• 10 years ago

We generate our password from a cryptographically secure random number generator. ... We found that sometimes (no predictable pattern), we have interoperability problems between systems.

If there was a predictable pattern, that would be even worse!

thecatspaw• 10 years ago

well, no. a predictable pattern would be "every time the password starts with 0100101 they are not compatible". its not about the number having a pattern

lyle_nel• 10 years ago

By just looking at the title and before reading the article, I though they meant that they are checking for illegal strings that are illegal to transmit. https://en.wikipedia.org/wiki/Illegal_number

pmontra• 10 years ago

I hope the developers of those sites that insist that passwords must be, let's say, between 6 and 18 characters are reading this.

DenisM3• 10 years ago

It has occurred to me recently that 12 random words from a 2000 word dictionary will give you 132 bits on entropy. More that you will ever need.

There's a number of advantages to this scheme:

- It's easy to write down and put in a safe.

- No funny characters required.

- Not case-sensitive.

- If a user makes a spelling error, we can correct it for him without any compromise in security.

- Guaranteed to be free from offensive words, unlike a randomly generated password.

- People are using words anyways, misspelled if necessary, whether we like it or not.

We can also make it even more user-friendly by carefully degrading security, such as trying common word substitutions for the forgetful users, or doing some kind of middle-ground "remember me" that does not let you straight in, but only demands first three words (to keep the family members at bay). Like I said, this required careful calculations as to how much security is sacrificed when we do that.

oniony2• 10 years ago

This was made clear many moons ago by the now famous xkcd strip https://www.xkcd.com/936/

DenisM3• 10 years ago

My innovation :) is to prevent users from selecting their own words, so that we can actually guarantee the entropy required. Otherwise a Markov chain will tear it to pieces.

munificent2• 10 years ago

Alas, many many many sites have inanely short password length limitations.

oniony2• 10 years ago

All of these systems suffer from the differing password restrictions imposed by the various services you need to register for.

• Some services require both upper and loweracase characters.

• Some services require punctuation characters whilst others prohibit them.

• Some services prohibit repeated characters.

• Some services impose a minimum password length; some impose a maximum password length; some impose both.

Covering all these situations is nearly impossible.

eterm2• 10 years ago

The first line of the article makes it clear this is for passwords used in programmatic use.

These restrictions exist for when humans are allowed to generate passwords. This article is discussing systems where humans don't generate passwords and don't in regular operation enter them.

Think of automatic password generation for a secure API endpoint, or automatic generation of private keys secured by a password that typically will be used in a system with it on the keychain but where the backup copies will be stored offsite.

In that case it's acceptable to have a password which is both secure but it has to be easy to type when required but doesn't have to be easy to type (i.e. short) for regular use.

The internet of things means that more and more devices will be creating "users" which are actually just other IoT devices. Those devices/device-users require these sorts of passwords.

munificent2• 10 years ago

News flash: taking a randomly generated sequence of bytes and treating it as correctly following some encoding may fail.

This is like saying "Why isn't this random sequence of bytes a valid JPEG file?" If you want a random image, you generate the image randomly, then you encode it. Strings are no different — though the encoding is simpler.

mikeash3• 10 years ago

Unicode mystifies a lot of people. Many people don't even realize that a sequence of 16-bit unicode value is UTF-16 (or UCS-2), let alone that not all sequences of 16-bit values are valid UTF-16.

The password bit is really just the hook, the main info in this article is the fact that these values are interpreted as UTF-16, and that UTF-16 has requirements beyond "sequence of 16-bit values."

Which is kind of obvious, but still not well known for many.

theophrastus• 10 years ago

I'll freely admit to being one of those mystified folk. As an easy example of which, can you roughly outline which n-bit values are not valid UTF-n? (are they reserved for shifting to other systems? and if so, isn't that still a valid, even an important value, within UTF-n?)

sdegutis• 10 years ago

What doesn't make sense though is why the password's bits MUST be a valid UTF-16 string. Password comparison should just be comparing bits and checking if they're equal.

Whether it may just so happen to represent real, correctly typed, English words, or be a valid ASCII string, or UTF-16 string, or whether it was typed it from your keyboard, or generated from a program, these should also be incidental.

dllthomas• 10 years ago

In the vein of "unicode mystifies people", one interesting fact is that converting a unicode string to lower-case is not length-preserving - whether you're counting bytes (in any of the standard encodings) or characters!

accounthere• 10 years ago

It has happened to me that a registration form would not allow characters like '<' which any keyboard will most likely have, while it allowed unicode characters like (ಠ_ಠ)

_kst_2• 10 years ago

Sounds like a clumsy attempt to guard against HTML injection.

matt_wulfeck2• 10 years ago

The article mentions that hex encoding solves most of the problems but isn't very space efficient. A good alternative may be base64k encoding, which was proposed here on hacker news[0].

This gives you safe strings as well as space-efficient encoding. As a bonus virtually all systems support unicode without any tweaking necessary.

I hope the author of base64k gets around to writing an RFC because it would be a great standard to use.

[0] https://www.npmjs.com/package/base65536

Randomly-generated passwords still have to be legal strings

Related Stories