Likely the most painful password hashing discussion you'll ever read

https://scott.arciszewski.me/misc/Linkedin/Fail.html

by sarciszewski • 12 years ago

44 110 12 years ago

scott.arciszewski.me

sarciszewski24• 12 years ago

Warning: This might ruin your day. Also, don't let tptacek see this or he might get very [angry/existential] (select appropriate).

opless2• 12 years ago

wow. just wow.

Then I noticed "PHP" and then I was enlightened.

sarciszewski24• 12 years ago

I'm of the opinion that PHP's reputation for being a "bad language" is due to the sheer volume of idiots who can write a script that doesn't parse error.

Alupis5• 12 years ago

I'm floored by the amount of people with "fancy" titles from supposed "consulting" companies in the link discussion.

The question was "I want to store password[s] in encrypted form." The answer should have been immediately -- "you don't encrypt passwords, you hash passwords using a well known, established, and tested cryptographic hashing algorithm + a salt, damnit!"

sarciszewski24• 12 years ago

Yeah, I attempted to do that (politely) with my first post. Though I made a few mistakes (it's password_hash() not hash_password()). As the discussion grew on, I became less nice.

owenmarshall8• 12 years ago

The answer should have been immediately -- "you don't encrypt passwords, you hash passwords using a well known, established, and tested cryptographic hashing algorithm + a salt, damnit!"

Your answer is as wrong as anything posted on that discussion.

The correct answer is that you don't encrypt passwords. You also don't hash passwords with a salt.

Instead, you use a key derivation function such as PBKDF2, bcrypt, or scrypt.

DanBC• 12 years ago

I'm floored by the amount of people with "fancy" titles from supposed "consulting" companies in the link discussion.

I'm gently curious if black hats will now target some of those companies?

Or are there enough low-hanging fruit available from www searches for them to not worry?

cyphunk• 12 years ago

in general anything coming from a php developer should be ignored. even reading official php documentation when it comes to anything concerning security should scare the shit out of grandmothers everywhere.

sarciszewski24• 12 years ago

The documentation has actually been going through a clean-up lately. For example, http://php.net/manual/en/function.crypt.php#refsect1-functio...

Do you have any specific examples of bad security advice in the documentation? I can try to raise hell with the doc maintainers and maybe even suggest an amendment.

FraaJad• 12 years ago

The first thing i did after reading the post was to check the official documentation. The docs are explicit about not using MD5 for hashing password. Good on PHP doc maintainers.

mercurial5• 12 years ago

I see that MySQL's documentation [1] recommends SHA2:

Exploits for the MD5 and SHA-1 algorithms have become known. You may wish to consider using one of the other encryption functions described in this section instead, such as SHA2().

Further down:

The PASSWORD() function is used by the authentication system in MySQL Server; you should not use it in your own applications. For that purpose, consider MD5() or SHA2() instead. Also see RFC 2195, section 2 (Challenge-Response Authentication Mechanism (CRAM)), for more information about handling passwords and authentication securely in your applications.

1: http://dev.mysql.com/doc/refman/5.5/en/encryption-functions....

orf• 12 years ago

> "MD5 is fine as long as your database doesn't get leaked and is the subject of offline cracking (like adobe)"

Wow. So MD5 is fine as long as it isn't needed to protect the data. Looks like a typical PHP discussion thread: plenty of misconceptions, a mix of of bad and plain stupid advice with a pinch of SQL injectable code. Silly PHP developers.

danielweber5• 12 years ago

To be fair, there are a class of problems you solve by storing your passwords behind a simple one-way function instead of as plaintext. I was asked earlier this week to audit a system that made the old passwords available to the user on their profile page. Using md5(), for all its shortcomings everyone has mentioned here, would at least make this architecture mistake impossible.

lttlrck• 12 years ago

I'm disappointed nobody suggested XOR

sarciszewski24• 12 years ago

I actually saved this before the grand-daddy of bad advice was posted. http://www.cryptofails.com/post/87697461507/46esab-high-qual...

mercurial5• 12 years ago

What about rot13?

hydrogen18• 12 years ago

rot13 is obsolete. You need to move on to rot39.

CaveTech• 12 years ago

But someone did suggest Base64!

mercurial5• 12 years ago

Multiple someones, including someones with fancy titles.

tedunangst3• 12 years ago

Base64 does have one use: assume you need to store a plaintext password because you need your service to login to another service. (Use oauth, yada yada. Not everybody has it.) base64 coding passwords prevents support staff from trivially seeing and learning passwords when poking about the database. It's not to protect the password from bad people, it's to keep the good people from accidentally learning something they don't want to know.

DoggettCK2• 12 years ago

I really want to suggest XORing the password with itself, as a goof, but it seems like the type of thing that will bite me in the ass later in life.

wglb2• 12 years ago

Trust me, someone very likely would take you seriously.

Then later credit you for your advice.

kazinator4• 12 years ago

"Do not use MD5" is repeated often in the discussion.

But are the weaknesses in MD5 relevant to hashing someone's 17 character password?

The problem with MD5 is that it's possible to alter a document such that the forged document still has the same digest. This is a problem for digital signing.

That doesn't seem directly relevant to password hashing. Nobody wants to find some 596 character string which has the same MD5 hash as your 17 character password, even if it does work in its place.

Your password isn't a document that is being digitally signed by its hash for the sake of certifying its authenticity. Nobody is even supposed to store that document!

(Someone tell me where I went wrong, if so.)

sarciszewski24• 12 years ago

The problem is, simply, that cryptographic hashes are very fast and not meant for user-generated passwords. For that, a special sub-class of cryptographically hashes (called Key Derivation Functions) are recommended to make brute-force computation much more difficult.

KDFs are meant to require a great deal more computation (e.g. an example of PBKDF2 might iterate HMAC-SHA256 8,000 times).

kazinator4• 12 years ago

But then "do not use MD5" is still a cluelessly strong statement.

It should be: "do not use any un-iterated message digest function for passwords: password hashing functions iterate a basic hash through numerous rounds, perhaps thousands; and some even encode the number of rounds as part of the salt, so the rounds can be increased in the future without changing the function, so it still recognizes old passwords."

laumars2• 12 years ago

You raise some good points, but the issue with MD5 are largely down to speed.

MD5 is fast - and while computational speed is normally your friend, with cryptography it's your worst enemy. Simply put, the faster the cipher, the more break in attempts a cracker can have per second.

This is why password strengths are often represented in duration (ie how many years would it take to crack). Though those figures are hugely estimated - to the point of being practically worthless.

kazinator4• 12 years ago

But other digest algorithms are also fast, so the argument is stupidly strong; it should be "do not use any fast message digest algorithm in its naked form for password hashing".

ceejayoz• 12 years ago

That doesn't seem directly relevant to password hashing. Nobody wants to find some 596 character string which has the same MD5 hash as your 17 character password, even if it does work in its place.

A SQL injection vulnerability lets you get the MD5 hashed passwords out of a database. As most people don't use 17 character passwords, you can check the MD5 hashes against a rainbow table (you can even do it online) and probably find a privileged user account pretty quick.

I actually broke into a legacy system my predecessor programmed in this manner when I couldn't find documented admin credentials.

peterwwillis4• 12 years ago

MD5 is vulnerable to collision and preimage attacks, which is bad for your password hashes, period. Other functions provide additional computational difficulty in doing exhaustive search of the keyspace. Even more functions prevent hardware implementations from speeding up the process by requiring excessive amounts of memory. This is why everyone suggests using bcrypt or scrypt, or if they're not available, PBKDF2.

danielweber5• 12 years ago

What's the smallest message anyone has gotten MD5 collisions on?

tedunangst3• 12 years ago

How are collisions and preimage attacks relevant for password hashes?

ownagefool• 12 years ago

Just suggesting you hash a password is pretty poor form as it'll be easy generate a rainbow table for, and likely many already exist.

You use a salt to mitigate this but even if you use a salt, it _has_ to be stored somewhere. Once an attacker has it, they'll be able to create rainbow tables with this in mind.

You then move to using a different salt for every password so any rainbow table only works for one result but if you use a fast hashing algorithm, attackers will still be-able generate rainbow tables based on those salts.

So you pick a slower hashing method but in the future, as computer hardware advances, you'll be in the same situation where it's trivial to brute force some rainbow tables.

So you need to keep on top of it and keep updating the stored hashes with something that is non-trivial to brute force, in PHP this gives you these functions: http://php.net/manual/en/ref.password.php.

In reality though, in the event of a breach, all you're doing is buying time for your users to change their passwords before the relevant rainbow tables are generated, so the next important step is to let your users know their hashed passwords were leaked, you followed best practices, but they should update the passwords wherever they used it.

Someone1234• 12 years ago

Let's make the following assumption: That no matter what algorithm we pick we're utilising per-user salts. So you'll always need to search the full keyspace (e.g. 6-9 character length passwords).

The reason why MD5, 3DES, and similar algorithms aren't really recommended in 2014 is that they're very fast/efficient on modern hardware. This is due to utilising GPU cores to do the hashing (which wasn't done back when MD5 was popular). This means the "time to crack" is low.

Here is benchmarks of different hashing algorithms (faster is WORSE): http://thepasswordproject.com/oclhashcat_benchmarking

But the really nice part about things like BCrypt and PBKDF2(Rfc2898) is that they accept "rounds." Which is number of repeated iterations over the password+salt. So every year you can respond to faster hardware by adding more rounds (e.g. 2014, have 10K rounds, in 2020 have 15K rounds, etc). If MD5 had supported rounds, it might still be safe to use today with enough iterations.

Even things like BCrypt and PBKDF2 will eventually fall by the wayside (MD5 used to be a more secure alternative to the much faster DES algorithm). But at least rounds (additional iterations to respond to changes in hardware) will keep them relevant for longer.

marcosdumay3• 12 years ago

Technically, no, it's not relevant. (Things change if you use "only" 13 characters.)

In practice, we don't know how crypto analisys will evolve, and non-broken algorithms are free for use, and just as easy as MD5. So, why do you insist on using a broken one?

danielweber5• 12 years ago

If your 17 characters are distributed equally along just the letters a-z, no one can build a useful codebook during their lifetime.

If your 17 characters are 3 lowercase non-archaic English words, maybe with two digits at the end, then a codebook for them already exists in MD5.

higherpurpose• 12 years ago

Why mention md5 at this point, or even SHA1, when BLAKE2 exists? (assuming the people there don't want SHA2 because of slower performance): https://blake2.net/

mercurial5• 12 years ago

Well, that's kind of the point. You don't want performance here.

tptacek4• 12 years ago

Nothing in that thread annoyed me as much as this comment does. Yes, BLAKE2 is trendy, and name-dropping is tempting. But BLAKE2 is also not a password hash, and can't safely be used to store authenticators for passwords.

peterwwillis4• 12 years ago

Right; it's a cryptographic hash. Technically there's no such thing as a 'password hash', only password KDFs and their results, which are composed primarily by cryptographic hashes. I think the guy's point was that if you're going to build a password KDF, you might as well use a hash function that's less shitty than MD5 and SHA1.

milesf• 12 years ago

The title of this post is lost on me. What's so painful about this discussion? It's not that I'm stupid, it's just that I'm ignorant and don't know.

sarciszewski24• 12 years ago

It's painful because:

- Bad advice is posted everywhere.

- Good advice is disregarded because nobody reads the preceding posts in the discussion.

Alupis5• 12 years ago

For the people not understanding why this discussion is a problem, in synopsis:

In order for a server to use encrypted passwords for authentication, the server must have access to the decryption key -- which means an attacker has access to your decryption key if they can compromise your box. Furthermore, if they don't get your decryption key, but they get your database dump -- they can then just brute your db and decrypt the passwords. They can do this because encryption is, by design, reversible. Encrypting passwords is almost as bad as storing in plain text.

Cryptographically hashing a password is the only way to go. When you cryptographically hash a password, using a known library that has been tested and written by actual cryptographers with passwords/authentication in mind (like bcrypt, etc -- no, sha's and md5's are not safe to use) - you get a one-direction algorithm in that a password goes in, is digested, and a fixed-length hash output comes out. There is no way to reverse this hash back into the original input. Adding salt(s) combats people making rainbow tables against your implementation, as they then would need to know not only which algorithm you are using, how many iterations it does, but also what your salt(s) are (sometimes even varied every so many iterations). This makes it almost impossible for someone to brute your leaked password database.

sarciszewski24• 12 years ago

Thank you for posting this. +1

aerovistae9• 12 years ago

I was under the impression SHA256 is valid...it's a cryptographic hash function that hasn't had weaknesses identified yet. 40 iterations of that + salting would suffice, no?

aerovistae9• 12 years ago

I do realize bcrypt is arguably the best option, but why is SHA not valid also?

Alupis5• 12 years ago

the SHA algorithms were designed to digest data as fast as possible -- which things like bcrypt purposefully slow it down plus can perform multiple iterations with the output of one iteration feeding as the input to another.

Here is some more info: http://forums.udacity.com/questions/6016855/hashing-password...

sarciszewski24• 12 years ago

If you're going to use SHA256 for hashing, why not use it in PBKDF2-SHA256?

https://defuse.ca/php-pbkdf2.htm

Sir_Cmpwn• 12 years ago

To expand upon that...

The problem with md5 and sha and such is that they are fast. bcrypt is useful because it takes a fixed amount of complexity which can be specified by the programmer (and later increased). It also uses salts to thwart rainbow tables. The complexity bit is important, though, because as computers get better, you can make it harder to reverse your passwords.

Also, the biggest reason any of this is a problem is because password reuse is rampant. You are being entrusted with the password to your user's life - their email, their bank account, their facebook account... treat it with respect.

tptacek4• 12 years ago

This summary created more questions than it eliminated.

Nobody has explained this better than Coda Hale:

http://codahale.com/how-to-safely-store-a-password/

(in place of "bcrypt", you can substitute "PBKDF2" [weaker] or "scrypt" [stronger], but all those options are just fine).

aerovistae9• 12 years ago

I'm a bit confused. This article says SHA-256 is a general purpose hash algorithm, but wikipedia says it's a cryptographic hash function. Is the wiki article wrong?

pinkyand• 12 years ago

True, that's the tech behind it.

Is there any reason most programmers should know any of this ?

The code should look more like this(and should use a standard library):

import Password_protector

P = Password_protector(date_of_first_use) #library upgrades algorithms every once in a while, only a single algorithm available per date. If there's a need for automatically upgraded hashing library should handle covertly.

protected_password = P.protect_password(password)

if P.compare_password(password, protected_password) == True ...

Or something similar. That at least should cover the basics - safely.

sarciszewski24• 12 years ago

I suppose that's Python? The whitespace got chewed up by the comment system; please pastebin/gist it instead.

mikeash2• 12 years ago

Quite simply, the reason programmers should know about this (if they're implementing a password system) is because the library you propose often doesn't exist, and when it does exist it's often implemented incorrectly.

In an ideal world, programmers shouldn't have to worry about this stuff. But this is not an ideal world.

aqme28• 12 years ago

"either you can use md5 or base64, there are also many other encryption are there.. md5 is very secure because it cannot be decrypted."

I'd say this has to be a joke, but Poe's Law is a tricky thing.

sarciszewski24• 12 years ago

I'm pretty sure a couple of the posts here are tongue-in-cheek, but most of them reek of cluelessness and tl;dr

waitwhatt• 12 years ago

It's linkedin. As expected.

mobiuscog• 12 years ago

The discussion is great - it shows all of the companies you should absolutely avoid doing business with, as they have no sense of security.

boogerhead• 12 years ago

Could a hacker learn the hash key and salting by opening a bunch of accounts on the site before stealing the database, then comparing his known passwords with the hashed values?

loumf• 12 years ago

Salts are usually dependent on user details (like the username), so knowing your salt doesn't help you with other passwords.

squidmccactus• 12 years ago

What about this is so bad to begin with? It looks like this is standard industry practice now?

Likely the most painful password hashing discussion you'll ever read

Related Stories