Please stop hashing passwords

http://blog.tjll.net/please-stop-hashing-passwords/

by DanielRibeiro • 12 years ago

68 95 12 years ago

blog.tjll.net

NateDad5• 12 years ago

My guess is that anyone who still does this does not actually read technology blogs or Hacker News. The answer for a long time now has been "use scrypt or bcrypt". I think that answer still holds (IANAC).

kzrdude2• 12 years ago

Hopefully the people behind password-hashing.net will give us an up to date recommendation when their contest concludes.

rlpb• 12 years ago

I get the impression that, even on HN, after some high profile database breach, people still ask "were they using hashed passwords?" rather than "were they using a key derivation function and if so which one with what parameters?".

This tells me that the HN crowd still hasn't got it.

My own attempt at this message: http://www.justgohome.co.uk/blog/2014/02/salting-hashing-and...

NateDad5• 12 years ago

Certainly, it's a message that bears repeating. New devs enter the workforce every day.

vertex-four• 12 years ago

Or some variety of PBKDF2 if adhering to standards is more your thing, or you intend to market the security of your system to enterprise customers. The likes of Python's Django use this by default.

NateDad5• 12 years ago

Yes, right. Sometimes I think people (myself included) leave out PBKDF2 because it's hard to spell/pronounce. :)

tptacek10• 12 years ago

Enterprise customers do not care about PKCS "standards" and don't as a rule really know much about password hashing. A good reason to use PBKDF2 instead of bcrypt is if your password hash needs to double as an actual key derivation function. Other than that, I can't think of any authentic good reasons to use it.

It doesn't matter, though. PBKDF2, bcrypt, scrypt: all fine. Just stop using salted, peppered, or lightly-battered hashes.

tim333• 12 years ago

For quite a while pleantyoffish.com sent me regular emails including "remember you password is"... in plain text. They seem have stopped now. Some evil hacker could have stolen my ever so valuable list of potential dates.

ttflee• 12 years ago

The tl;dr from the article:

Stop associating “hashed” with “secure” when it comes to passwords. If you’re storing user credentials MD5-ed without salt, you’ve put an thumb tack in front of a steamroller - just a minor annoyance that won’t offer you any real security. You have to do it right or hashing means nothing.
Start relying on key derivation functions for password storage.[1] Are they perfect? Will they be the best practice in five years? Probably not, but they’re your best option when rolling your own user credential storage.
Don’t sacrifice security for performance. A slightly slower malloc in OpenSSL would have been much preferred, in my opinion, than the disaster that is Heartbleed.

- [1] There’s a fair bit of dispute over the ideal KDF for storing passwords, but generally speaking, any of the three covered in this topic are popularly accepted options.

DiabloD3• 12 years ago

Re: slower malloc, there is no actual proof of, say, modern glibc's malloc implementation is slower in any useful measurement of the term. Heartbleed, however, didn't happen because someone failed benchmarking 101, it happened because someone failed bounds checking 101.

kzrdude2• 12 years ago

The point that I and others want to make, is that regardless of the ultimate cause of failure, we want to criticize the whole design approach. Security in layers needs conservative coding style, both safe and sound internal APIs, bounds checking and an overly cautious memory allocator.

It's not just about finding and fixing bugs, but how to improve the whole process.

chrisrohlf• 12 years ago

It is frustrating that the confusion around heartbleed and the OpenSSL free lists persists: http://blog.leafsr.com/2014/04/11/my-heart-is-ok-but-my-eyes...

oleganza3• 12 years ago

The problem with expensive KDF on the server is that you are opening yourself to a higher risk of DoS. More computationally intensive KDF - higher protection against crackers, but easier for someone to DoS you.

Better approaches:

1) Let the client do expensive KDF (e.g. in JavaScript if it's a web app) and send you only the resulting key. You can then store SHA256(key) instead of KDF(password, salt).

2) Use public keys. This requires some fancy infrastructure like a keychains, keybase.io, PGP software etc, but it is the most secure solution. User saves only his public key on the server and signs authentication token when he wants to get in. Server has absolutely no secret material to leak, while the user has complete control over safety of his keys. This is how SSH works. The reason it is not widely used on the web is because we don't yet have nice infrastructure: nice UI for keychains and protocols for one-click sign-in via crypto signatures. We all should work on this to make it better for our kids ;-)

mschuster91• 12 years ago

1) Let the client do expensive KDF (e.g. in JavaScript if it's a web app) and send you only the resulting key. You can then store SHA256(key) instead of KDF(password, salt).

So $hacker has to simply do a rainbow-table attack... no security gained there, except for a vastly bigger key space.

crusso• 12 years ago

The reason for his suggestion was to prevent an expensive hashing algo from being a means for creating a DOS, not to make the rainbow table attack any harder... although if you offload an even more expensive operation to the client than you were going to perform on the server, it does make the attack harder.

taejo3• 12 years ago

The vastly bigger key space is all you need to make the rainbow-table attack completely beyond the realm of possibility (isn't it?)

matthewmacleod2• 12 years ago

Neither of these is a better option, in my opinion, because the complexity of setting them up and ensuring security far outweighs the risk of a DoS attack on your infrastructure.

It's relatively trivial to rate-limit access to particular endpoints if you are particularly concerned about DoS attacks, which means you've got a simpler system overall – an simplicity in security is reaaaaaallly worth it!

tptacek10• 12 years ago

People have been "worrying" about DoS attacks from password hashes for years. Strangely, they haven't become an issue in practice. My presumption is that this is because if you're going to launch an application-level denial of service attack against a particular application, it is already trivial to come up with one regardless of how you store passwords.

oleganza3• 12 years ago

People haven't ever deployed reaaaally hardcore KDFs on their servers yet. I myself would like my password stretched with >100 Mb of memory and taking about a second on 8 threads. This kind of setup might be too expensive for a shared server.

NateDad5• 12 years ago

If someone can DoS your login api, they can do it to another api. Rate limit it. It's not hard. It's actually even easier because people login so rarely, you can really ratchet down the rate and not mess up the casual user's experience.

drdaeman3• 12 years ago

Just curious. Isn't this is kind of KDF "abuse"? Because, for storing password information in databases, KDFs aren't used for key derivation (the generated key material is never used as a key for any cipher), but just as slowly performing [salted] hash functions.

If so, why say "don't use hash functions and use KDFs" when KDFs are used as hash functions in such cases? Maybe it's better to say something along the "don't use fast hash functions, use slow ones" lines?

tptacek10• 12 years ago

A hash function is a primitive. A KDF is a construction. The construction best used for storing passwords happens to have basically the same goal as the one used to stretch a low-entropy secret into a higher-entropy secret.

drdaeman3• 12 years ago

Thanks, that clarifies it.

But would it be (more) correct if we'd say "key stretching function" instead of KDF?

kitd• 12 years ago

I wouldn't say that KDFs used this way is "abuse" any more than hash functions used for secure storage is abuse. It all comes down to what you mean by "hashing".

The problem is that the same word "hashing" is widely used (incl by experts) for 2 different things, normal insecure hashing and cryptographic hashing. As he points out, the 2 have different purposes, ie CRC for large files & git blobs, etc for the first, and cryptographically secure storage of small strings for the 2nd.

KDFs meet all the requirements of cryptographic hashing. Maybe a new term is needed for the process of cryptographic hashing, like "scrambling", or something that suggests secrecy, rather than just generating a unique value.

jakobe2• 12 years ago

The biggest issue here is that users are allowed to pick their own passwords in the first place. Sure, you can require them to use passwords with a capital letter and with a number and with a punctuation character, but that will just make them pick "Password1."

Better: Use one time passwords sent via SMS. Or send a one-time-login URL via email.

If you do have to use a password, just generate a 10 digit numeric code. Sure, some of your customers might complain, but at least you aren't responsible for disclosing people's ebay password when your site gets hacked.

matthewmacleod2• 12 years ago

That's not "some of your customers might complain" territory, it's "your business failed because nobody signed up" territory.

2FA basically ensures security via a second channel, and it's perfectly possible to store passwords in a secure format. I'm not convinced your ideas there are worth the cost.

jakobe2• 12 years ago

your business failed because nobody signed up

Why? Almost all websites require email confirmation; sending someone a login-URL via email actually has less friction because the password-choosing step is removed!

it's perfectly possible to store passwords in a secure format

But it's very hard to do so. Even if you use scrypt, it is very hard to make sure your whole system is actually secure against password leakage.

The simple truth is that letting your users choose their own passwords is a liability; and I've decided to avoid this liability.

the_af• 12 years ago

Re: "Password1". There was an interesting paper, I think by someone from Microsoft, that argued that when users pick silly passwords they are actually being rational. They (the users) informally decide that the pain of overcomplex password schemes just isn't worth it. In other words, remembering passwords or using security-related programs and practices is a high price they have to pay everyday (while we computer literate people often disregard this cost, it is there), while the relatively uncommon security breach is something they often never see.

Maybe I'm misrepresenting what the paper states, but my takeaway from it was "don't assume users are dumb when they pick silly passwords. They simply are not willing to use an overcomplex system that for them turns out to be not worth the effort."

I just tried to find this paper online but I can't even remember the title :(

mnw21cam4• 12 years ago

Correct horse battery staple. http://xkcd.com/936/

We are told to not re-use passwords. This is not helped by every single shopping web site out there requiring an account (and therefore a password) in order to buy something. Fair enough for big sites like Amazon - I'm actually likely to come back at some time in the future, although I dislike the way it tries to store my card number each time.

On most sites, requiring me to create an account discourages me from shopping there. I'm not likely to come back unless I suddenly have a burning need for another obscure once-in-a-lifetime widget, so why do I need an account? If I do come back, you still only need my card number and a delivery address.

As it stands, the sheer number of accounts that I have means that I invariably set an impossible to remember password and immediately forget it, relying on the password reset mechanism. This is not ideal.

nichochar• 12 years ago

You don't talk about using a pepper also, combined with a Salt. A pepper is a constant secret you append before hashing, increasing entropy and globally making the guessing much harder. Webapp2 from google has a nice implementation, coming from werkzeug to be found here http://webapp-improved.appspot.com/api/webapp2_extras/securi...?

nialo• 12 years ago

This is deeply pointless. Any attacker that gets the password database is practically guaranteed to get the "pepper" as well. Just Use Bcrypt, pick as high a work factor as you can afford, and worry about something else instead.

(if you have a place to keep this extra secret where such an attacker can't get it, why not just put the passwords there?)

NateDad5• 12 years ago

Yes, exactly.

mnw21cam4• 12 years ago

As a counterargument, yes I can think of circumstances where an attacker gets read access to the password database, but not the application configuration (or code, depending) that contains the pepper.

Using a pepper is another example of security in depth. It doesn't protect against the situation where your attacker has root access on your server, but it increases the work that an attacker has to do (or the luck that the attacker has to have).

adwf3• 12 years ago

Correct me if I'm wrong, but I think a large proportion of database hacks are still SQL injection attacks.

If you assume that the password handling code is on a webserver and the database is on a separate server, there is no reason to assume that an attacker getting read access to the DB through an injection attack will be able to access a pepper on the webserver.

Defense in depth is rarely a bad thing. Especially when it doesn't actually harm the end-user experience.

danielweber3• 12 years ago

(if you have a place to keep this extra secret where such an attacker can't get it, why not just put the passwords there?)

The argument is that you can put the key into the code base or into the deployment environment. Passwords are user changeable and can't go there. And, just maybe, you only lose one or the other but not both.

(NB: I'm not defending the pepper, but at least one very major software security consulting firm uses this argument. I don't know how much time they spend doing pen tests and getting a feel for when the pepper would have actually saved anyone's ass, and when I pressed I only got a vague mumble about customer confidentiality.)

eldelshell• 12 years ago

What I'm using for my latest project is a not a hash, but using an internal 128 long key to encrypt using PBKDF2WithHmacSHA1 and persisting the result to a blob.

tptacek10• 12 years ago

If you have a secure place to put a key for reversibly encrypting passwords, just put the passwords in that place instead. I'm sure you'll be fine.

mnw21cam4• 12 years ago

Might be nice to mark your comment as a joke - someone may take it seriously.

mcescalante• 12 years ago

After reading this, the title is linkbait IMO (the author calls it a "sensationalist title")

Wohui• 12 years ago

It is, insultingly so.

baby2• 12 years ago

So several iterations of md5 + salt is not secure? I'm really wondering how much time it takes to find a password hashed by 5 iterations of md5 + a salt by exhaustive search.

triangleman83• 12 years ago

Hashing a hash is not generally considered secure because you have to assume that if your system was compromised, the hackers will know what methods you used, including the list of salts. If the hash runs quickly then you didn't really cause them any more time/work.

baby2• 12 years ago

That's right! KDF seems to be better than hashing in the case the server gets compromised.

danielweber3• 12 years ago

Assuming "several iterations" means "a million or more iterations" then you have captured most of what bcrypt gets you. You've broken their rainbow tables and they have to brute-force to find users using "passw0rd". You can tune the "several iterations" the same way you can tune bcrypt.

That said, don't roll your own. You probably screwed it up somewhere. Just use the bcrypt library call. Or scrypt to let you roll +2 against GPU attacks.

TheCoelacanth• 12 years ago

It's not advisable to use MD5 for anything. It's security is severely compromised. Weaknesses in MD5 have already been exploited to create a fake root CA[1] and a fake Microsoft code-signing certificate[2]. There isn't practical publicly-known preimage attack, but there was a theoretical attack published in 2009[3]. Additionally, MD5 can be calculated far too quickly to use for password hashing. A commodity GPU from 2009 can calculate 200 million MD5 hashes per second [4]. For password hashing you should use something like bcrypt or scrypt which is specifically designed to be computationally expensive.

[1] http://www.win.tue.nl/hashclash/rogue-ca/

[2] http://blogs.technet.com/b/srd/archive/2012/06/06/more-infor...

[3] http://link.springer.com/chapter/10.1007%2F978-3-642-01001-9...

[4] http://bvernoux.free.fr/md5/index.php

tptacek10• 12 years ago

With many, many iterations, simply iterating a decent hash function is a valid 80% solution to the problem. 5 iterations of a cryptographic hash function won't buy you anything at all, though.

keithwinstein2• 12 years ago

I agree with the basic idea here ("Stop associating 'hashed' with 'secure' when it comes to passwords), and I agree with the recommendation to use scrypt or bcrypt when necessary because "they’re your best option when rolling your own user credential storage."

But let's go for a part 3: please try not to store user credentials at all, if you can avoid it! It's not a great idea for every random Web site to create its own notion of identity for every user -- which requires a means of proving that identity, usually a password, and a way for that means to get compromised.

When it's feasible, better would be to outsource the job of verifying user credentials to some third party and only accept online cryptographic proofs of identity, whether that's via:

* an SSH client keypair (e.g. git push to GitHub)

* requiring requests be signed with a PGP keypair (e.g. uploading to Debian/Ubuntu)

* Mozilla Persona/BrowserID

* a TLS client certificate

* Facebook Connect

* Google Accounts

* OpenID

There's another benefit here -- no matter how many times you iterate a KDF, your website is still going to be vulnerable to an online compromise where the badguy just grabs passwords as users log in. (I understand systems like Meteor are nudging sites into using SRP-in-JavaScript, which is great, but in an online compromise that can just change.)

And while it may be good to use bcrypt vs. SHA-1, after all it's the users whose interests are ultimately at stake if the password DB is revealed, and yet the users generally have no way of knowing (cryptographically) that any given site really is using bcrypt. That suggests that the security dust is being applied in the wrong place -- the party that actually cares should be able to verify that the right thing is happening.

It would have been great if Mozilla had had more resources to stick with Persona (and of course if Facebook and Google could have afforded to endorse it).

Unfortunately even sites that are founded by ex-Facebook employees (like Quora) don't actually trust Facebook Connect enough to rely on it -- Quora uses FB Connect to let you "Sign Up with Facebook" but then requires you to establish your own Quora password just in case Facebook screws them over someday. (Then they store the password via unknown means...)

sillysaurus32• 12 years ago

If a website's only option for credentialing is Facebook Connect, I won't be using that website. Ditto for Google Accounts.

Maybe it's just me. But if many others on HN feel the same way, then that could be relevant for a new startup who wants to gain traction among a core group of initial users. E.g. I never would have used Dropbox or Airbnb if their only option to login was Facebook.

keithwinstein2• 12 years ago

Let's talk about why -- do you place greater trust in AirBnB (or random websites X, Y and Z) to keep your credentials safe than you do in Google?

Do you not want the identity provider (e.g. Google) to know every place you log in? (This was something Persona solved, alas...)

Do you not want to be reliant on a megacompany like Google whose spam filters might someday hit a false positive, causing Google to ban your account, and there's nobody to call and you're locked out of everywhere?

Or is it about separation, i.e. you don't want any single notion of your identity to have too much power if compromised (and you're careful to use unrelated credentials, e.g. distinct passwords, on every website)?

Or something else?

Are you ok with the way that GitHub and Ubuntu outsource the storing of credentials to authenticate a "push" by checking against a public key, for which only the user holds the private key? What if more services worked this way?

cruise022• 12 years ago

You can offer to let users log in with Facebook Connect and Google Accounts, etc. If I'm going to try out a new service, I'd rather use one of my existing log-ins than trust a brand new start-up to store my information securely. It's not their core competency.

jimmaswell• 12 years ago

If you use one account for everything there's a single point of failure, how is that better for the user's security? One account gone and everything's gone if you can't get it back.

lazerwalker• 12 years ago

Anecdotally, the problem isn't that sites (e.g. Quora) don't trust FB Connect, it's that users don't.

For every project I've worked on that had both FB Connect and in-house login, almost no users used FB Connect, no matter how the two options were visually presented. I've seen iOS apps get absolutely HAMMERED in their App Store reviews simply because they don't have any login other than FB and users refuse to use it.

I agree this is a problem, but if I'm a line-level engineer or product guy, my short-term goal isn't to come up with a high-level solution, it's to do what works and won't fuck with my acquisition funnel. Which is currently roll your own.

raverbashing3• 12 years ago

I think I know why is that

Users don't trust the app won't do anything else beside authentication, like: posting to your wall, spamming your friends, reading all your information, etc, etc

drdaeman3• 12 years ago

please try not to store user credentials at all

Sorry for nitpicking, but in my understanding of terminology, non-anonymous authentication always require some sort of credentials and a public key/OpenID/Google account/whatever is one, just as well as an old good username-password pair.

And in case of verifying certificates/signatures, we're not outsourcing jobs to any third party, but doing the verification ourselves. This is important distinction between user-owned keypairs and Persona/Facebook/Google/OpenID - I'm not sure entrusting user identities and authentication to a third party, instead of establishing secure means of authentication, is a wise decision.

cocoflunchy• 12 years ago

I see people saying "just use scrypt or bcrypt"!

I noticed in Django's admin this line (in the user section, with the default authentication app):

    algorithm: pbkdf2_sha256 iterations: 12000 salt: 8c8ttQ****** hash: pN+2tq**************************************

Am I fine? Is there something more secure? Why isn't Django's default scrypt or bcrypt? And what is pbkdf2_sha256 exactly?

schrodinger• 12 years ago

http://en.wikipedia.org/wiki/PBKDF2

I think it's fine, although some people believe [sb]crypt to be more secure. I think PBKDF2 is actually the default in ASP.NET now.

goblin892• 12 years ago

PBKDF2 with SHA256 hashing is default password storage mechanism in Django. Recommending PBKDF2 is one of the things that tptacek criticized “Practical Cryptography With Go” for[0].

Edit: Django has support for bcrypt but it's not the default because of third-party dependencies[1].

[0] https://news.ycombinator.com/item?id=7596280

[1] https://docs.djangoproject.com/en/1.7/topics/auth/passwords/

po3• 12 years ago

tptacek faulted 'Practical Cryptography With Go' for recommending PBKDF2 over bcrypt and scrypt however this is not what Django is doing.

Django chose PBKDF2 of the three primarily because it was easily implemented without pulling in third-party dependencies, which is very helpful for Django's deployment. Given the choice of the three, I think most all Django core developers would say that scrypt or bcrypt would be better but of the three options, PBKDF2 was deemed to be the most acceptable for driving adoption. Before PBKDF2 was added it was using a fast hashing algorithm and had no way to select one so all three were considered improvements and PBKDF2 was considered the 'safest' for adoption, not the optimal one. Note that it is now fairly easy to add bcrypt or scrypt into your django deploy if you choose to.

I'd also add that the parent article, talks about PBKDF2 but I am not sure it is using it correctly. It makes no mention of how many rounds it is configured for and I'm not sure what they mean by 'dumb iterations'. It should be able to make it as slow as scrypt or bcrypt or on a single machine benchmark. The advantage of something like scrypt is that it pulls in complexity on memory as well as CPU.

Edit For anyone interested in the decision making here, this was the summary ticket on Django password hashing and my summary at the time:

https://code.djangoproject.com/ticket/15367

https://groups.google.com/d/msg/django-developers/ko7V2wDVsd...

Tomte• 12 years ago

PBKDF2 is an older KDF that is probably a bit less secure (especially in the light of FPGAs), but well-known and standardized.

It's perfectly okay.

damon_c• 12 years ago

I found a decent, almost but not quite exact match for your question here:

http://stackoverflow.com/questions/4433216/password-encrypti...

raverbashing3• 12 years ago

pbkdf2_sha256 means they're feeding your password first through sha256 then applying pbkdf2

It's to prevent a limitation in the password size. (I can't find the page explaining this though) https://docs.djangoproject.com/en/dev/topics/auth/passwords/

You can switch to other hashers, of course, as explained on the page.

Now, is it only my impression or in the benchmark there's a bug and the number of iterations for pbkdf2 is fixed to 1?

cordite2• 12 years ago

The recent post "Fairy Tales in password hashing"[0] in regards to scrypt seems somewhat relevant here.

[0] http://vnhacker.blogspot.com/2014/04/fairy-tales-in-password...

skrebbel• 12 years ago

Hardly, because it's not about scrypt itself but about how (according to the author) the library's API encourages incorrect usage.

A reader of your comment might assume that the article is about some perceived flaw in scrypt itself.

cordite2• 12 years ago

I can see that perspective. I could have made it more clear.

I see it as it being more about incorrect usage or application, rather than a fault with scrypt.

tptacek10• 12 years ago

No, it's not relevant to this discussion at all.

po3• 12 years ago

This article shows PBKDF2 as being barely any slower than sha2 but makes no mention of how many rounds it is configured for. The note at the end implies that something is misconfigured. I'm not a rubyist, but something is definitely wrong there. PBKDF2 is maybe not the best choice but is certainly safe and better than sha3 if configured correctly.

If the author is reading here, it's probably best to take PBKDF2 out of the graph until you get it properly configured. Otherwise, people may choose sha3 as their password hashing function.

Please stop hashing passwords

Related Stories