Don't use bcrypt

http://www.unlimitednovelty.com/2012/03/dont-use-bcrypt.html

by bascule • 14 years ago

177 185 14 years ago

jgrahamc• 14 years ago

However, if you're looking for a key derivation function for a new project, bcrypt is probably not the best one you can pick.

I'm confused. Why would I pick bcrypt as a key derivation function when there are nice key derivation functions out there that are widely documented?

lhnn4• 14 years ago

As stated in the article, a popular stance on Hacker News and Stack Overflow is "USE BCRYPT". It's chanted to crypto-noobs and webdevs as a simple-to-use library for password storage that is more secure than MD5/SHA/Whatever hashing, and with built-in salts.

The whole point of this article is to say that, in fact, there are other options.

tptacek38• 14 years ago

Bullshit. There is one very-well-written article at Coda Hale's site that says "just use bcrypt", but in discussions of adaptive hashing on HN, people who know what they're talking about are continuously at pains to vouch for PBKDF2 and scrypt (it helps that one of the people who knows what they're talking about on this subject is (a) vocal on HN and (b) designed scrypt).

It irritates me that despite going through the effort of vouching for PBKDF2 and scrypt every time this f'ing topic comes up on HN, people still manage to reduce this issue to another tribal conflict.

And who are these "crypto noobs" you speak of?

there2• 14 years ago

The whole point of this article is to say that, in fact, there are other options.

Then maybe it shouldn't have such an inflammatory title as "Don't use bcrypt". "Alternatives to bcrypt", perhaps.

DanBC• 14 years ago

I like this article. But this sentence is odd.

> […] with an academic pedigree from RSA Labs, you know, the guys who invented much of the cryptographic ecosystem we use today.

Appeal to authority fails a little bit when RSA opens random Excel attachments from unknown untrusted sources - attached to an email that have to be retrieved from the junk mail folder.

(http://blogs.rsa.com/rivner/anatomy-of-an-attack/)

EDIT: As other commentators point out, I am wrong to suggest that anything coming from RSA Labs is somehow "weak" because someone at RSA fell for a phishing attack. I do find it odd that a security article suggests "These people are good; they did 'this thing' which everyone uses". That's not a great way to approach choosing crypto components. Even experts make mistakes.

SoftwareMaven7• 14 years ago

As does ad hominem...

Edit: realized I should probably explain. What an administrative assistant does in a social engineering attach has nothing to do with the quality of cryptographic research at RSA. Humans are ALWAYS the weak point in cryptography.

yock• 14 years ago

No, ad hominem is "You're wrong 'cause you're a jerk." DanBC pointed out that their opinion might not be as above reproach as is often assumed because of past lapses in good judgment.

tptacek38• 14 years ago

What can you share with us about the quality of cryptographic research at big software companies? And, besides that one: what are some other big companies whose cryptographic research you would rely on when selecting algorithms and crypto constructions?

Are you sure you know what you're talking about here? I'm not saying you don't, but most HN'ers who would write a comment like yours don't.

lhnn4• 14 years ago

tl;dr

PBKDF2 and scrypt each have supposed upsides to bcrypt and all the benefits.

PBKDF2: RSA tested and widely used. scrypt: memory hard as well.

tptacek38• 14 years ago

"Memory hard" is a serious benefit that scrypt actually has.

"RSA tested and widely used" is subjective, not particularly meaningful, and in some senses erroneous, and so makes a poor case for PBKDF2.

If people want to seriously push for scrypt as a replacement for bcrypt as the "default" function, I'll design and print flags and pennants for the movement. But when people say "use PBKDF2 instead of bcrypt", I think the net effect is to scare people back to salted hashes, and my general response is going to be to poke holes in their arguments.

dhimes4• 14 years ago

Question: what does "memory hard" mean? Uses lots of RAM to preclude simultaneous connections running attempts in parallel?

user2459• 14 years ago

Long story short, if a bcrypt exploit is found, you'll be sorry. So use http://en.wikipedia.org/wiki/PBKDF2 or http://www.tarsnap.com/scrypt.html instead. They're apparently better tested.

I'm no expert opinion, but seems a bit unnecessary and that bcrypt is still a perfectly good choice for most password stores.

cperciva8• 14 years ago

> While scrypt's cryptographic soundness, like bcrypt's, is poorly researched

I think we may have different notions of what "poorly researched" means.

zheng• 14 years ago

I think he meant it hasn't undergone the same level of public scrutiny. You've certainly spent ample time researching it, and it's obviously been tested, but possibly not as much something like AES.

cperciva8• 14 years ago

OK, but PBKDF2 hasn't had very much scrutiny either. The entire field of key derivation functions has been very much neglected.

tptacek38• 14 years ago

bcrypt is better tested than PBKDF2: far more publicly "testable" systems use it (bcrypt was the password file format for OpenBSD).

dchest3• 14 years ago

Note that scrypt uses PBKDF2 before and after the memory-hard function.

cperciva8• 14 years ago

Only as a cryptographic mixing/expansion function. There is no reason to think that scrypt's security would be any less if the PBKDF2 calls were replaced with xor.

dchest3• 14 years ago

Doesn't it provide more protection against possible flaws in Salsa?

Also, somewhat related question -- what if Salsa core in scrypt is replaced with BLAKE core (with fewer rounds than in hash), and SHA-2 in PBKDF2 with BLAKE, thus making it possibly smaller (hardware and lines of code). Will this work well?

SkyMarshal2• 14 years ago

TLDR: Use PBKDF2 or scrypt instead.

PBKDF2 has had longer public exposure, and also features an adjustable CPU work factor (though with a lower theoretical safety-to-compute-time than bcrypt).

scrypt is newer, but features both a CPU and memory work factor (memory-hard algorithm), and is algorithmically superior to both.

_delirium• 14 years ago

> PBKDF2 has had longer public exposure

I don't know whether or not it's had more public exposure, but I don't think it's had longer public exposure. Afaik, bcrypt's canonical reference document is the June 1999 Usenix paper (http://static.usenix.org/event/usenix99/provos/provos_html/i...), while PBKDF2's is the September 2000 RFC (http://www.ietf.org/rfc/rfc2898.txt).

SkyMarshal2• 14 years ago

Thanks for the reference, didn't realize that, though I think that's what the article was implying.

ceol• 14 years ago

So the author says to use PBKDF2 because it's researched and tested better than bcrypt, and then suggests scrypt as another alternative despite having, from what I can tell, less research done on it than bcrypt.

The crypto space can be intimidating to your average dev, but almost every app needs some sort of protection (at least for user information). I think the author is fair in wanting to push for the "default" to be PBKDF2 instead of bcrypt, but should he really be advocating a less-tested function in the same article?

tptacek38• 14 years ago

Is there a reason other than branding that you think it's fair to push for PBKDF2 to be the "default" instead of bcrypt?

pbsd4• 14 years ago

To be honest, I don't see why anyone would use bcrypt over PBKDF2, if the security of the primitives is of any serious concern. I am new here on HN, so I'm not aware of your arguments on why bcrypt is better than PBKDF2.

As for the "branding" question, would you recommend XTEA or MARS or Threefish over AES for someone in need of a block cipher? Of course not. Standards are not always perfect (and many a flaw has been found in standards), but they are generally beneficial.

PBKDF2 also has the advantage of being modular. It takes in an arbitrary PRF (although HMAC-SHA1 is the usual); maybe HMAC-SHA1 turns out to be poor for the job, just plugin a better PRF (hell, you can plugin a provable PRF that reduces to integer factorization or the elliptic curve discrete log). bcrypt is just bcrypt --- a seemingly not too peer-reviewed modification of an ancient cipher, that is not even recommended anymore.

scrypt is better than both, of course. Provable time-memory hardness is great, and should be made standard.

marshray20• 14 years ago

The reason I would prefer it is that PBKDF2 is standardized in an RFC with proper test vectors an a reference implementation in C.

Bcrypt and Scrypt do not. We could debate the relative quality of the cryptographers behind them all day, but realistically Bcrypt has had at least one widely-deployed implementation bug that caused a real decrease in strength which can be traced back to a lack of proper test vectors.

tptacek38• 14 years ago

PBKDF2 is worse than bcrypt.

scrypt is better than bcrypt.

bcrypt has the advantage of being both very good, and also broadly available on web platforms. scrypt does not yet have that advantage; when it does, I will start saying "just use scrypt".

But the simple fact is: all three of these functions are fine. ANY of them is a huge step forward from what people do without them.

"Just use bcrypt" is 1000x more effective as a meme than "just use adaptive hashing" (which is what all these constructions are).

So, while I have few specific technical qualms with this article (e.g. why do I care whether something is a PKCS standard or not?), the overall message is a bit hyperbolic.

cperciva8• 14 years ago

PBKDF2 is worse than bcrypt.

By roughly a factor of 5.

scrypt is better than bcrypt.

By roughly a factor of 4000.

why do I care whether something is a PKCS standard or not?

You don't care, and I don't care, but I'm sure you know lots of companies which do care (especially since PBKDF2 is a NIST standard too).

tptacek38• 14 years ago

For what it's worth: when dealing with banks and financial firms, where any technology without an adequately staid and reassuring web presence is frowned upon, we happily recommend PBKDF2. There's nothing "wrong" with PBKDF2. It's just not as good as bcrypt.

regularfry• 14 years ago

When we were looking at password hashing, and the choice came down to bcrypt or scrypt (about a year ago, so recently enough), I said we should go for bcrypt because scrypt was comparatively new; inasmuch as it makes a difference, it's just had less time to be attacked.

Was I wrong?

marshray20• 14 years ago

> By roughly a factor of (5|4000).

Do you have this written up anywhere? Link?

bdhe• 14 years ago

PBKDF2 is worse than bcrypt.

Can you elaborate a little bit on this? Or point to a reference. Wikipedia mentions that PBKDF2 is easier to implement with ASICs or GPUs. Is that the primary reason?

j_baker• 14 years ago

How does one stay up-to-date on this stuff? Or is it just something that's easier to remember for someone who specializes in security?

tptacek38• 14 years ago

bcrypt is something like a decade old. You don't have to stay up-to-date on it. The vulnerability that bcrypt accounts for dates back to 1972.

All you have to do is not design your own password hashes with SHA-2 (or Whirlpool or CubeHash or whatever some random Stack Overflow answer says you should use). You can safely keep PBKDF2, bcrypt, and scrypt in your bag of tools --- of those, only scrypt is recent --- and reach for whichever one is easiest.

haberman• 14 years ago

> PBKDF2 is worse than bcrypt.

You seem to be speaking only to its compute time -- what do you say to the article's claim that bcrypt has a higher probability of having an unexpected attack that mitigates its computational complexity?

Also, since all of these algorithms have adjustable work factors, what does it even mean to say that one is stronger than another? Couldn't you just calibrate the work factors so that they are equivalently strong? Though naturally scrypt has strength in another dimension also (memory).

tptacek38• 14 years ago

It's nonsensical. It essentially argues that SHA2 is less likely to have cryptographic results relevant to hashing in the next 5 years than Blowfish. It's also an argument the post doesn't support with any actual evidence.

But that's not my issue with the article. My issue with the article is that it takes a simple security issue with no "real" wrong answers and turns it into a tribal conflict, which has the net effect of reducing the number of people who will use adaptive hashing at all.

I'm familiar with Tony Arcieri's work and generally think highly of him; this article, though, is inexplicable and smacks of hipsterism. "I liked Nirvana but then they got popular and sold out, so now I listen to Sleater Kinney". Well, that's going to sound dumb in 10 years.

Let me say yet again that if you use PBKDF2, bcrypt, or scrypt, you are going to look smarter than the average webdev, no matter which one you pick. Do whichever is easiest.

nextparadigms• 14 years ago

Isn't that a self-fulfilling prophecy? If you buying into the "just use bcrypt" idea, how will scrypt ever change the fact that it's not widely available on the web?

tptacek38• 14 years ago

I addressed this directly downthread. I don't care whether TweetYourCatFood.com uses bcrypt. Specifically, I care that:

* It's got a reliable Gem for Ruby

* It's got a reliable easy_install package for Python

* It's got a good reliable CPAN entry for Perl

* It's got a Java jar file from a reputable source

* It's got a .NET assembly from a reputable source

And what I'm saying is not that scrypt will be "safe to use" when that happens. scrypt is safe to use now; safer, marginally, than bcrypt. What I'm saying is that when that set of things happens, I will personally stop recommending bcrypt and start recommending scrypt. And I only point that out because I always feel a little bad about not recommending scrypt, which is strictly speaking better than bcrypt.

bascule5• 14 years ago

I can't speak for other platforms, but at least as far as Ruby goes:

    $ gem install scrypt
    Building native extensions.  This could take a while...
    Successfully installed scrypt-1.0.3
    1 gem installed

tptacek38• 14 years ago

You can't go wrong with Ruby scrypt.

But then, unless you give up and don't use any of the three, you can't go wrong with any of these constructions.

Xylakant5• 14 years ago

Still I'd stick with bcrypt for ruby. It's in ActiveModel, so it's in Rails. It's in Authlogic. It's the default password storage in Datamapper. I can just point my fellow developer towards the documentation and say "use bcrypt" and be reasonably sure that a basically competent developer will get it right. That's a good thing in my book. Neither scrypt or PBKDF2 have that level of integration so far. When that changes, I'll reevaluate my decision.

Still, it's certainly good to keep in mind that there are alternatives, usage profiles and requirements differ and so do the solutions.

tedunangst2• 14 years ago

"that won't help you if an attack is discovered which mitigates bcrypt's computational complexity."

AND you lose control of your database. Even if I had a magic instant bcrypt reverser, it does me no good if I don't have the hashes. You cannot be compromised by a bcrypt mistake, it would only make your already existing compromise slightly worse.

tptacek38• 14 years ago

And of course that's a nonsensical argument in favor of bcrypt, as it applies identically to PBKDF2 and scrypt.

You could use a similar argument to recommend Elliptic Curve over RSA, or RSA over Elliptic Curve.

tedunangst2• 14 years ago

It's not an argument in favor of bcrypt. It's a rebuttal to an argument against bcrypt.

I have seen several people, whenever this comes up, make the mistake of thinking that your whole app could somehow be compromised by the wrong choice of password hash.

grandalf• 14 years ago

How is being from RSA labs an advantage? I'd estimate that it's more likely that there are government backdoors in RSA crypto than in other variants.

tptacek38• 14 years ago

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA IT BURNS IT BURNS

marshray20• 14 years ago

For one, it's something the government uses for its own crypto.

Unlike some crypto standards, PBKDF2 is just something too simple and too user-configurable to be able to hide a meaningful hole in. Given that PBKDF1 was found to be less-than-ideal and depracated in favor of the better PBKDF2, it would be a very risky proposition to attempt to weaken it in a way that gave a meaningful advantage to one party over the other.

JoachimSchipper5• 14 years ago

PBKDF2 is a fairly "obvious" construction (it's mostly just repeated HMAC'ing); it would be really hard to hide anything clever in it.

noonespecial• 14 years ago

I wouldn't exactly call "just use bcrypt" a cargo cult. It has had real benefits to the web dev community because it factors down to "don't store it in plain text or use MD5 or something".

Several people working on small projects have already come to me and asked me "what's this bcrypt thing? should I use it here?" I guarantee these folks would have just stored it plaintext otherwise. So I've directly observed the mantra making stuff safer. Win.

tptacek38• 14 years ago

I strongly agree that when asked by friends what to use, you should think about which library is going to be easier for them to use, and not try to think through which construction is "better". That this blog post gets the real ordering of betterness wrong is tangential to the real problem, which is that it gets the value proposition of bcrypt wrong.

smacktoward4• 14 years ago

> I write this post because I've noticed a sort of "JUST USE BCRYPT" cargo cult... This is absolutely the wrong attitude to have about cryptography.

No. This is incorrect. This is exactly the right attitude for most developers to have about cryptography, because on a subject as complex as cryptography most developers (including me!) are nowhere near smart enough to understand the ins and outs.

Encouraging people to make their own decisions on subjects they aren't equipped to understand fully is dangerous advice. It leads to all sorts of bad outcomes. People who have to choose between options they don't really understand end up choosing things at random, or on the basis of incomplete or misleading information, or getting seized up by the need to make a choice and choosing nothing at all.

This makes cryptography one of the very few cases where a cargo-cult approach is better than the alternatives. A simple message that "this is the approach people smarter than you agree is correct, use it," repeated consistently, will help more people more completely than dumping them into the deep end of the crypto pool ever will.

pleasebehonest• 14 years ago

That attitude is abrasively condescending.

Why not just say, "people who spent a lot of their time studying cryptography strongly recommend this approach. They feel the approach that you are considering is simply insecure."?

Framing things in terms of intelligence isn't going to win anyone over, if that's your goal. And it probably isn't accurate, either.

tptacek38• 14 years ago

Sounds pretty close to accurate to me. Do you want to feel good, or do you want to choose good crypto? Pick one or the other.

dhimes4• 14 years ago

He's probably using "smart" in the sense of "accumulated knowledge in a particular space" instead of in the sense of raw mental horsepower. http://news.ycombinator.com/item?id=3583985

smacktoward4• 14 years ago

By all means, use different wording when passing this message along if you're worried about hurting the recipient's fee-fees. But you won't be doing them any favors if you change the message to a Stuart Smalley-style "You can do it! You're good enough! You're smart enough! And gosh darn it, people like you!" Because on this specific subject, the odds are very, very, very unlikely that they actually are.

jiggy20113• 14 years ago

it's not necessarily about intelligence per se.

I think "smarter" in this context means more well read about the particular subject of crypto, although people good at that are likely to very intelligence all round too.

A different approach is not necessarily "less secure" it's just that it may have had less people banging on it trying to figure out ways to break it.

tpsreport• 14 years ago

The point here is that this particular cargo cult around bcrypt (one subscribed to by some really loud people) has a shaky foundation and does not deserve its reputation. He's offering alternatives that have been better studied.

So, by all means, subscribe to a cargo cult for crypto. But pick the cult carefully.

tptacek38• 14 years ago

He's wrong. (Read downthread for why I said this; my bluntness here is a mercy).

smacktoward4• 14 years ago

But just the existence of multiple cargo cults causes damage, because it leads people to assume that "the experts are divided." Which will lead some people to go with the wrong batch of experts, and others to just throw up their hands in confusion and store their passwords in plain text because it's too hard for them to figure out which group of experts is right.

This is a case where unanimity in the message is important. If all the experts say "use A," people will take that to mean there's no debate about the merits of A over B and C, and use A. If some say "use A" while others say "use B" or "use C", some fraction of listeners will give up and use nothing at all.

Firehed• 14 years ago

He's correct in that if you've selected bcrypt for key derivation, there's a good chance you could be doing things better (for one, its output is only 184 bits long; insufficient for AES256) where PBKDF2 works in a way where you can customize the output length.

However, the point of the bcrypt argument is not that bcrypt is the best algorithm for certain things, but that it's (at a minimum) about four orders of magnitude better than most people's "secure" password storage algorithm: sha1(password). Because it requires both a salt and a work factor, even a dictionary attack is wildly impractical unless there's a massive flaw discovered in the algorithm.

If developers are going to be trained to pick a specific algorithm for password storage, I'd much prefer bcrypt (no known flaws, many benefits) over sha1 or md5 (designed to be fast for checksumming, salt not required). Might PBKDF2 be a better choice still? Very possibly; I haven't done enough research to intelligently answer - and since this is crypto, I will not best-guess it.

My real point here? The article attacks bcrypt as a key derivation algorithm, but I've never seen someone suggest it to be used in such an application. Even the post that started what you may call the bcrypt movement (http://codahale.com/how-to-safely-store-a-password/) is linked in the article, and it's titled "How to safely store a password". It is NOT titled "How to safely derive encryption keys".

So yeah, I'm calling linkbait.

mturmon2• 14 years ago

One more example: pseudorandom number generation (non-cryptographic). There are lots of algorithms out there, and lots more you could design, but the Mersenne twister has pretty much become the first-choice algorithm.

Especially with linear congruential generators, it's easy for people who don't know what they're doing to add "extra randomization" that makes the resulting numbers worse than the originals.

It's better for most people to not get fancy and use the standard algorithm.

marshray20• 14 years ago

MT uses a lot of memory. IIRC, it's multiple KiB where 64 bits ought to do fine.

A modern CSPRNG like the SHA-3 candidate Skein can produce random data at a steady rate of only a few cycles per byte. I wonder if at some point something like that will dethrone MT for non-crypto PRNGs.

InclinedPlane2• 14 years ago

Option 1: Rely on what everyone else does.

What happens if that method is flawed? You hear about it as soon as it's discovered and it gets fixed quick.

Option 2: Roll your own stack based on a personal understanding of cryptography.

What happens if that method is flawed? Perhaps only you and an attacker could possibly know such a thing. You have to be ever vigilant and you have to acquire an incredible amount of crypto knowledge. If you ever leave the company they are pretty much fucked from then on until whatever you wrote is replaced.

Don't roll your own system.

InclinedPlane2• 14 years ago

To add on to this, it is of course necessary to have enough familiarity and competence with the basic principles of cryptography to know that you're "doing what everyone else is doing" correctly, instead of just some silly pantomime, such as: http://thedailywtf.com/Articles/Topgrade,-SHA1-Encryption.as...

But it's not necessary to understand the depths of cryptography beyond that there are various black boxes that have certain properties.

pnathan• 14 years ago

> No. This is incorrect. This is exactly the right attitude for most developers to have about cryptography, because on a subject as complex as cryptography most developers (including me!) are nowhere near smart enough to understand the ins and outs.

I hear that a lot, and it always reminds me of Jante Law[1]. Its a disservice to keep telling people that they are too stupid to understand something. Too ignorant, perhaps - that can be remedied - but everyone is not too stupid to understand crypto. It's simply another field, mostly mathematical, and goof-ups are easy to make and often very costly.

edit: I should also make another point. Cryptography is exactingly and excruciatingly hard to do at industrial strength. I am not recommending people go out and roll their own crypto for production systems. It's possible for the initiated to do right; it's possible to get initiated. The uninitiated almost certainly will goof. I'd like to further point out [2], which is a discussion and break on a homebrew crypto, for a taste of the difficulties and mathematical sophistication needed.

[1] http://en.wikipedia.org/wiki/Jante_Law

[2]http://phrack.org/issues.html?issue=64&id=10#article

specialist3• 14 years ago

Cryptography is exactingly and excruciatingly hard to do at industrial strength.

Related: Which is why I'm against any and all forms of electronic voting. I've done a fair share of crypto (as a user of crypto libraries) and I barely understand how it works. There's ZERO hope for the layperson to understand crypto-based voting systems.

One of the central tenets of American style voting is a public vote count. Using crypto puts the "public vote count" into the hands of a the high priesthood (a few adepts), which is a really bad idea.

pygorex2• 14 years ago

> Its a disservice to keep telling people that they are too stupid to understand something

That's not the point. Specialization - developing a deep understanding requires time and effort. As a developer choosing a way to store passwords securely is one task out of 1,000 I'm responsible for. So I acquire a general understanding of cryptography to make a decision - but that decision relies heavily on experts and consensus. The worst thing I can do as a developer is to go down the rabbit hole of cryptography and spend 10 hours researching an optimal password hashing scheme for my app. Sure, I've begun the journey of having a deep understanding of crypto - but I still have 999 things to do!

Which reminds me I need to get off HN and get some work done .....

bascule5• 14 years ago

You're defending cargo cult cryptography? If you try to use crypto without understanding it, you're at great risk of using it wrong.

tptacek38• 14 years ago

The opposite turns out to be true in practice. People have the choice of using a simple crypto library interface (like BouncyCastle PGP) or "really getting to understand" AES, and so end up fielding software with vulnerabilities PGP addressed in the '90s.

A pithier way to say the same thing is, "you're right, except for the words 'without understanding it'".

timwiseman• 14 years ago

This is true, but most programmers don't and won't ever get a deep understanding of crypto. For that large majority, it makes complete sense to ask "What do the people who do understand crypto use in a case like this?" and use that.

That kind of copying is clearly inferior to either gaining the knowledge to make an informed decision or hiring an expert to do it for you, but for most projects that need crypto neither one of those is practical. So, copying the experts is the best practical approach in most cases.

derefr• 14 years ago

I can use a microwave without any understanding of its functioning, or I can try to understand electromagnetism well enough to build my own. I know which method I'd trust not to render me sterile.

jiggy20113• 14 years ago

It depends what level you are using it on. If you are writing crypto libraries on your own then sure you are at significant risk of screwing it up, but a good crypto library should have good defaults that will provide proper security.

Expecting every programmer who needs to build an application that stores passwords or personal information etc to have a rigorous mathematical understanding of crypto (this probably means having a PhD or similar in the subject) is just plain unrealistic.

It's sort of akin to suggesting you shouldn't allow a mechanic to change the brakes on your car without them having a detailed understanding physics.

gfaremil• 14 years ago

It is very dangerous to be ignorant about critical computer science fields like cryptography and just go with the cult.

I'm not saying you should build your own cryptography, but a good hacker (or a good engineer how we call them in early 90s) should understand difference between bcrypt, PBKDF2, and scrypt. At least to understand why bcrypt is better than salt+SHA-1. And some other aspects of security.

However, if somebody has no clue about cryptography and security then she/he should go with bcrypt - but I'm not sure if that person should be responsible or in business of storing somebody's critical data at all.

moxie3• 14 years ago

This article doesn't feel very well done to me. He writes: "The first cipher I'd suggest you consider besides bcrypt is PBKDF2."

PBKDF2 is not a cipher. It's a KDF, and it's almost always used with an HMAC or a cryptographic hash rather than a cipher. The thesis of this article seems to be "PBKDF2 is well understood, where bcrypt is not." In fact, the opposite is probably true.

bcrypt uses a block cipher (blowfish) to create its underlying compression function. Block ciphers are extremely well understood, have been studied to death for years, and are modeled on extremely well understood constructs. They can be used to create cryptographic hash functions, but usually aren't, because they're slow (which we don't care about in this case).

Cryptographic hash functions, by contrast, are not well understood at all. They are "magic" in many ways, and aren't modeled after anything. Many more "bad things" happen in this space than in the block cipher space. The only reason people mess with them at all is because they're faster than block ciphers, which again, we don't care about in this case.

The other appeal to PBKDF2 is because it "comes from RSA." This doesn't feel like an extremely compelling argument, but if we were going to believe it, then why not use the PKCS#12 KDF? PBKDF2 was proposed in PKCS#5, and "12" is a larger number than "5", so if we're going to do what RSA tells us we should do, they're essentially saying we shouldn't actually use PBKDF2.

tptacek38• 14 years ago

Ladies & Gentlemen, Moxie Marlinspike, whose comment should be heading up this whole thread.

marshray20• 14 years ago

Now I'm as much a fan of Moxie's as anybody, but I think one part of cryptography that needs to change is this tendency to excessively appeal to authority. Especially when speaking about practical issues.

bascule5• 14 years ago

These are all valid criticisms. Apologies for using "cipher" where I meant "algorithm"

cdavidcash• 14 years ago

Is this a common opinion amongst practitioners? The opposite philosophy (e.g., that a random oracle is a "weaker object" than an ideal cipher) underlies some lines of work in the theoretical cryptography literature.

tptacek38• 14 years ago

Yes, it's a common opinion (I quoted Schneier on it downthread).

pbsd4• 14 years ago

The two models are equivalent, in fact: http://eprint.iacr.org/2008/246

This of course says nothing about how to go about building an actual cipher/hash that withstands all kinds of cryptanalysis.

Edit: apparently not so clear, I'm told: http://arxiv.org/abs/1011.1264

justin_vanw• 14 years ago

I guess it's hip to have an opinion, but JUST USE BCRYPT. It's secure and available. Don't spend time thinking about it, just use bcrypt, it does everything you want, move on to something more worth your time.

Unless you're some blogger who needs to generate some page views, then pick some obscure topic like how to store password hashes and rake muck.

baq• 14 years ago

> JUST USE BCRYPT

multiple things come to mind:

- Millions Of Flies Can't Be Wrong

- Citation Needed

- JUST USE SCRYPT

etc.

zobzu2• 14 years ago

Yeah, don't think about stuff. Never think about stuff, in fact! Let us think for you! Or if possible, random self-proclaimed experts!

That's the way to go.

(and i bet many won't even notice the sarcasm)

Freaking hell, do critise, do research, and that apply to everything, not just crypto. It's not rocket science.

SoftwareMaven7• 14 years ago

As long as we've got some smart people hanging around: where does HKDF[1] fit into this? More specifically, when would you use HKDF over scrypt?

1. http://eprint.iacr.org/2010/264.pdf

tptacek38• 14 years ago

This makes PBKDFs very different than the general-purpose KDFs studied here. In particular, while passwords can be modeled as a source of keying material, this source has too little entropy to meaningfully apply our extractor approach except when modeling the hash function as a random oracle. Also the slowing-down approach of PBKDFs is undesirable for non-password settings

SoftwareMaven7• 14 years ago

But, as mentioned by @cperciva elsewhere in this thread, generating a key and creating a password hash are nearly synonymous. Using HKDF for passwords would be silly, but the more interesting question is: when would you use scrypt for key derivation in a system?

More to the point: what are the tradeoffs you'd consider in choosing one over the other?

(Addressed more to @cperciva...) I'm assuming tarsnap uses scrypt as its actual key derivation function for file encryption and authentication. Why scrypt instead of something else (and I have faith that it's not "not invented here" syndrome)?

jiggy20113• 14 years ago

To be honest, if anybody breaks into your web application it is very unlikely to be because they broke any encryption. SQL injection or man in the middle type attacks are far far more likely.

What you need out of an encryption package is in the event of being tested for PCI compliance or any legal liability investigation into a breach. You need to be able to say "all of our encryption is done with bcrypt , it's the industry standard an complies with X Y and Z".

m0nastic2• 14 years ago

The debate around secure password storage is sort of orthogonal to initial compromise of a web application.

The reason that it's important (although as others here have noted, less important than primary application security concerns) is what areas of attack are opened up by using insecure password storage "after" an initial compromise.

This could be something as simple as being a nuisance to users of the system (having to send out those "our password database was compromised, and we didn't do a good job of storing them securely, so you should probably change all of your associated passwords), to something much more serious (using said insecurely stored passwords to attack your other systems for example).

PCI doesn't really care how you're encrypting your data at rest. I cracked the password storage from an application once which was literally just a simple substitution cipher (which was positionally dependent...it was for all intents and purposes as secure as a newspaper cryptogram puzzle). That application was PCI compliant.

While "what you need out of an encryption package" might just be the bare minimum of "cover your ass", that's no reason to settle for insecure password storage.

marshray20• 14 years ago

It's also something of a public health issue given how most users re-use some passwords across multiple sites.

"Site A" may be less than careful about security since they perceive their data as being of low value (e.g. "register for a chance to win free movie tickets"). But when they get hacked and their users' passwords cracked, it will likely expose plenty of Facebook and online banking credentials.

16s• 14 years ago

Microsoft uses PBKDF2 for newer domain cached credentials (DCC2). These password hashes are stored in the registry of Windows clients (laptops and desktops) and allow users to logon when the domain is unavailable. They use 10240 iterations. It's very compute intensive to crack... roughly 330 guesses per second. Great article BTW!

Edit: I only post this to add to the examples of who uses PBKDF2 in addition to what the article lists.

steveh73• 14 years ago

I'm a web developer, so while security is obviously a huge concern, it's not my main area of expertise. I don't have the knowledge to evaluate the pros and cons of each cipher, and the situations in which it's appropriate to use each one.

It would be wonderful if someone with more knowledge of the subject could throw up a 1-page site with an appropriate security choice (or a few choices with situations in which each would be more acceptable) for a given range of situations, to establish a 'sane default', taking into account their availability on a number of platforms and programming languages.

For example:

  Need to sign a message? Use HMAC-SHA1
  Need to checksum a file? Use SHA-1
  Need to hash a password? Use bcrypt
  Need to transmit data over a network? Use SSH2
  Need to secure HTTP? Use SSL 1.2 with (these ciphers in order of preference)
  Need to secure home WiFi? Use WPA2-PSK
  Need to encrypt files? Use GnuPG
  Need to do (this type) of encryption? Use CBC. For (this type), use ECB
  Need to create a TrueCrypt volume? Use (this cipher) with (this many) bits.

chrismsnz2• 14 years ago

I guess the question is - who do you trust to make these calls?

In this case, the answer probably is OWASP which is a great and often overlooked resource, contributed to by a lot of experts in the area. They have a lot of pages in their wiki that address crypto concerns...

https://www.owasp.org/index.php/Guide_to_Cryptography

https://www.owasp.org/index.php/Cryptographic_Storage_Cheat_...

https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet

tptacek38• 14 years ago

OWASP is a great resource with a terrible track record for application cryptography.

tptacek38• 14 years ago

    Need to sign a message? Using S/MIME or PGP.
    Need to checksum a file? Use SHA256.
    Need to hash a password? Use bcrypt, scrypt, or PBKDF2.
    Need to transmit data over a network? Use HTTPS/TLS
    Need to secure HTTP? Use HTTPS/TLS, preferring AES in CTR and then CBC.
    Need to secure home WiFi? Use WPA2PSK.
    Need to encrypt files? Use any implementation of PGP.
    Need to do (this type) of encryption? Use PGP. Never use AES directly. Never use ECB for anything.
    Need to create a TrueCrypt volume? Can't help you; we use PGP.

orthecreedence• 14 years ago

Oddly enough, this article prompted me to setup bcrypt hashing for our unreleased app. I tend to not follow the tide when it comes to people screaming on the internet about something. People get highly emotional about the "next big thing" and defend with every ounce of willpower the decisions they've made. While I understand this mentality, it can make it hard to be objective.

Seeing an article where someone disagrees with the buzzing hive mind is always refreshing for me and made me actually consider, for the first time, that an algorithm aside from sha1(app_salt + user_salt + password) would be a good idea.

I did some research and decided that for PHP, bcrypt is the absolute easiest option to implement. scrypt is too new, PBKDF2 while administratively accepted has much less info on using it in PHP than bcrypt does.

So while I ultimately ended up disagreeing with the author, the article was invaluable in the end.

m8urn• 14 years ago

Yes, Use Bcrypt. And Scrypt: http://news.ycombinator.com/item?id=3725284

tzs• 14 years ago

Suppose hypothetically you knew everyone using your service used a strong password. Say they all have good password managers that generate 40 character random passwords for them.

Is there then any need to do more than a simple salted hash? (Remember, the hypothesis is that all your users are using strong passwords).

tptacek38• 14 years ago

The difference in that (obviously unrealistic) case is (I think?) between a cost per password of hundreds of thousands of dollars versus high tens of millions of dollars.

emboss• 14 years ago

While we agree on the fact that using either of the three can't be a bad thing, I'd like to give my opinion on why I favor PBKDF2 over bcrypt, and probably even over scrypt, although I admit the "memory-hardness" of the latter makes it superior in principle. But still, my reasons for going with PBKDF2:

It's not just that it's endorsed by RSA, no it's actually the NIST recommendation for password hashing and I find it rather unfair that people on this thread turn that against it!

It's the same argument why we generally recommend AES over let's say Twofish or Serpent. We all agreed here that in crypto it's a good thing to be mainstream. And being recommended by NIST makes you Justin Bieber, or not? Standard algorithms may be poor, true, but being a standard has one important advantage: most of the public scrutiny goes into the standard. Much more money and fame there. So it's much more likely that the public will get to know about a flaw in the standard faster than it will get to know about flaws in non-standard algorithms. And that's why I follow standards - even if it's a crappy algorithm, I will know immediately when it's broken and I can react by replacing it right away. The time between an algorithm gets broken and the fact becoming public knowledge is potentially higher the less common an algorithm is. And the time in between being broken and being public knowledge is the most dangerous in my opinion.

I'd like to point out that bcrypt is not equal to Blowfish. It piggybacks on Blowfish's key setup. But note that it just piggybacks, on top of that it further extends the original key setup. Blowfish's key setup was probably never invented to do what bcrypt does now, and the last 30min of googling have not brought up any papers about bcrypt cryptanalysis. Compare that to HMAC. Compare that to using PBKDF2 with HMAC SHA-3 when it's out. I'm not saying that Blowfish or any of its parts are bad, but if not PBKDF2 itself, but then most certainly its building blocks have received a lot more analysis than bcrypt or scrypt. With SHA-3 on the horizon the research community knows a lot more about hash algorithms and there is a lot of research going into these topics. That's why I personally feel safer with a construction that maybe in itself has not received more research than the other two alternatives, but where its building blocks almost certainly have, unless somebody proves me wrong. And when that happens, I'll stand happily corrected and will use the next standard.

dlitz• 14 years ago

I wasn't aware that anyone suggested using bcrypt for key derivation. The idea, as I understood it, was to avoid writing your own password hashing implementation. Bcrypt is a complete password-hashing implementation, so use it, rather than cobbling something together yourself. This is the standard advice for cryptographic software.

PBKDF2 isn't a password hash: the specification doesn't define a storage format for iterated, salted password hashes. It's not that hard to invent one if you already specialize in writing cryptographic software, but most programmers still shouldn't be doing that. It's just too easy to make mistakes that go unnoticed until it's too late.

If you insist on using PBKDF2, then I suggest using my PBKDF2.crypt() implementation at https://github.com/dlitz/python-pbkdf2. I'm not a cryptologist, but I'm the maintainer of PyCrypto, so presumably I can be trusted to do a better-than-average job of this sort of thing. If people want, I'll write a proper spec and add SHA256 support with a different algorithm identifier (the current implementation still uses SHA1).

But really, if you need a password hash, just use bcrypt and get back to writing the code that actually provides value to your user base and differentiates you from your competitors. Bcrypt is good enough for now. This advice might change in the future, so do pay attention, but for now, just use bcrypt.

antihero• 14 years ago

I know this may be dumb, but why don't we chain all the algorithms, then if one is broken the passwords are still secure.

Don't use bcrypt

Related Stories