Introducing Sodium, a new cryptographic library

http://labs.umbrella.com/2013/03/06/announcing-sodium-a-new-cryptographic-library/#

by kzrdude • 13 years ago

207 81 13 years ago

labs.umbrella.com

rosser• 13 years ago

NaCl isn't new; it's been around since at least 2009.

brianm• 13 years ago

Sodium is effectively a fork of NaCl, and is new.

tptacek18• 13 years ago

Nobody uses it, because it lacks bindings to common languages, which is the problem this is solving.

count• 13 years ago

Is that why, or is it because nobody can find it? At first I thought you were talking about google's Native Client, and it's the Google results for 'nacl' or 'python nacl', etc. have both the crypto lib. and the browser vm interspersed.

alexjeffrey5• 13 years ago

If it works as suggested, this library binding will be a godsend to frontline developers. Cryptography is exceedingly hard to get right and unfortunately existing libraries often tend to be written for developers who are already familiar with cryptography. Hopefully this will see the adoption it deserves and we'll see less incidents like the Mega encryption scandal.

tptacek18• 13 years ago

NaCL/Sodium would not have been deployable in Mega's design. I agree with everything else you say, though!

alexjeffrey5• 13 years ago

that's true, but it's the highest-profile encryption drama I could think of - a more appropriate example would probably be the onslaught of companies which are storing passwords in plaintext or simple MD5 hashes.

missinginaction• 13 years ago

Unfortunately, like NaCl, it's missing 2 common and critical functions:

    1. Password-based key derivation
    2. Persistent key storage

So developers are still expected to work out those 2 problems and will reliably screw them up.

bascule9• 13 years ago

Sodium will ship whatever algorithm wins the upcoming Password Hashing Competition:

https://password-hashing.net/

mpyne• 13 years ago

I thought the first one was solved at least 3 different ways over now, and that we should just pick one of [bcrypt, PBKDF2, scrypt]?

tel• 13 years ago

Okay, that's where I'm at these days. What is the right way to do persistent key storage?

qznc2• 13 years ago

Persistence means storing it in a file or database. Not a job for library of primitives.

kzrdude• 13 years ago

I don't think NaCl keys are suitable for persistance. It seems to require absolute uniqueness of nonces.

tptacek18• 13 years ago

This is based on NaCL. NaCL is an extremely great library that we virtually never recommend because it is very difficult to integrate into most people's dev environments.

Libraries like NaCL, Keyczar, and Cryptlib work by removing all the design choices from cryptography. You don't pick the key sizes, you don't pick the algorithms, you don't even pick what kind of keys you exchange. They implement a whole cryptosystem, as if for some new unreleased version of PGP, subtract the file format, and present it as an API. They're great. They are literally the only way you should be deploying cryptography in your applications.

I think this is a great development, but I am not qualified to say. This sits in a weird place in between a language binding for NaCL and a fork of NaCL. As a bindings package it's an overachiever; as a fork, it seems to have made extraordinarily conservative changes.

You'd really like a real crypto dev --- Matthew Green, Colin Percival, DJB (NaCL's primary author; @hashbreaker on Twitter), Steve Weis, Trevor Perrin, &c --- to say "this is all totally sane, and if you buy the premise of NaCL, go ahead and use this instead.

You'd also really like Sodium to say "this is as much as we're changing from NaCL and nothing else", because what makes NaCL worth building on is the expertise that went into designing it in the first place, which, like me, Frank Dennis probably doesn't have.

alexjeffrey5• 13 years ago

In what way is NaCL difficult to integrate into existing dev environments? I've not personally used it before, but I've been looking for a library that abstracts away some of the difficulty of cryptography (granted you can't hide it all) and NaCL appears to label itself as such.

tptacek18• 13 years ago

C library with no bindings that won't trivially† build on Windows: dead on arrival.

† Definitions of "trivial" including "not requiring understanding any of the details of a Unix toolchain at any stage of adoption"

bascule9• 13 years ago

While the people you've mentioned haven't been keeping an eye on it, Jean-Philippe Aumasson has. There have been a lot of other "mid-level" crypto people who have been keeping an eye on it as well and giving feedback to Frank. For example, at one point Frank switched out the default random number generator to use XSalsa20 instead of /dev/urandom. When this was pointed out, Frank addressed it by switching back to /dev/urandom (and a separate codepath for Windows)

Note: I've talked to Matt Green about Sodium. About all he had to say was as soon as you modify NaCl, you've violated the djb warranty ;)

moxie• 13 years ago

This looks like new development that's just API-compatible with NaCL though, right? Which feels a little strange.

The reason I'm drawn to the work of DJB is because of his fanatical attention to security-related detail. He wrote and released his own high performance mail server, DNS server, web server, and logging replacement during a period of time when server-side software was plagued with exploitation, and people have only ever found, what, two somewhat minor bugs? Ever?

So what he creates is almost always amazingly solid, but for all his attention to security perfection, his software is also amazingly unusable. Want to run publicfile? First, forget everything you know about standard unix directory structures, because DBJ doesn't like them.

I feel somewhat the same way about NaCL. We can bet the lives of our first born children that the implementation won't suffer any of the bugs that OpenSSL has over the years. Side channel attacks, memory corruption bugs, and timing problems will almost certainly be nonexistent, and someone could probably do their PhD thesis on documenting why.

But like his other software, NaCL has weird usability quirks. This project points out that some of those quirks were certainly manifest in the implementation, but some of them shine through the API itself as well. Like his other software, my sense has been that interacting with NaCL is something that we would almost have to put up with in order to avail ourselves of DJB's greatness, rather than simple joy.

So a rewrite that's API compatible isn't super immediately appealing to me, in the same way that I don't think I'd be particularly drawn to an interface-compatible version of qmail or publicfile that was written by someone else, when the value was in DJB having written them to begin with rather than the interface (the horror!) they provided.

Although to be fair, it's certainly not as if anything else is that much more usable in this world right now.

tptacek18• 13 years ago

I thought he had implemented it in terms of NaCL (I hadn't bothered to look at the code, because I worried I'd decide I was smart enough to endorse it).

The actual crypto in this is DIY?

jedisct17• 13 years ago

As discussed on the mailing-list, there will be a high(er) level API, similar to the C++ API, that doesn't require dealing with input/output being placed at different offsets in a shared buffer.

This is not a rewrite. It uses the reference implementations from NaCl and the same constructions. What it brings over NaCl is a standard build system.

bascule9• 13 years ago

Just a note on the approach taken here: I can't say djb would approve of Sodium, but when I asked him about NaCl's portability, he had this to say:

"The real work here is making everything PIC. Of course, if what matters is the API rather than speed, then achieving PIC is easy: just remove the asm."

Sodium is mostly a "just remove the asm" project, so it seems like djb doesn't hate the concept at least.

apaprocki2• 13 years ago

It is unfortunate so many projects have overlapping names, even playing off the same kind of puns.

This: NaCL + Sodium

Google: NaCl (Native Client) + Pepper

tptacek18• 13 years ago

DJB's NaCL is an academic project that dates at least back to 2008.

apaprocki2• 13 years ago

I'm just lamenting to no one in particular that in the universe of available words/acronyms/abbreviations to describe projects we have as many collisions as we do. If I had to choose (ugh), DJB's NaCL acronym is at least more specific to its purpose than NaCl meaning "Native Client" just to make a salt + pepper pun. I guess all I'm saying is that if I ever write an open-source slab allocator, I won't call it OpenSSl.

bathat• 13 years ago

It's not quite as bad as Maxima (http://en.wikipedia.org/wiki/Maxima_%28software%29), though, where it almost feels like the project was intentionally named in a way to make googling difficult.

newman3142• 13 years ago

Like Go?

jsdalton2• 13 years ago

Yeah, cryptography libraries in general are a pain to use and typically require a lot of seemingly arcane configuration and confusing (to the novice) setup.

I wrote this library recently (primarily to scratch an itch on another project), which really does nothing more than pass sane defaults to PyCrypto and eliminate crypto jargon:

https://github.com/jsdalton/secrets.py

Honestly I think it took more time to wrap my head around the simple use cases than it did to implement this wrapper once I did.

sneak• 13 years ago

Your key setup leaves a whole lot to be desired. (There's a very real reason that PBKDF2 exists.) Bad crypto libs like this are the EXACT reason that things like NaCl and KeyCzar were created.

Please don't do things like this.

PS: It's also not clear if you're doing the HMAC check in a time-invariant manner: https://github.com/jsdalton/secrets.py/blob/master/secrets/c...

See: http://codahale.com/a-lesson-in-timing-attacks/

tptacek18• 13 years ago

Stop using this and start using Sodium, which is more secure than your wrapper library in significant ways that illustrate why people shouldn't think of this stuff as "jargon" that they can just write wrappers for; for instance:

* It generates encryption keys insecurely instead of using a cryptographically secure KDF

* It leaks timing information on the MAC comparison

* It makes verification of messages optional; verification should never be optional (in your case, when verification is disabled, I think you have the CBC padding oracle vulnerability)

jsdalton2• 13 years ago

Thanks for your comments and recommendation. Your points (and my own failings here) illustrate why libraries like Sodium (which was not on my radar a few weeks ago) are a good thing.

As an ordinary developer, I just want to use a library like PyCrypto in a safe manner. When I described the language as "jargon" I did not mean to be dismissive of it, but rather to say that libraries like PyCrypto force you to make decisions about terms that are nothing more than jargon to a non-expert -- when really what a non-expert needs is an extremely simple API which makes those decisions on your behalf in a safe manner.

Thanks again and these are humbling lessons to learn.

DeepDuh• 13 years ago

If it keeps its promises this will be huge. To be honest it's almost too good to be true (and to have such an open license). Looking forward very much to using this in future projects.

newman3142• 13 years ago

Next step, openssh ported to use Sodium as an alternative?

tptacek18• 13 years ago

SSH is a good example of the kind of thing that can't use high-level crypto libraries, because the constructions required to implement SSH are part of the specification.

We're all waiting for Daniel Bernstein to replace SSH, but it hasn't happened yet.

jedisct17• 13 years ago

Do we need a replacement for SSH? The OpenSSH team is doing an amazing work. Besides making the tool as secure as possible, they are also improving the protocol. The recently added support for encrypt-then-MAC modes is a great step forward.

peripetylabs• 13 years ago

This is great, especially the Python bindings. I've been looking for something similar for a while. I was hoping to package just a reference (portable) implementation of 'crypto_secretbox', but the code seemed to be split up in multiple files and I couldn't understand their build system...

anonyfoo• 13 years ago

Excuse my ignorance -- I find myself fairly knowledgeable in crypto/security, but I'm confused by "Sodium also provides a secure, chroot()-resistant drop-in replacement for the arc4random() function family, including the ability to generate random numbers within a given interval with a nearly random distribution."

What does chroot have to do with arc4random()?

tsewlliw• 13 years ago

you can chroot yourself away from access to /dev/random

jedisct17• 13 years ago

Take a look at arc4random() implementations for a hint. The generator has to switch to a new key after (in the OpenBSD implementation, too lazy to check other implementations now) 1600000 bytes of output. How do you switch to a new key? OpenBSD implemented a sysctl to retrieve the output of the kernel prg. This is great, as it perfectly works in a chroot()ed environment.

Other operating systems don't necessarily provide the same facility. They will try accessing /dev/urandom, and if it doesn't exist, revert to something pretty bad, like read whatever is on the stack and use that as a key.

Sodium provides randombytes_random(), randombytes_uniform() and randombytes_buf() that you can use as drop-in replacements for arc4random(), arc4random_uniform() and arc4random_buf(). On Unix, the file descriptor to /dev/urandom will be kept open, and accessible after a chroot() call. On Windows, the crypto services provider will be transparently used instead.

nwmcsween• 13 years ago

Hmm linux provides AT_RANDOM from the kernel at program startup usually for randomization of stack canaries (but it can also be used for other things).

SeoxyS2• 13 years ago

I just completed a major cryptographic project overhauling the security at my company. (Re-doing our password hashing and authentication, and the protocol for all our networking and communications.)

The password hashing was easy: a portable implementation built as a static / shared C library + language bindings linking to OpenSSL's `libcrpyto` for PBKDF2 (with an alternate implementation using CommonCrypto on iOS / OSX).

The encryption and authentication layer for the communications was much tricker. The first draft was an implementation based on industry standards: RSA2048 + AES256. It needed to be portable to iOS in addition the various other platforms supported, and Apple has deprecated OpenSSL on iOS and OSX. Annoyingly, OpenSSL does not ship with darwin-arm support out of the box, so a custom compile was not an easy option either.

In the end, I ended up picking NaCl, and specifically `libsodium` as a portable implementation. The library, unlike OpenSSL, is beautifully designed and very easy to use, and implements asymmetric crypto functions (based on elliptic curves) which are actually much superior to RSA, providing much greater security for much shorter key length.

Libsodium is highly recommended.

jedisct17• 13 years ago

s/elliptical/elliptic/ Thanks for using libsodium!

SeoxyS2• 13 years ago

Thanks, fixed.

Btw: the CocoaPod for libsodium is broken. I had two problems: 1) the headers include path for the pod was not set correctly, and 2) one of the algorithm's alternate implementations would not compile with clang. Removing it and using the ref implementation instead did the trick though.

I might submit a patch / pull request to fix (1).

andrewcooke• 13 years ago

how is this better than keyczar?

keyczar is about to support python 3 (there's a patch) and i was planning to make simple-crypt delegate to keyczar (or just delete the project entirely, since its only reason for existence was nothing better existed on python 3, but people seem to be using it). should i delegate to this instead of keyczar? what is the difference?

http://www.keyczar.org/ https://pypi.python.org/pypi/simple-crypt

bascule9• 13 years ago

Keyczar is a library with similar goals to NaCl: relative ease-of-use for authenticated encryption.

NaCl is using more modern cryptographic algorithms (e.g. XSalsa20, Poly1305, and Curve25519). These provide faster encryption with smaller keys.

The elliptic curve cryptography, most notably, provides keys an order of magnitude (or two) smaller than what's available today with Keyczar/NaCl.

Also, the NSA recommends you switch from RSA to ECC:

http://www.nsa.gov/business/programs/elliptic_curve.shtml

sweis• 13 years ago

Keyczar has support for Java, Python, and C++ and uses the usual suspect standardized algorithms: AES, RSA, DSA, HMAC-SHA1. Google has been pretty active in maintaining Keyczar lately. ECC support should be added at some point.

As bascule said, NaCl / Sodium are using DJB's suite of algorithms, which are going to be faster and have smaller key sizes.

Introducing Sodium, a new cryptographic library

Related Stories