We build X.509 chains so you don't have to

https://blog.trailofbits.com/2024/01/25/we-build-x-509-chains-so-you-dont-have-to/

by ingve • 2 years ago

221 75 2 years ago

blog.trailofbits.com

caeruleus2• 2 years ago

Congratulations to the authors, this was a feature that was dearly missing from pyca/cryptography. It took a long time to get right.

For the history: https://github.com/pyca/cryptography/issues/2381

reaperhulk• 2 years ago

Thanks for posting! I had forgotten just how long a road this was, but I'm very pleased that the requirements we talked about years ago have been met. :)

That's not to say this is done! There's a set of features we still want to add and we'd like to gain some confidence in the APIs as structured before we mark them as stabilized.

caeruleus2• 2 years ago

Looking forward! Thanks for maintaining such an awesome library btw, most of my internal PKI (X.509 and OpenSSH) is managed with it. It's a bliss to work with, even more so compared to other crypto libs. :)

EGreg• 2 years ago

Do we have this in PHP? Javascript ?

tob_scott_a5• 2 years ago

I think getting it into cURL would be more impactful, since that's already tapped into by many OSS projects.

hlieberman• 2 years ago

I'm curious -- what was the inputs to the decision to redo the logic from scratch, rather than wrap something like the `webpki` crate?

woodruffw14• 2 years ago

This is a great question! The answer is twofold:

1. Compatibility: PyCA Cryptography has strict compatibility requirements, including being able to build with older versions of Rust that crates like webpki and rustls-webpki may not be interested in supporting.

2. Generality: PyCA Cryptography is a general purpose cryptographic toolbox, and the plan for X.509 validation is not intrinsically linked to just the Web PKI profiles. In the future, they may wish to make their APIs more flexible than a strictly CABF validator would require, meaning that they would either need to carry patches against `webpki` or get such flexibility upstreamed (to the detriment of the crate's single purpose).

(I am not a maintainer of Cryptography; this is my understanding of the reasons as one of the implementors of this feature.)

ivanr16• 2 years ago

Genuine question: does the `webpki` crate provide a good path building implementation?

tob_scott_a5• 2 years ago

Carcinize existing C and C++ X.509 users.

This could be game-changing for a lot of open source software.

I spent years avoiding X.509 (and ASN.1, for that matter) in my designs because every time someone I trust poked it, a remotely exploitable bug fell out. Most often, it was a Denial of Service issue rather than Remote Code Execution. Moving to Rust would demonstrably improve the security of the entire Internet.

You might be tempted to ask, "What about BouncyCastle?" (or similar queries).

Sure, you're not overwriting the EIP in most Java X.509 bugs, but check the release notes for X.509 and ASN.1 mentions: https://www.bouncycastle.org/releasenotes.html

When I worked for Amazon, we disclosed a few X.509-related vulnerabilities to projects that we almost found by accident.

otabdeveloper4• 2 years ago

Moving to Rust would ...

... do absolutely nothing to fix denial of service attacks.

tob_scott_a5• 2 years ago

By itself? No.

The other details covered in the blog post, however, would absolutely do something to fix denial of service attacks.

To wit: x509-limbo

woodruffw14• 2 years ago

I don't think this is true. Rust cannot prevent all possible forms of denial of service, but there are plenty of underlying DoS causes that Rust either outright eliminates (such as memory corruption without further control) or mitigates through stronger types.

A recent example of this is CVE-2024-0567 in GnuTLS: an invariant that otherwise would likely have been noticed at the type level is instead checked with an assert, leading to a remotely trigger-able DoS.

saurik• 2 years ago

Nor the other myriad of logic and parsing bugs that led to incorrect behavior (more than just denial of service) in the Java library that was somehow not as good as Rust :/.

bagels• 2 years ago

Exploiting a memory safety crash, leading to a downed service, is the first class of DOS that Rust can help with.

blibble4• 2 years ago

how would rust fix most of those issues?

they're logic bugs

tob_scott_a5• 2 years ago

woodruffw already wrote an excellent comment for this question: https://news.ycombinator.com/item?id=39131723

Rust isn't just memory-safety. The type system also coaxes developers towards eliminating some types of logic bugs.

Not all, granted, but it does move the needle.

woodruffw14• 2 years ago

The "classic" example of this is enums as sum types, rather than a thin wrapper over an integral type: Rust makes it possible to construct in invalid enum variant, whereas plenty of C logic bugs stem from taking untrusted user input and converting it into an enum variant.

My understanding is that Java doesn't allow this directly, but has adjacent historical deficiencies (e.g., not allowing exhaustive enumeration handling until recently).

e12e2• 2 years ago

Fantastic!

Curious about revocations though - the post doesn't mention it?

woodruffw14• 2 years ago

Thanks for calling this out. Revocations are not currently supported by the public API; we made a decision early in the MVP planning process to exclude them for complexity reasons.

(This is consistent with Go's crypto/x509, which does not do revocation checking as part of the `Verify` API.)

colechristensen• 2 years ago

Not planned for MVP or not planned ever?

ivanr16• 2 years ago

Do you think that it's possible to implement a good path building algorithm without support for revocation? For example, without revocation checking you may select a chain with a revoked certificate in it, only to discover that later. Then what? :) On the other hand, if you check the revocation status as part of the process, you can pivot to another certificate.

kachnuv_ocasek• 2 years ago

That's a whole another can of worms.

ak2174• 2 years ago

This is an absolute game changer for the ability to write secure PKI-based software in Python. Before now, you had to use pyOpenSSL (ignoring its warning to not use it for anything other than an SSL connection) and avoid the many footguns associated with building the certificate chain on your own, or leave Python entirely and talk to a separate Rust or Go process.

Thanks to Trail of Bits for pulling this off.

teddyh2• 2 years ago

Note: If your application does not absolutely need it, there are alternatives to X.509 certificates in TLS, like raw public keys (RFC 7250).

zokier• 2 years ago

Added to OpenSSL last November: https://www.openssl.org/blog/blog/2023/11/23/OpenSSL32/index...

teddyh2• 2 years ago

GnuTLS also has support for it since five years ago.

thadt• 2 years ago

While the contribution to Python Cryptography is incredibly awesome, I'd suggest that perhaps the lede is slightly buried. Having a large suite of well documented test vectors, including both positive and negative examples, raises the bar for the entire ecosystem. Go and OpenSSL are mentioned, but any other environment implementing X.509 now has another very well developed test suite that can be plugged in directly.

While I'm a huge fan of formal methods for developing secure code, IMHO well developed suites such as x509-limbo[1] with common, good, problematic, and outright sadistically wrong vectors are the way forward for developing robust cross platform protocols. These are the kinds of unit tests a spec should come with.

[1] https://x509-limbo.com

ivanr16• 2 years ago

Agreed. I don't see NIST's test suites [1] on the list, though. Maybe they're embedded somewhere and I've missed it?

[1] https://csrc.nist.gov/projects/pki-testing

EDIT If the authors are reading, are any of the test cases taken from Chrome's codebase?

woodruffw14• 2 years ago

(I'm one of the authors).

I looked at NIST's PKITS while building the test suite. We ultimately decided to punt on it for the first round of tests, primarily because it fell slightly out of scope of our MVP on Python Cryptography (being strictly a RFC 3280 suite). We instead chose to integrate the BetterTLS suite[1] (warning: very large page) for the MVP.

With that being said: x509-limbo is open source, and I would be overjoyed if someone added a new `nist::pkits` or similar namespace for these :-). I can't promise that it's an immediate priority for us to add ourselves, however.

are any of the test cases taken from Chrome's codebase?

Nope. That would also be a valuable thing to have here.

[1] https://x509-limbo.com/testcases/bettertls/

baby2• 2 years ago

Check Frankencerts as well if you're interested in that: https://github.com/sumanj/frankencert

thriftwy• 2 years ago

Meanwhile, JVM ecosystem looks down on the choice between OpenSSL and pure-Rust Python chain, because it have been using a managed, pure-Java X.509 chain for 25 years now.

I believe it also had various implementations under common API.

notaluddite• 2 years ago

Given the ambiguities and degrees of freedom different TLS clients can exercise during chain validation (mentioned in this post), we wrote a paper proposing that validation policies should be expressed as interchangeable Prolog programs: https://dl.acm.org/doi/pdf/10.1145/3548606.3560594

flumpcakes• 2 years ago

Aren't revocations a pretty important part of certificates, considering the whole point of this is security? I don't understand the rationale for skipping.

woodruffw14• 2 years ago

Revocations are a great idea. In practice, they're both operationally complicated and an unreliable source of truth about certificate validity. In practice, a lot of implementations (including Go's) simply ignore them.

'mcpherrim posted a comment below[1] with some additional information, including changes to the Web PKI and CRL distribution techniques that may eventually make them more useful.

[1] https://news.ycombinator.com/item?id=39132430

ivanr16• 2 years ago

Any chance you could expand on "In practice, they're both operationally complicated and an unreliable source of truth about certificate validity."?

baby2• 2 years ago

can OpenSSL use this as well? This would be amazing

im3w1l• 2 years ago

I'm sure it was even worse before, but I'm not entirely happy with the look of that.

* I shouldn't have to care where the certificates are stored. Just load the os default ones without asking me.

* I shouldn't have to know what a pem is, and I shouldn't have to open() one.

* I don't want a PolicyBuilder. Just give me the normal policy.

* I shouldn't have to construct some verifier object specific to a given dns. I don't want to verify a billion certificates for one domain, just one.

* I shouldn't have to know what an untrusted intermediary is.

Here's what I think it should look like

  verifier = Verifier()  # constructs a verifier with sensible defaults, overrides are possible though
  verifier.verify(chain, "cryptography.io")

paragon_init• 2 years ago

* I shouldn't have to care where the certificates are stored. Just load the os default ones without asking me.
* I shouldn't have to know what a pem is, and I shouldn't have to open() one.

Agreed, but what you're requesting is separate from the work being discussed in this blog post, and both are actually compatible.

For the PHP community, we made Certainty - https://github.com/paragonie/certainty

You can just...

  <?php
  use ParagonIE\Certainty\RemoteFetch;
  // cURL boilerplate
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);

  // Fetch the latest CACert bundle, verify its authenticity, save it locally  
  $fetcher = new RemoteFetch('/path/to/certainty/data');
  $latestCACertBundle = $fetcher->getLatestBundle();
  curl_setopt($ch, CURLOPT_CAINFO, $latestCACertBundle->getFilePath());

Writing a Python client should be relatively straightforward, should anyone want to.

For Python specifically, there may be some value in storing the relevant PEM files into something like SigStore. That could be an easier proposal for the PyCA team to consider.

RustyRussell• 2 years ago

This looks great!

I wanted to bundle a chain of certs, check that the leaf was <some domain> and its key signed <some data>, and that the bundle provided a path to a locally-trusted root. This allows third-party serving of signed data, rather than having to trade your IP address for validation.

Doing so with existing tools was so terrible that I was pretty sure my result was grossly insecure :(

I look forward to the promised lower-level APIs!

We build X.509 chains so you don't have to

Related Stories