sleevi

All the time. Many CA distrust events involved some degree of “amateurs” reporting issues. While I hesitate to call commenters like agwa an amateur, it certainly was not professionally sponsored work by root programs or CAs. This is a key thing that Certificate Transparency enables: amateurs, academics, and the public at large to report CA issues.

At the same time, it sounds like the issues you describe aren’t CA/issuance issues, but rather, simple misconfigurations. Those aren’t incidents for the ecosystem, although definitely can be disruptive to the site, but I also wouldn’t expect them to call trust or identity into disrepute. That’d be like arguing my drivers license is invalid if I handed you my passport; giving you the wrong doc doesn’t invalidate the claims of either, just doesn’t address your need.

Because it wasn’t actually a server misconfiguration, nor was it, as others have speculated, about Postel’s Law.

The way X.509 was designed - to the very first version - was the notion that you have your set of CAs you trust, I have my set, and they’re different. Instead of using The Directory to resolve the path from your cert to someone I trust, PKIX (RFC 2459-et-al) defined AIA.

So the intent here was that there’s no “one right chain to rule them all”: there’s _your_ chain to your root, _my_ chain to my root, all for the same cert, using cross-certificates.

Browsers adopted X.509 before PKIX existed, and they assumed just enough of the model to get things to work. The standards were developed after, and the major vendors didn’t all update their code to match the standards. Microsoft, Sun, and many government focused customers did (and used the NIST PKITS test to prove it), Netscape/later Mozilla and OpenSSL did not: they kept their existing “works for me” implementations.

https://medium.com/@sleevi_/path-building-vs-path-verifying-... Discusses this a bit more. In modern times, the TLS RFCs better reflect that there’s no “one right chain to rule them all”. Even if you or I aren’t running our own roots that we use to cross-sign CAs we trust, we still have different browsers/trust stores taking different paths, and even in the same browser, different versions of the trust store necessitating different intermediates.

TLS has no way of negotiating what the _client’s_ trust store is in a performant, privacy-preserving way. https://datatracker.ietf.org/doc/draft-kampanakis-tls-scas-l... or https://datatracker.ietf.org/doc/draft-davidben-tls-merkle-t... are explorations of the problem space above, though: how to have the server understand what the client will trust, so it can send the right certificate (… and omit the chain)

Because this is not the “start of drafting” but roughly “final text that is largely a rubber stamp approval.” This is the output of having been through the the trilogue process - where the Parliament would/has shut things down before - and not been shut down.

[1] https://en.m.wikipedia.org/wiki/Formal_trilogue_meeting

and the recent legislation coming from there seems very inspired by the great firewall.

https://www.europarl.europa.eu/RegData/etudes/STUD/2020/6487...

The EU is currently proposing to mandate the inclusion of roots that have been government approved, and to limit browsers from removing/distrusting them without notice/approval.

https://www.eff.org/deeplinks/2022/12/eidas-20-sets-dangerou...

It’s actually a huge issue - look at how eliminating a key difficulty in obtaining certificates massively increased HTTPS adoption (via LetsEncrypt and others)

Similarly, automation affects how easy or hard it is to replace a CA, for example, if moving to distrust a CA. If you rely on QWAC attributes, you can only use QWAC CAs, and changing CAs becomes significantly more complex.

The audit issue is definitely an issue: the audits used are fundamentally different than what browsers try to achieve, and so having to adopt the lower standard definitely impacts user security. However, my point was that in addition to those concerns, the technical design itself results in less robust and less agile systems, and that makes things less secure.

Yes, the current regulation is targeted at government sites authenticating citizens, but the goal with these revisions is to require VLOPs to support this, along with allowing them the ability to require this for all websites. The original roadmap called out by the European Agency for Cybersecurity (ENISA) suggests a long-term goal of making this mandatory, effectively reviving the idea of the “Internet drivers license” (for users) and “Authorized domestic website” (for servers).

Source: https://www.enisa.europa.eu/publications/qualified-website-a...

The QWACs can be issued by anyone who meets the minimum requirements, which are substantially less than those required for TLS server CAs in browsers. So while it’s true that banks can issue these, in practice there are many small companies with fewer than a thousand or so certs out there which have the same requirement that they must be accepted.

The eID certificates do come with probative (legal) effect, but this is where it gets complicated.

If the CA is hacked or screws up, yes, the CA is liable. But only if you did everything you were supposed to, such as checking every element of the certificate. These certificates have a variety of fields, such as “liability only up to XX euros”, and you (the site or user) are liable if you use it for more than that.

PSD2 has shown that the standards are a nightmare to fully implement. https://wso2.com/blogs/thesource/all-you-need-to-know-about-... gives a useful overview of how it’s worked for PSD2, and the new Digital Identity Framework/eIDAS Revisions proposes to make that the approach the standard everywhere.

In practice, this means that the server accepting your certificate needs to implement all of this correctly (spoiler: they don’t), or they bear the liability if the CA gets hacked - and they can’t distrust that CA. It also means the CA potentially learns every site you visit, because the sites have to check with the CA (if using OCSP).

Of course, if the government themselves directed the CA to misissue - e.g. at the direction of law enforcement - no such liability would be presumed, because it was a presumably lawful issuance.

The proposed regulation requires that QWACs MUST be accepted and recognized as such, such as using the European List of Trusted Lists as part of the root store.

That is, if a QWAC is issued by a CA that is not part of the browser root store, it must not be rejected (as any other untrusted certificate would be).

One element that results in less security is that it becomes more difficult to replace.

For example, QWACs cannot legally be automated (e.g. via ACME), because of certain restrictions applied to needing to validate the natural or legal person making the certificate request. This actually was an issue for one CA (BuyPass) that tried to support ACME but ran afoul of the framework.

While originally QWACs were proposed as optional, regulation such as PSD2 attempts to make them mandatory for (financial services) servers to obtain. If one of those keys is compromised, then the server wishing you obtain a replacement certificate may have to wait weeks to obtain such a certificate, or make an in-person visit to the CA (e.g. the post office).

A considerable number of compromised or misissued certificates have failed to been revoked on the industry-agreed upon timelines (24 hours or 5 days, depending), because of challenges CAs have faced because their customers haven’t (or legally can’t) automate replacement, and because the additional information in the certificate requires manual validation, despite having no technical impact on the TLS connection.

Yes. It requires the EU Trustmark, a logo designed through a secondary-school competition, to be displayed with certain colors and sizing, as directed through Implementing Acts (which have the force of law, but decided at the Commission level).

The draft revisions actually propose such authentication to be mandatory to implement for service providers if their users would like to use it.

That is, it specifically targets websites (particularly Very Large Online Platforms) that they MUST accept such ID in lieu of an email or password, at the user’s request. This was part of the original motivation for the revisions, to target “Sign in with Facebook” or “Sign in with Google” and require such sites also offer a “Login with EU” option.

Source: https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=COM%3A20...

https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=COM%3A20...

Do you have links to documentation on the Apple Pay requirement?

That sounds like Apple Pay is encouraging certificate pinning, and I suspect the Apple Root Program may have opinions to the contrary, given how it puts Apple users at risk to encourage pinning.

As others have noted, many of these issues are fundamental to XML DSig, which is insecure by design. [1]

However, the “what does the future hold” of OIDC is not much brighter. OIDC is based on JSON Web Tokens (JWT), which manages to avoid some of these issues (e.g. signs the encoded value), but introduces new ones (JSON interpretation bugs, algorithm substitution bugs, etc). They’re similarly terrible by design [2].

However, what OIDC does relating to signing is far worse. In many OIDC deployments, the idea is you use something called “OIDC Discovery” [3] to discover the expected signing keys for the OIDC server. You fetch those regularly (e.g. daily), and do so over TLS. With SAML, you exchange certificates, and then rotate them every 2-3 years (with things blowing up on expiration), but with OIDC, you often end up using OIDC-Discovery, and thus can change keys daily.

This means that a single malicious TLS certificate can be used to MITM your OIDC Discovery exchange, and from there, impersonate any user from the identity provider to your system, the relying party.

I spend my days in the TLS trenches, working to improve the CA ecosystem, but I absolutely would not trust the security of all users to a TLS certificate. The reality is that BGP hijacks are still a regular thing, as are registrar hijacks. Even if you find out about a malicious certificate (via Certificate Transparency), and revoke it, virtually none of your tools doing the OIDC-Discovery fetch (like programming languages or curl) support revocation checking, and even if they did, it doesn’t work at Internet scale. To deal with this problem, some relying parties do a form of poor-man’s certificate pinning, but now they’re at risk of even greater operational failures than SAML expiration in the start.

In practice, it seems plenty of OIDC clients just shrug and go “yolo” - if they’re talking TLS to the IDP, that’s good enough, and no need to bother with signature validation of the assertion at all.

For all my hatred of XML DSig and SAML, I’ve seen few auth standards as bad as OIDC: because it looks good, but is hell to implement correctly. At least with SAML, you know it looks bad to begin with, so you’re hopefully on guard.

[1] https://www.nccgroup.com/globalassets/resources/us/presentat... [2] https://news.ycombinator.com/item?id=14292223 [3] https://openid.net/specs/openid-connect-discovery-1_0.html

Example app: NuGet for .NET on Linux and MacOS, from Microsoft: https://github.com/NuGet/Announcements/issues/56

It used SSL/TLS and S/MIME roots to verify code signing and timestamping responses. When Symantec, which was removed for TLS trust, was also removed for S/MIME, NuGet broke, because it was no longer able to verify the TSA signature.

As covered in https://github.com/NuGet/Home/issues/10504 , this then led some Linux distros, notably Debian/Ubuntu, to re-add Symantec.

Any application using the ca-certificates package thus end up trusting CAs that Mozilla does not trust, despite being derived from the Mozilla Root Store.

So the news is already out there, this was just a reminder to folks to not do silly things like Microsoft did.

Using (defined) properties of the TLS ClientHello to determine how the server will respond.

For example, changing the certificate used based on the ALPN identity, the SNI server host, and the advertised client ciphersuites.

No more related to Android than any other Chrome supported OS.

The post linked in the article, https://g.co/chrome/root-policy makes it clear: the goal is to provide a consistent, cross-platform experience to the Web.

Users, and developers, hate “it doesn’t work on my machine” bugs, which are incredibly common. Sites that work on Chrome on macOS 10, but not macOS 11, or work on Chrome on Windows 10 but not Chrome on Windows 7. Giving developers and site operators predictable guidance, so that it “works in Chrome, works in Firefox” is good, no different than what Web Platform features you can use as https://caniuse.com

Yes, Android is important, but it’s a bit like saying the forest exists because of this specific tree here, when in fact it’s made up of hundreds of trees, of all sorts of types.

https://cloud.google.com/docs/chrome-enterprise/policies?pol...

There’s more that could be said, but I can see this is an emotionally loaded subject for you, so perhaps it’s best dealt with as it rolls out. Every feature has a cost: to complexity (CTLs are notoriously ‘weird’ on Windows, for example), to security (LDAP is a terrible protocol due to BER), or simply in the maintenance costs that prevent security improvements. Software engineering is about weighing those trade-offs: do you implement the feature that 10-100 users use, or do you spend time improving things for the other billion users? There are limits to what can be supported, and limits to what is even good to support in the first place, and while such pragmatism may seem like arrogance, the reality is less malicious: it just isn’t good software.

In any event, making sure usage statistics are enabled, and bugs are filed when actually, is a good way to help prioritize.

Some of this isn’t correct (e.g. Firefox totally reads HKCU - https://searchfox.org/mozilla-central/source/security/manage... - which is necessary to work with AV ), but I’ll the the first to tell you that a number of these things aren’t planned to be supported, because the goal is not to reimplement CAPI and it’s 200+ undocumented (except by support contract) flags and features. LDAP CDPs are definitely a no-go, for example. CTLs, as another, are almost exclusively for server apps like IIS, and don’t correctly work in Chrome today, so no regression there. Definitely, making sure to enable metrics, and report issues, is a good way to help prioritize things that are important vs things that are possible, but profoundly ill-considered.

Happy to be up front here: as we call out in https://g.co/chrome/root-policy , the goal is to support the things the user installed and authorized.

The priority of constituencies for the Web is a little different than on Android. Android gives a lot of deference to app developers, even against user wishes (think things like screenshot detection or making it hard to intercept traffic without rooting). This gives it a more “secure” reputation for some purposes, but certainly isn’t always what users want. The web has a different set of priorities, as spelled out in https://www.w3.org/TR/html-design-principles/#priority-of-co...

I’m not going to argue which is better, but I will acknowledge there are tradeoffs in both. Chrome, the browser, tries to be the user’s agent to the Web.

Sometimes, however, being the users agent isn’t always cut and dry obvious. Extensions are a great way to empower users to mix the Web as they want, but they can also be a really harmful abuse vector. Finding the balance is tricky, and I know folks aren’t always happy with how that balance is struck, but a lot of thought and community engagement definitely goes in to it.

Now, how does this relate to what you were speculating about? Well, installing roots is similarly one of those things that’s dual-use. I love tools like Fiddler and CharlesProxy, and have fought hard to make sure they work well (see the token binding discussions, which would have totally broke these tools). These tools use locally installed certs to really empower developers and users, and I don’t want to break that! But there’s also stuff like https://security.googleblog.com/2019/08/protecting-chrome-us... , which can completely destroy users’ privacy and security, using locally installed roots. So, like with extensions, there’s a balance to be had, between enabling great use cases, while also protecting users from abusive things.

Obviously, as you note, problems like this could be “solved” by not allowing locally installed roots at all. When Kazakhstan required a root to be installed to compromise traffic, for example, Android and iOS users were largely protected precisely because “install this root” doesn’t really work as KZ intended. That’s great! But it’s also got downsides, like breaking Fiddler and Charles.

I can absolutely say that the reason for this new root program is exactly what is said on the policy, and similar to Mozilla’s explanation at https://blog.mozilla.org/security/2019/02/14/why-does-mozill... . This isn’t about trying to block all locally installed certs, like Android, but about providing good, safe, cross-platform and interoperable defaults.

But I also will acknowledge that there totally are concerns about abusive roots, whether from Unwanted Software, device compromise, or the like, and this will give better tools to help communicate, explain, and warn users as appropriate, so they have a chance to make an informed decision. For example, this can close off quite a few abuse vectors for malware, today, and that’s already a win. However, any big changes to how locally installed roots work are quite a while away, and, like so many of the changes that involve complex tradeoffs, will involve a lot of research to understand users’ needs and desires and use cases. So no, this isn’t some secret plot to prevent users from managing their device as they see fit, or from choosing the CAs they trust.

No, that’s not it: none of the code has landed yet It’s a big enough change that we’re explaining before landing.

We’re taking an approach similar to Firefox’s implementation ( https://searchfox.org/mozilla-central/source/security/manage... ), which I helped advise on in the long ago days. It may end up using like that bool, but if you read through the commit history for how that code in Chrome works, you can see there are a lot of sharp edges.

https://g.co/chrome/root-policy , which is linked to in that mail, states in the third paragraph:

If you’re an enterprise managing trusted CAs for your organization, including locally installed enterprise CAs, the policies described in this document do not apply to your CA. No changes are currently planned for how enterprise administrators manage those CAs within Chrome. CAs that have been installed by the device owner or administrator into the operating system trust store are expected to continue to work as they do today.

The overlay is only used by Chrome on ChromeOS (the doc goes into some of the details there), so effectively, the “Chrome OS” root store is being sunset.

On a practical, technical level, I’ll probably just leave the overlay in place because libnss gets grumpy when there’s no ckbi shared object to load. Chrome itself won’t be consulting ckbi (i.e. it will take the same path as Linux, which will be to consult the Chrome store, even on Chromium OS builds, and only use NSS for user-added roots and client certs).

Whether we keep that overlay in sync with the Chrome store, or stop patching it and let it match upstream, it’s neither here nor there, and mostly just about what ends up being easiest and causing the least confusion. I’m lazy, but in practice the contents there won’t matter, since it’ll be baked into Chrome proper.

The etc-based store is a little more complicated, but it’s also not used by Chrome on CrOS, nor by the system apps that only talk to Google (they use the Google service set from pki.goog). Any code using that set doesn’t use the Chromium cert verifier, which as the mail and https://g.co/chrome/root-policy explain, go sort of hand in hand with the root program, so it may just continue to use upstream as it does today.

Firefox has historically checked OCSP by default everywhere but for Firefox mobile, where it was only checked for EV certs.

With the introduction of CRLite, the default is disabled, but those using Firefox with internal-use private CAs on local networks can renable via preferences, which can also be controlled by enterprise policies and tooling.

https://cloud.google.com/docs/chrome-enterprise/policies/?po...

1. Legacy

2. Trust Domains (half completed, but littered throughout the code as nss3 prefix). This was being lead by Sun and stopped when Oracle acquired them.

3. libpkix: This was done by porting Java code to C using preprocessor macros to simulate exception handling. It implemented path discovery, and not just verification, and was used by Firefox for EV processing (AIUI), and was always used by Chrome on Linux/ChromeOS (until recently replaced with the Chromium built-in verifier)

4. mozilla::pkix, which started off as Brian Smith’s insanity::pkix rewrite of a minimalist path builder/verifier. I can’t remember if this launched while Brian was still at Mozilla or after he had left, but Brian would later take the approach he used for insanity::pkix when writing the Rust webpki project ( https://github.com/briansmith/webpki )

Each of the above APIs had significantly different interfaces for controlling verification. The trust domain stuff wasn’t as visible, because it was only half-completed.

https://medium.com/@sleevi_/path-building-vs-path-verifying-... also has some utilities to visualize this using JS to explore these relationships, and understand the code tradeoffs.

I think you’re right for questioning the end-state, but you may be missing the current status quo.

The current status quo of multi-purpose roots is that supervision of a root, by a browser, auditor, and the CA themselves, constantly has to consider all the purposes a certificate is trusted for when considering the effective policies and design.

The classic example I give is that multiple CAs would have policies like “If it’s intended for TLS server auth, we put the domain name in the CN field and make sure it’s a real domain name. If it’s for document signing, we put the company name in the CN field and make sure it’s a real company name”, and with no further distinction between the two. So what happens when you have a company named “pets.com” - did they validate the domain or not?

What about if they misissued a TLS certificate. For TLS, you can quantify the impact, and you can move to distrust the CA, only thinking about how TLS works in your application. But what if they also issued certificates for lawyers used in judicial proceedings, and rely on the CA being trusted in the OS? This was the exact problem with DigiNotar, and why it took Microsoft over a month to fully respond, where Google and Mozilla took days.

We see similar issues with audits, and CAs having sub-CAs not “intended” to issue a particular type of certificate, but totally doing so.

For lack of a better analogy, and since earlier in this post I was talking about API design, the choice of singular trust purposes is like the S in SOLID. You want a single responsibility, clear, and do that well. It makes it easier to change and evolve that API when you don’t have to worry about breaking 30 odd unrelated use cases. And it lets you narrowly focus the supervision to the task at hand.

Eh, my point was just that:

  - Policy is largely encapsulated on the certificate properties in the root store
    - Local Policy is implemented via registry keys, with somewhere like 100+ odd registry keys (... many undocumented, as they implement customer-specific features whose documentation is provided under a MSFT support agreement)
  - Policy is a mix of documented (via WinCrypt.h) and undocumented flags
    - The reason for not documenting some of this is that they’re flags that Microsoft may or may not support; that is, they’re implementation details
    - CAPI provides, by design, several ways to fully replace their policies, either for single trust purposes or for all
  - The UI exposes a suitable “general purpose” expression

I can also totally understand the complaint that this isn’t all meticulously documented. Not everyone can ring up their pet Microsoft engineer to get documentation and show how it connects to a problem that Microsoft benefits from helping us solve (the linked-to crt.sh bug, which as a side-effect helps provide greater automation for Microsoft and the CAs they supervise). However, Microsoft also, from the get-go, designed it to be extensible so you could replace this.

Importantly, Microsoft has been able to roll out new features, and remove trust in CAs, without major re-engineering work. That was the “beautiful engineering” part of my comment. They defined a stable API that could be easily extended, or even wholesale replaced, in the Windows 2000/XP era. That API has brought considerable improvements WITHOUT requiring rewriting your code for Windows Vista, 7, 8, or 10 to leverage that. That’s a considerable difference from some other libraries and tools; LibreSSL was hugely constrained fixing OpenSSLs broken chain building, Mozilla Firefox/NSS has undergone three complete and separate rewrites from the ground up of the engine, Apple deprecated dozens of APIs when they ported iOS’s verifier to macOS, etc. As an engineer, 20+ years of stability IS impressive!