I revoked $1M worth of EV certificates

https://scotthelme.co.uk/extended-validation-not-so-extended/

by amaccuish • 7 years ago

153 66 7 years ago

scotthelme.co.uk

kadoban• 7 years ago

Another data point in the ongoing theme of EV certificates having basically no value.

The other big data points being: browsers don't want to display them differently, and users don't notice them being any different.

Now we have that nobody is really checking them that closely.

How much longer will these even exist?

Mathnerd314• 7 years ago

There was https://www.troyhunt.com/extended-validation-certificates-ar... from a year ago.

Max length of a certificate is 2 years, so probably 2020. Maybe 2021-2022 if you count stragglers and businesses that are locked into long contracts.

cmroanirgo• 7 years ago

Mr Hunt also updated this statement recently: https://www.troyhunt.com/extended-validation-certificates-ar... "Extended Validation Certificates are (Really, Really) Dead"

jillesvangurp• 7 years ago

Ours are about to expire. I replaced them with free amazon certificates recently. The process of getting and deploying certificates three years ago was super painful. The process of replacing these certificates was super easy and renewal is now automated. Things have improved a lot since the letsencrypt movement kicked off a few years ago.

donatj2• 7 years ago

I used to pay for certificates. Recently however, I got burned with a crappy Symantec one sold by Name resulting in near continuous customer reports of console errors in Chrome.

I fought with Name to get them to reissue it, they wouldn’t. I decided it was time to figure out how to setup Let’s Encrypt. They lost me as a customer, and I’ll likely never buy a certificate again.

CameronNemo• 7 years ago

I wish InCommon/internet2 supported ACME. The admins at my work are quite busy as is, and ACME would definitely make for one less factor to be constantly concerned with.

drdaeman• 7 years ago

browsers

EV certs are still useful for code signing. At least on Windows UAC warnings look differently if binary is signed with EV certificate.

GordonS2• 7 years ago

Does it?

Windows is my daily driver, and I'm security-minded - yet I haven't noticed different colours for UAC dialogs depending on signature type. I notice if it's not signed at all, of course, but otherwise... no?

I would imagine most typical end users are the same. Similar to how browsers are not actually, where they make it really obvious if a site doesn't use TLS, otherwise it all looks the same.

tialaramex5• 7 years ago

These certificates aren't in the Web PKI and so they aren't covered by (most of‡) the Web PKI's rules and there is no requirement to log them publicly before use.

For example they would lack the 1.3.6.1.5.5.7.3.1 EKU which says their purpose is to identify an SSL/TLS server.

The market for these certificates is pretty different from the Web PKI market.

‡ When they're issued by a CA which would also be trusted in the Web PKI some requirements cross over because of that, particularly Mozilla wants to be sure that certificates issued under these hierarchies are never "accidentally" valid in the Web PKI without obeying its rules.

weinzierl2• 7 years ago

How much longer will these even exist?

EVs will soon be gone when browsers stop recognizing them. We are already well on our way there. From the same author from a month ago [1]:

Both Google Chrome and Mozilla Firefox have announced that they plan to move the EV indicator out of their main UI.

[1] https://scotthelme.co.uk/gone-for-ever/

amenod• 7 years ago

I don't think that's an indication that their CAs will be removed. Correct me if I'm wrong, but they are not less safe than other certificates, they just don't add any additional value. So they will probably be treated the same as other certificates by browsers. If people will continue paying for them is another matter of course (but it might very well happen that they will have many customers for years to last).

mytailorisrich3• 7 years ago

browsers don't want to display them differently, and users don't notice them being any different.

That's the only thing that matters, really.

The selling point of EV certificates is that (1) they are very visible in the browser and (2) users care enough about that to make a difference to your business.

I haven't seen any surveys or A/B testing results about (2) but since (1) is no longer true this is moot.

It means you should just go for the cheapest certificate, which means free nowadays.

megous2• 7 years ago

Firefox displays them differently.

wbond• 7 years ago

Just yesterday someone asked if there was a way to verify if the email from a purchase was leading to the correct domain to log in to. They wanted to make sure they weren’t being phished. How I wished there was a simple button they could have clicked in their browser that would have shown a window with the company details for the website certificate.

Everybody rails on how useless EVs are, yet it seems like no browsers have spent any time over the past years even trying to improve the UX related to certs. I mean, sure they’ve changed the color and removed the ambiguous company name and country. Which makes you ask why did they ever display country in situations where that isn’t the jurisdiction that issues business licenses, such as in the US?

I mean, click on the lock and through N buttons, then expand various sections and read through dotted identifiers that only ASN.1 experts recognize (seriously, who though it would be a good idea to use OIDs for known fields in the UI?), and then maybe you’ll understand who the certificate is issued to.

At least in the US, when obtaining and EV cert you need the jurisdiction (state), business registration number, they call the business phone number. They basically do some research - usually you need a DUNs number, publicly listed phone number, etc. Wouldn’t it make sense to have a button that when you click it you see a user-friendly window showing the information about the company for OV/EV? I mean, sure, you can register Stripe in a different state - how about providing some statistics like what year the company was formed (usually only file with business registration), a link to the company registration details on the jurisdiction website, the top level domain Whois info, etc? If you wanted to get fancy you could provide a link to the user’s search engine with the company name as the search terms.

Instead it seems like we have a bunch of cryptographers (or maybe just infosec people) obsessed about 1. The money companies are charging for EV certs 2. that it is possible for mistakes to happen. This instead of improving how we present information about a company.

So since there isn’t a perfect, foolproof identity verification system for companies online, now every user online needs to do their own reconnaissance to make sure this is really the website they should be at.

tialaramex5• 7 years ago

Your proposal seems like it doesn't really change anything? Users are still left to "do their own reconnaissance".

Research suggests they don't do a very good job at that.

jeroenhd• 7 years ago

You're right in that a working EV system can indeed protect people. However, as this article suggests, even a basic check can be very hard for CAs to implement. If you can't even do a spell check on the state or country name, can you be trusted to verify the identity of those who rely on you to meticulously verify a company?

There's also the point https://stripe.ian.sh/ made: it's not just about registering the same name in a different state. People don't know in what state your (parent company) headquarters are so they would need to dig deep into the organisational structure to find out if the EV cert they're presented with isn't from a company with the same name in another state. Want to add more years of registration? Just buy an old, probably no longer used letterbox company and change the legal name. Whois info can be set to anything you want for most domain registrars as well. People will still need to find out if all the data matches.

There's also this demonstration of how EV can actually be used to help phishing by using the "green address bar" and the hilariously stupid Apple address bar design: https://www.typewritten.net/writer/ev-phishing/

I believe there is value in EV certificates if you can rely on the identity verification that they come with. Sadly, most providers of EV certificates seem to view them as a way to make more money, not as a tool to bring trust to the web.

An EV-like system will probably be reinvented in the future, giving us a fresh start on identify verification on the web without the crappy practises and standards that plague the EV system today.

Lastly, for such a system to work, all (big) companies need EV certificates. Even before browsers started cutting back on EV UI, nobody knew what site does or doesn't have EV. Some subdomains used different certificates, some didn't have certificates at all. How do you know if a (sub)domain is supposed to have an EV certificate? Will you tell people to just not log into Facebook anymore because they don't have an EV at their login page?

EV doesn't work because it's the wrong way around. For security indicators to work, you need to warn for danger instead of turning off the "everything is okay" light. The OV/EV system has been designed around the latter idea and as long as EV/OV isn't freely available all over the world, nobody's going to mark DV certificates as "insecure".

commandlinefan2• 7 years ago

improve the UX related to certs

They’re actively making them worse, actually. I can’t even view the details of the certificate chain in Chrome on iOS. I used to be able to save them from the browser on a computer, and now I can’t do that, either.

commandlinefan2• 7 years ago

When I saw that google, amazon, apple and Microsoft weren’t bothering with EV certs, I figured I didn’t need to either.

megous2• 7 years ago

Google's cert is not really domain validated either. They have their own CA.

Microsoft also has a CA.

Apple has EV and Amazon uses DigiCert, which is the only one that may be domain validated, perhaps.

smashah• 7 years ago

Are all of these EV issues not data entry errors?

teraflop• 7 years ago

Presumably they are, but the entire point of an EV certificate is that the company's information is supposed to be vetted and verified. If a single human error can cause an invalid certificate to be issued, it's not just a problem with that one certificate. It's potentially a sign of systemic issues that call into question the trustworthiness of all other certificates issued by that CA.

A signed certificate that contains invalid or incorrect data is considered to have been misissued according to the CA/Browser Forum's Baseline Requirements [1]. Misissuance not only obligates the CA to quickly revoke the certificate upon discovery, but also frequently leads to discussions in places like Mozilla's dev-security-policy mailing list. [2]

If you follow some of those discussions, you'll quickly see that browser vendors hold the CAs to fairly high standards. Blaming a misissuance on a "data entry error" is generally not considered sufficient or acceptable; they're expected to be able to explain why the errors were not caught by technical controls or verification processes, and how those defenses can be improved to prevent the same errors from recurring.

[1] https://cabforum.org/baseline-requirements-documents/

[2] https://groups.google.com/forum/#!forum/mozilla.dev.security...

donatj2• 7 years ago

I mean maybe I’m wrong here, I’m probably wrong, but 99% of people don’t know nor care about the difference between an EV certificate and a Will Issue certificate like Let’s Encrypt.

Is it really worth revoking these over tiny discrepancies, especially in cases where you can still determine the entity beyond a doubt?

StavrosK2• 7 years ago

Yes, because revoking over tiny discrepancies is the difference between the two. If the EV certificate can have any old crap on it, it's no different than any other type.

mkopinsky2• 7 years ago

If a private entity is issued a cert saying "Government Entity", is that a "tiny discrepancy"? https://twitter.com/SectigoHQ/status/1171808269528109058

tasssko• 7 years ago

Once upon a time i was interested in them my marketing team wanted them. The hassle of dealing with the people that vetted our company documents ensured I never did it again. Totally hopeless process and worse still it was a total waste of my time. Terrible user experience and we paid for the privilege. Today we use the free amazon certificates and it doesn’t cost me time any more. I keep waiting to hear how the certificate business is headed for ruin.

Theodores• 7 years ago

Let's encrypt has developers empowered to just get https done in a couple of minutes. This includes the time it takes to search for certbot syntax on stack overflow.

Years ago the certificate was a big deal in project specifications, project managers could lay out the FUD and trick clients into spending more money on more snake oil. This was a pain for developers as they would have to get the work done. Sometimes the client would not have access to the one or two email boxes needed for this archaic system so there would be an added hurdle of setting up info@example.com or whatever it was that was needed to get EV to happen.

Nobody misses those days and the pain and hassle just was not worth it. Certbot came along and made it so easy. Nobody questioned why they needed EV certificates, sometimes you cannot argue with free and easy.

Ayesh3• 7 years ago

As a developer/server admin, one my frequent tasks were to install SSL certificates my clients send to me. My clients used to get the certificate and email/Dropbox me the certs along with the private keys to me.

raverbashing2• 7 years ago

Edit: yes, I got the numbers mixed up

I really don't see why having a Danish number in a Swedish registered company is a big deal.

The phone might simply their contact for whomever is responsible for the certificate and it might be possible these contact is in Denmark.

I mean if it was a Canadian company with an US number and most likely they wouldn't even know.

Symbiote• 7 years ago

A Danish business registration number (CVR in Danish), not a phone number.

raverbashing2• 7 years ago

Ah thanks for the clarification

tialaramex5• 7 years ago

Scott isn't trying to mislead us here, but one sentence from the article seems like it needs expanding certainly for a lay audience like HN.

"One thing that we can do with CT logs is search all certificates for things that we're interested in"

The CT logs themselves implement only a terribly simple API and you can't search them per se. This makes good sense because they are required to hit really onerous availability and performance targets to remain "qualified" as logs. You definitely wouldn't want to put a sophisticated "search" feature in this code.

However, since they're public we can either read all the data (my employer does this) and process it however we like or there are two famous services which already did this for you, crt.sh (operated by Sectigo, one of the CAs) and Censys (from an ex-University of Michigan team). Scott used Censys for this article, and either Censys or crt.sh are good places to play around with this if you have a passing interest that may not last.

fanf2• 7 years ago

We get daily notifications of certificates from https://sslmate.com/certspotter/

Mojah• 7 years ago

We use a similar service called Oh Dear that does CT monitoring: https://ohdear.app

borumpilot• 7 years ago

Great service.

To add, I like the checks of https://sslping.com/ too.

gsylvie• 7 years ago

Little silly to say "I revoked $1M worth" since the price paid for each certificate includes unlimited renewals to fix problems like he identified. They are still worth $1M despite everything he did, they just have different private keys now and slightly improved data.

Maybe a better title would be, "Humans not 100% reliable about ensuring accurate information stored within thousands of data records, despite such accuracy kinda being the whole point."

mkopinsky2• 7 years ago

I know for a fact that at least one of those certs was replaced with a DV cert - once the team in question had to replace it, they realized there was no point in replacing it with an EV since the browsers are removing the EV UI anyway.[0] I'd bet that a meaningful proportion of the certs in question are similarly being replaced with DV.

[0] https://www.isc.upenn.edu/alerts-outages/planned-weblogin-ss...

_ME_• 7 years ago

I think the traditional concepts of CAs has showed again and again that it does not work. Yet cryptography is worthless if you can't validate who you are talking with. I would propose a system where multiple parties could sign the SSL-certificate and these would be shown as badges in the browser. The more badges the more trustworthy.

Say for example that you have an online store. Then the local tax-office could sign your certificate if you are a registered company in that country. If you have a contract with VISA, they could sign your certificate. The authority for your top-domain could also sign the certificate to prove you own the domain and so on.

And for a customer buying from this store, seeing these badges in the browser would let them decide how trustworthy the website is. I actually think with this solution getting the users to understand would be the least problem. The big problem would be to get developers to authenticate the certificates while doing back-end calls. I mean how many developers even does this today?

ben_w• 7 years ago

Number of badges is not going to help if you can’t trust any given badge — no matter what the details are, given how automated attacks are, it is reasonable to presume that as soon as you’ve worked out how to circumvent one, you’ve probably figured out how to circumvent more of them than will fit in a browser bar at once.

als0• 7 years ago

And for a customer buying from this store, seeing these badges in the browser would let them decide how trustworthy the website is. I actually think with this solution getting the users to understand would be the least problem.

Given that a non-technical user can't tell the difference between an EV cert and a regular CV, I can't understand why they would care about the actual number of badges. Sliding scales just don't work, they just care about whether it's safe to use a website or not.

The browser vendors _could_ establish a baseline for the number of badges that would make a site acceptable. But the economics would have to match real world incentives (e.g. small and large businesses), and protect against problems like sybil attacks...but then we're back to the heart of the problem.

rocqua• 7 years ago

I like the idea of multiple signatures, but mostly so it is much easier to drop trust in a CA.

dboreham• 7 years ago

You invented Web of Trust.

Spare_account• 7 years ago

Is there any kind of transparency index listing CAs and the number of mistakes they've made?

Ayesh3• 7 years ago

A single severe mistake can result the CA being kicked out, but CAs like Symantec were too big to fail, so browser vendors had to spend time and effort to kick them out.

There is CT logs of course. Even if the certs were issued by error or fraud, they must still be included in the log. There must be automated processes scanning these logs and validate the certs (x509lint, cablint, zlint, etc).

I suppose you can check CA CRLs or combined CRLs like Mozilla OneCRL to find recently revoked certs and lint them to see if there were issued with errors.

zokier• 7 years ago

So were the misissuances evenly spread among CAs or were there some that were clearly worse than others?

perlgeek• 7 years ago

At $work I maintain and develop a CMDB and the software that is used to manage it.

All of this sounds like very familiar data entry and maintenance problems.

In general, good data quality can only be achieved by giving the people who enter it some incentives, or make it impossible to enter wrong data (then data can still be missing).

For example, we have automation that configures monitoring and backup for hosts from our CMDB, and this information tends to be well maintained.

Other information that we need e.g. for compliance, but that no automation is tied to, tends to be of much less quality.

The second part is the feedback loop: if you enter some data incorrectly, and weeks or months later somebody complains that it's wrong -- that's too far off to efficiently learn from it.

We as an industry might have expected better from CAs, but we haven't pressured them into building the necessary feedback loops, or indeed given them incentives to provide higher-quality data in the EV certs.

peterwwillis• 7 years ago

It would seem that one could develop validation standards for certs that check for data entry errors (and missing data) and apply them at CAs so they can't issue them, and optionally at clients so they can't even accept them

cat1992• 7 years ago

higher-quality data in the EV certs.

Part of this is probably the fact that many people were opposed to the entire concept of EV certs as basically being a money grab by the dominant CA's from day 1, and indeed the trend is away from highlighting their existence in web user agents...

sp332• 7 years ago

Is it possible to get an EV cert without registering with a specific government database first?

Ayesh3• 7 years ago

It doesn't have to be a government list. Any recognized (vague measurement I know) will do. But you need to be registered in a country.individuals cannot take one.

jrochkind12• 7 years ago

Since the cert log is public, all of this is auditable, as indeed the OP audited it.

Should there be some formal/official mechanism of auditing _every_ EV issued?

Perhaps cert authorities should be required to pay for an auditor to audit their certs -- not just the process or a sampling of certs, but automated auditing of every cert. The problem with this of course is the usual 'auditor capture', if the auditor is getting paid by the entity getting audited, the incentive is not to continually approve the auditing, but to make all the audits pass.

In some sense, this is a success story -- the transparency of the cert issuing log allowed the OP to choose to audit some of them, voluntarily. But transparency of the cert log clearly isn't enough, it's been years of bad certs before someone decided to volunteer their time to audit some of them. Just transparency isn't enough without social/business structures in place such that they actually _are_ being checked.

(This has parallels/analogies with other things...)

cat1992• 7 years ago

auditing _every_ EV issued?

this is what Extended Validation is supposed to mean in the first place.. and lends credence to the argument that EV solves a problem that shouldn't exist to begin with for 'regular' certs

jrochkind12• 7 years ago

No, I am not talking about what EV is supposed to mean in the first place.

EV, in contrary to ordinary certs, is supposed to mean the cert authority verified the registrant really is the name on the cert. Ordinary certs the CA doesn't even claim they did that, they don't even claim that a Cert that says Joe Smith on it was registered by someone who's legal name is Joe Smith or a cert that says Widgets Inc on it was registered by a company with that name. The CA's are claiming to do a different thing with EV certs.

I am talking about a third-party auditing the EV certs to ensure the CA was doing what they committed to do. Like a third party audits a bank by analogy. The fact that banks get audited doesn't mean the bank is useless in the first place. But the auditors do (theoretically if the system is working) help keep them honest.

One of the points of the transparent log is to make this possible. But if nobody is actually checking... it is not quite accomplishing this point. The fact that a transparent log is a good thing so someone can audit doesn't mean the EV is "useless in the first place".

(An EV might be useless in the first place for other reasons, but not because auditing is a good thing).

mxpxrocks10• 7 years ago

Can someone explain how revocations work? how many there are? what the rules are - etc?

I revoked $1M worth of EV certificates

Related Stories