WoSign Incidents Report Update [pdf]

https://www.wosign.com/report/WoSign_Incident_Report_Update_07102016.pdf

by xnyhps • 10 years ago

113 74 10 years ago

wosign.com

cperciva• 10 years ago

Is the CEO stepping down because fraudulent certificates were signed, or is he stepping down because they got caught?

hueving2• 10 years ago

The latter. Otherwise he would have stepped down weeks ago.

mcbits• 10 years ago

He is being "relieved of his duties as CEO" according to the report. Saying he's "stepping down" is IMHO like saying a fugitive "turned himself in" when police arrived at his house with an arrest warrant. (Although I don't think he is being accused of anything criminal here.) Most of the incidents appear to stem from technical errors and poorly implemented processes, but the whole list points to his inability to lead this kind of company effectively.

I'm always skeptical of whether replacing a few executives can actually fix the cultural problems that were fostered/ignored in a company over time. The new leadership has to overcome a lot of inertia, and that's assuming they're any better than the old. They're also still answering to the same investors and pressures.

zamber• 10 years ago

Cross linking the earlier WoSign scandal reports: https://news.ycombinator.com/item?id=12389573 https://news.ycombinator.com/item?id=12582534 https://news.ycombinator.com/item?id=12617659

So if I get it right, WoSign will cease to exist as a CA given the 1y probation proposed by Mozilla and the general distrust that will follow?

inimino• 10 years ago

That was Mozilla's action plan, this is Wosign's response, now it is back in Mozilla's court.

gogopuppygogo• 10 years ago

Mirror of the PDF version of the incident report if they take down the original or it goes down: http://clicky.strapr.com/3g1s1y1k0Y2J

dredmorbius2• 10 years ago

This story is a bit of a mess to make sense of coming in cold and reading a Google Groups summary. Here's my read, which may help clarify the story for others.

Mozilla have an excellent explanation document covering the backdated certs in detail here: https://docs.google.com/document/d/1C6BlmbeQfn4a9zydVi2UvjBG...

(Thanks to @xnyhps for the link in a reply to this comment.)

WoSign, described elsewhere as China's largest certificates authority, are a CA who have been found to have backdated SHA1 ceritificates to work around browser restrictions on SHA1 cert issueances. SHA1 is no longer considered secure. Resolution of that issue is discussed in new mozilla.dev.security.policy Usenet group peered by Google Groups: https://groups.google.com/forum/#!msg/mozilla.dev.security.p...

A better source for WoSign's update to the story is in the PDF posted to the newsgroup, here: https://www.wosign.com/report/WoSign_Incident_Report_Update_...

Titled "WoSign Incidents Report Update". Which is even less descriptive than the title presently given on this HN post, though perhaps what HN posting guidelines prefer. I'll let @dang wrestle his conscience on that one.

In that document are several issues listed, the one relevant to this HN post appears to be:

"9. Issue S: Backdated SHA-1 Certs (January 2016)

"WoSign has issued certificates after January 1st 2016 but backdated the notBefore date to be in December 2015. This has the effect of avoiding the blocks in browsers regarding SHA-1 certs issued after January 1st 2016. The number of certs affected is probably 67, but may be a few more or less."

Following down from there, several corporate restructuring steps are mentioned, including:

360’s Corporate Development team has been notified to execute the process to legally separate Wosign and Startcom and to begin executing personnel reassignments. StartCom’s chairman will be Xiaosheng Tan (Chief Security Officer of Qihoo 360). StartCom’s CEO will be Inigo Barreira (formerly GM of StartCom Europe). Richard Wang will be relieved of his duties as CEO of WoSign.

There is background on the story from:

"WoSign Mis-Issued SHA-1 SSL Certificates [Updated]" (August 24, 2016) https://www.thesslstore.com/blog/wosign-mis-issued-sha-1-ssl...

"Mozilla Ready to Ban WoSign Certificates for One Year After Shady Behavior" (September 26, 2016)

The second article details Mozilla's issues with WoSign, including purchase of an Israeli CA, StartCom http://news.softpedia.com/news/mozilla-ready-to-ban-wosign-c...

I'm not claiming anything other than a 15 minute familiarity with the situation here. I may have heard earlier rumblings but really haven't followed this at all and wasn't consciously aware of particulars.

xnyhps• 10 years ago

The best place to start with this story would be this very well writen document by Mozilla: https://docs.google.com/document/d/1C6BlmbeQfn4a9zydVi2UvjBG.... This is WoSign's first public response after that document was published.

dredmorbius2• 10 years ago

Thanks, I've updated my comment with that.

ctz2• 10 years ago

"Many customers in China find it important to use a domestic CA for purposes of security."

That's not how the CA system works. Your security is unaffected by what CA you choose; it is invariably the minimum of all trusted CAs.

hueving2• 10 years ago

It certainly is if you strip down the certs trusted.

ctz2• 10 years ago

Well, true. But that would mainly affect availability :)

angry_octet5• 10 years ago

Which is a good reason to trust fewer CAs, especially those from countries which have poor transparency.

pfg5• 10 years ago

I honestly don't think the location should be a factor. How many countries are there that don't have things like national security letters?

I'd rather have a system where the same criteria apply to all CAs independent of their jurisdiction, coupled with mechanisms that guarantee transparency (like Certificate Transparency) and stuff like key pinning.

hannob3• 10 years ago

True. But I've heard this before by European providers avoiding US-based certificates. That doesn't make it any better, but this is a persistent myth, not limited to chinese customers.

ajdlinux• 10 years ago

Direct link to PDF report: https://www.wosign.com/report/WoSign_Incident_Report_Update_...

tarancato2• 10 years ago

So much money they are making and they can't hire anybody who can write English properly or write a PDF that is not composed of a handful of font faces and sizes.

I also find it funny they have fired the CEO (the PDF does not say he stepped down voluntarily) but he's the one sending that link to the mailing list. I call bs.

johnnyfaehell3• 10 years ago

So much money they are making and they can't hire anybody who can write English properly

If you're going to make fun of someone's English then you should probably write better English yourself. At the end of the day English isn't their first language and they'll probably very rarely have to use English. I think some slack should be given on this front.

tarancato2• 10 years ago

I don't speak English natively, and any corrections are welcome. On the other hand I don't run a multi-million business based on writing comments on Hacker News so currently hiring a PR person to proofread these comments is not within the boundaries of my budget.

WoSign runs a business that is basically based on trust, and Nigerian scammers have sent me PDFs that looked far more convincing and trustable than that one WoSign posted.

Normal_gaussian• 10 years ago

tarancato isn't making any money from these comments, so it seems reasonable that they are not using some form of proofreader.

geofft• 10 years ago

and they'll probably very rarely have to use English.

The CA/Browser Forum Baseline Requirements and the mailing list discussions all happen in English. The major root programs all use English as their language of communication, to the best of my knowledge. The SSL/TLS and X.509 specifications are in English. The major browsers and TLS stacks have technical documentation, code comments, and code review in English. mozilla.dev.security.policy is in English. The international language of scientific research (including cryptographic research) is in English. The CVE program is in English. All of these are reasons not only to have people who are fluent in technical English, but to make hiring such people a priority.

In particular, the inability to communicate precise, subtle technical concepts was very relevant to some of the problems here, like "You may not issue SHA-1 certs after this date" meaning the actual, calendar date of issuance and not the recorded validity period in the certificate. (There was also an element of malice, but it could have gotten sorted faster if communication were easier.) Even if all the others were in some other language, the CA/Browser Forum alone would be enough to make anyone who wants to be a CA (or an HTTPS client implementor) must make fluency in technical English a priority.

It is, of course, highly unfortunate that people from non-Anglophone countries are at a disadvantage here. In an ideal world we wouldn't have that disadvantage. But there doesn't seem to be a way around picking a language to be the scientific lingua franca.

hannob3• 10 years ago

There are a lot of reasons to criticize Wosign. Lack of English skills isn't one of them.

I didn't realize this until I first visited China, but there are parts of the world where it's really hard to find people fluent in English. If you require people to be fluent in English in order to participate in running the Internet you're locking a large number of world regions out.

wbl• 10 years ago

Learn. You only need a handful of people to read the language in which 99% of all scientific publications are written in.

Reading knowledge of a language can be gotten fairly quickly.

AdmiralAsshat• 10 years ago

I wouldn't hold it against a Chinese company for not having a native English speaker. But I would hold it against them for not at least having a native speaker on retainer to proofread a document that they're about to send out to an international community. It just looks unprofessional.

And that's all it is, really. Stuff that just screams, "Did no one proofread this thing before sending it out?" Probably not. Because whoever wrote it was probably the most fluent person at the company.

---

FWIW, I have native-English speaking programmer friends who ask me to proofread their stuff. I don't mind. One is an amazing programmer, but his writing ability is somewhere around 9th-grade proficiency. The fact remains that being able to write correctly is considered a "bare minimum" quality in the professional world, and it doesn't matter if you wrote your own virtual machine in Assembly in your spare time--if you sound like a college dropout in your resume, they'll skip over you.

It is similarly difficult to inspire confidence in a CA when their "transparency report" sounds unprofessionally cobbled together.

nikcub• 10 years ago

Interesting aside that came out of this case is the issue of cross-signing intermediary and root certs and it not being disclosed.

The WoSign roots were cross-signed by[0] Comodo and StartCom (owned by WoSign, but we didn't know that), so even with WoSign roots being revoked, there would still be a verification path.

Nice to see that now there is an effort to disclose all of these[2][3], and[1]:

Mozilla now requires the disclosue of all intermedidate certificates, including those cross-certificates.

[0] https://wiki.mozilla.org/CA:WoSign_Issues#Cross_Signing

[1] https://groups.google.com/d/msg/mozilla.dev.security.policy/...

[2] https://crt.sh/mozilla-disclosures

[3] https://secure.comodo.com/products/publiclyDisclosedSubCACer...

terom• 10 years ago

Awesome use of load balancing for request retry across multiple backend servers:

This was caused by the CMS (Certificate Management System), when it sent the signing request of the certificate to the signing server A, which had no response, then the CMS sent it to the other newly added signing server B. After a while the signing server A signed the certificate and sent to the CMS and also to the subscriber, then the subscriber installed the cert in its website and hat's why Censys recorded this certificate; in the meantime, the signing server B also signed this certificate some time later (in seconds) and sent it to the CMS, the CMS accepted it and rewrote it in the DB.
This issue happened after adding another signing server on Jan 5th 2015, and found it on April 9th 2015. When had the two signing servers added a load balancer, but the configuration was not properly done because it didn't lock the request.

Mind you, that's a perfectly legit technical bug. Maybe they were using nginx for load balancing POST requests? :)

https://news.ycombinator.com/item?id=11217477

therein• 10 years ago

Yup, retrying PUT/POST is a bad idea unless you're really careful.

lifeisstillgood• 10 years ago

So I am wondering how this mess got started

Is it just incredible incompetence on WoSign's part or is there a stronger reason why China's largest CA is trying to keep issuing weak certificates? Is is tinfoil-hattery to assume that the Chinese government is not ready for SSL to start getting unreadable again?

im3w1l• 10 years ago

Compatibility with legacy software that only understands the older certs is my guess.

nikic• 10 years ago

The charged 42 backdated certificates are an intentional activity that we try to help the desperate customers since there are more than 3M users still using Windows XP sp2 in China. We like to make things simple that don’t realize how serious this solution was.

I think this is all there really is to it. Some customers wanted to have backdated SHA1 certificates so they could continue supporting platforms that do not accept SHA256 certificates. WoSign decided to oblige them.

koolba2• 10 years ago

When you have a business that effectively prints money, why do something this stupid?

I also wonder how much more effort they thought it was to write the code to backdate the certs (rather than use "now") v.s. code for upgrading to SHA-256.

0x06• 10 years ago

It's not about a lack of support for SHA-256 on the CA side. But backdated SHA-1 certs are in demand because they allow other systems to keep working, systems like embedded/PoS computers that lack support for SHA-256, while simultaneously dodging errors on modern browsers that only allow SHA-1 certs if they were issued before a certain date.

Wosign was certainly capable of issuing SHA-256 certificates. But the customers needed SHA-1 certificates with a backdated issue timestamp. And Wosign was willing to fake the issue timestamp on new certificates, probably a lucrative market because no other reputable CA would be willing to do so.

koolba2• 10 years ago

Wosign was certainly capable of issuing SHA-256 certificates. But the customers needed SHA-1 certificates with a backdated issue timestamp.

So they had a special side deal to get backdated certs? There's no way they were doing this for everybody or for "regular price" right?

nly• 10 years ago

A shame. WoSign were super generous with their free certificate offering long before LetsEncrypt was a thing. They were a a handy alternative.

We should be thanking them for their free certs, and thanking them again now for giving us another example of how the PKI is a farce. The chances are there are a bunch other 'reputable' CAs out there playing these games.

icebraining• 10 years ago

But that's why browsers are increasingly requiring CAs to implement https://en.wikipedia.org/wiki/Certificate_Transparency

bigiain4• 10 years ago

Interestingly - StartCom's free certs used to only have 1yr validity - one I got last week is good for 3 years. Cynical-me suspects they're doing that to improve their chances of sitting out a limited 1yr timeout. It'd be almost indistinguishable from a death penalty if all their existing client certs expired while they were unable to renew them - I wonder how long they've been signing certs for 3yrs instead of 1?

WoSign Incidents Report Update [pdf]

Related Stories