Githubassets.com Cert Has Expired

https://news.ycombinator.com/item?id=24967170

by lysp • 6 years ago

78 53 6 years ago

news.ycombinator.com

Cert expired 5 mins ago.

https://github.com/

roryrjb• 6 years ago

Just came to Hacker News to post this, GitHub's assets don't load and the site's front end is broken.

KriiV2• 6 years ago

This is a wonderful example of what can happen when you don't keep track of your cert expiry dates.

perryizgr82• 6 years ago

That's why they are serving un-styled pages, I guess. I thought my browser was acting up.

KriiV2• 6 years ago

The pages being served are styled, we just can't securely load the stylesheets.

lysp2• 6 years ago

Or javascript.

leoek• 6 years ago

Just ran into the same issue... Most/Everything still seems to work though, because only the assets are missing ^^ However it is not pretty.

melco• 6 years ago

github.githubassets.com expired 1 day ago github.com is OK now

daenney• 6 years ago

That doesn't seem to be the case.

    openssl s_client -connect github.githubassets.com:443
    verify error:num=10:certificate has expired
    notAfter=Nov  2 12:00;00 2020 GMT

That's today about 22min ago from time of writing, not 1 day ago.

Depending on what your browser still has in its cache, github.com will be in various states of degradation.

ta8645• 6 years ago

Maybe browsers should put up a warning when a certificate is about to expire; say two weeks away. Nobody should let their certificate get that close to expiring, but if it does, you'd rather it generate a lot of visible warnings before simply ceasing to work at all.

hannob• 6 years ago

Browsers are very careful in introducing warnings to users that are meant to go to admins. They dislike the indirect "but users will complain to admins" route and I can't blame them - every warning a user will see and not understand leads to warning fatigue.

diggan4• 6 years ago

Or how about the responsible engineers/testers/managers/whatevers setup proper testing infrastructure for the multi-million projects run by their huge international corporate? Feels like it's not a lot to ask for the most basic testing like checking when certs expire, setup automation to renew it and have fail-safes/error reporting in case the renewal fails.

KingOfCoders4• 6 years ago

Tests can have bugs too.

nkrisc• 6 years ago

I don't agree. A cert that is just about to expire is as valid as one that's new. It only has two states: expired or not. For the user is either expired or not, they needn't be concerned until it actually is expired.

tfsh• 6 years ago

Crowd sourcing errors by relying on user generated reports is sometimes a good idea, however in this instance it's not. Cert expiry is something that can be automated and monitored because the expiry is deterministic. Crowd sourcing is for the non-deterministic.

mgkimsal• 6 years ago

perhaps a plugin which admins can add to their browsers to act as a reminder for themselves? having core browsers warn the general public about this all the time will lead to chaos.

raptium• 6 years ago

I thought it was just a local CDN node issue until I see this on Hacker News :-(

lordelph• 6 years ago

There must be some kind of law "Every company will forget to renew a TLS certificate once"

I know I can't throw stones in this particular greenhouse :)

diggan4• 6 years ago

Indeed, together with "deleted production database" it's a rite of passage, which is fine. Problem becomes when same issues pop up over and over again (looking at you Microsoft, re ssl renewals) and nothing is seemingly done to prevent it in the future.

spydum• 6 years ago

Seems like microsoft has a real issue tracking their cert expiry lately

SiliconSolder2• 6 years ago

Not known on https://www.githubstatus.com/ yet.

SiliconSolder2• 6 years ago

Ok, it is now, although they call it an issue with pull requests.

thorin• 6 years ago

Github not working for me either

mg4• 6 years ago

This is a script I call "certdays.sh" which you can call from your regular tests like this:

certdays.sh somedomain.com 14

If the certificate for somedomain.com is valid less then 14 days, it will fail:

https://github.com/no-gravity/certdays

gregoriol• 6 years ago

Don't call it from tests though, call it from a monitoring system!

contravariant• 6 years ago

In case anyone is looking for a good option, I've had great success with InfluxDB's telegraph utility which automatically checks both uptime and certificates for HTTPS endpoints, combined with a grafana dashboard that sends alerts when the deadline approaches.

unilynx2• 6 years ago

Is the '25' in 606025 intentional to give a bit more margin or should it just have been 24?

mg4• 6 years ago

Ha, I guess I made a typo when I wrote the script 6 months ago. At least I cannot remember that I had some clever idea back then, why a day should have 25 hours.

Fixed now. Thanks.

brianhicks• 6 years ago

What shell are you running this under? Unquoted exclamation marks will do things that may be outside your intent for this script. It’ll still fail, just perhaps not in the way you expect.

mg4• 6 years ago

You mean due to history substitution? If so: I have never seen that kick in in a script. Only in interactive shells.

brian_herman• 6 years ago

Thanks awesome script!

chillaranand• 6 years ago

We can also use ssl-cert-check cli tool. `ssl-cert-check -s avilpage.com -p 443 -x 30` will return if the certificate will expire in 30 days.

We can use this command in CI pipeline or setup a cron job to monitor it.

kingsloi2• 6 years ago

Hmm, if I copy/paste this, I get a different output:

    ~  ~/certdays.sh google.com 14
    date: illegal time format
    usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [- 
    v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] 
    [+format]
    google.com: -18568
    -n Lasts longer then 14 days?
    NO!!!!!!!!!!!!!!

mg4• 6 years ago

Works for me. In Bash? Maybe copy/paste gone wrong?

I have put it into a Github repo now and put the link into the post. Does that work for you?

stereo• 6 years ago

Are you running it on macOS or something non-Linux?

floatingatoll• 6 years ago

You can brew install coreutils and edit it to use gdate instead of date.

unilynx2• 6 years ago

if you're using nagios, setup something like

  command_name  check_certificate
  command_line  $USER1$/check_http -H $ARG1$ -p $ARG2$ -4 -S -C 21 -t 20 --sni

this won't warn about certificate name mismatches though

mschuster91• 6 years ago

That warns only after the fact, not in advance... depending on how the infrastructure (and especially the cert procuring process, for those not using LE for whatever reason) is set up, too late.

switch007• 6 years ago

Nagios checks have thresholds for WARNING and CRITICAL. With these kind of checks it's usually 'days until expiry', so it certainly can be used to warn in advance.

gerdesj• 6 years ago

"command_line $USER1$/check_http -H $ARG1$ -p $ARG2$ -4 -S -C 21 -t 20 --sni"

-C 21 means: Check the certificate. Warn if less than 21 days and critical when expired.

lysp2• 6 years ago

Update: Seems to be fixed now.

sz4kerto• 6 years ago

I remember a couple of years ago I was on the top of some old volcano on an island, and I was dictating shell commands to a colleague so that we can quickly fix our expired certificate. Good old days.

user5994461• 6 years ago

Don't see any error. The certificate was renewed after 10 minutes of issue?

Validity Not Before 11/2/2020, 12:00:00 AM (Greenwich Mean Time)

Validity Not After 11/9/2021, 11:59:59 PM (Greenwich Mean Time)

perryizgr82• 6 years ago

They renewed it just now.

sergiotapia• 6 years ago

Certs are such a pain in the ass.

pupdogg• 6 years ago

Once identified, I would expect their "Incident Report" in some way would mention the root cause of the incident. However, that does not seem to be the case here: https://www.githubstatus.com/incidents/4mzhxxpwgvqg

Maybe someone could get fired for this or does it have more to do with Microsoft's stock/public image?

Githubassets.com Cert Has Expired

Related Stories