GitHub was down

https://www.githubstatus.com/

by bombcar • 3 years ago

117 40 3 years ago

The status page shows stuff about slack https://www.githubstatus.com but it seems authentication is down hard, can't pull, comment, or even view public repositories when logged in.

Edit: https://www.githubstatus.com is updated as of 19:08 UTC and everything is on fire

athorax• 3 years ago

Getting 404 on all github.com repos, oh boy lol

bombcar4• 3 years ago

I got a glimpse of some code on a reload of notifications, and then - poof.

vb-84482• 3 years ago

+1 same here

thiag0• 3 years ago

same here!

adtac• 3 years ago

We, uh, fixed the glitch^W^W all bugs

contingencies• 3 years ago

Yep, can't push to repo. Can't view repo. Thanks guys.

richwater• 3 years ago

chill dude, shit happens

hobofan• 3 years ago

Yup, I'm also seeing very sporadic issues with 404 pages and some API actions not being able to complete, though it seems mostly related to private repositories for me.

uncomputation• 3 years ago

Seems intermittent. Sometimes a refresh will work to display files and commit history.

ezekg2• 3 years ago

Implementing that type of multi-account support likely had some big authz changes (I've done a couple of those projects before), and GitHub defaults to a 404 for authz errors instead of leaking information with a 403. Just a wild guess.

jrochkind1• 3 years ago

The fact that https://github.com/ (which requires no specific authz check) seems to be up, and that some users report logging out and in again at least temporarily restores access, seems to support that hypothesis.

Still seems odd that no CI/QA would have caught such a widespread bug.

grepfru_it• 3 years ago

Oh if you only knew the code base they are up against. Or in other words “what QA <laughing.emoji>”

netcraft• 3 years ago

spot on:

Update - We’re in the process of rolling back an authorization-related change that is causing 404s and other errors.

ezekg2• 3 years ago

Like I said, not my first rodeo for implementing a multi-account system on top of a single-account system lol.

hirsin• 3 years ago

No, it was an authorization internals upgrade we were testing, not the account switcher.

d3mcfadden• 3 years ago

nailed it man

ta12433• 3 years ago

Nobody would deploy such a change on a Friday afternoon though

xeonmc• 3 years ago

#famouslastwords

alfanick• 3 years ago

It kinda makes sense though to do it on Friday afternoon, as on the weekend enterprises and most people just go skiing or do other things so less of customer base is affected in case of (unfortunate) outage.

paulddraper• 3 years ago

What day is the best day to have downtime?

strongly-typed• 3 years ago

Yep, seems to be down for me too.

h1fra• 3 years ago

Yep, only 404 or 500.

They were experiencing issues with Slack at the beggining of the day.

If I had to guess, a fix had unexpected consequences or it's related to the release of multi-account login https://github.blog/changelog/2023-11-03-multi-account-suppo...

mparnisari2• 3 years ago

This is nature's way of telling us to stop working on Friday, y'all.

casenmgreen• 3 years ago

I run my own git server and users clone from that.

smug ;-)

rlizzo• 3 years ago

first cloudflare... now github... goodbye internet

seneca• 3 years ago

Is there a major service that has a worse uptime than Github? I can't think of one that feels less reliable.

joeywas• 3 years ago

Microsoft 365

bombcar4• 3 years ago

Logging out and back in seems to .... somewhat make it work? So maybe that could help.

bombcar4• 3 years ago

Nope, still busted.

vb-84482• 3 years ago

I've tried multiple times + incognito while trying to access a public repo and still randomly get 404.

I bet there is something wrong with caches.

layer8• 3 years ago

Good thing that Git is decentralized.

derwiki• 3 years ago

I mean yea, I'm still happy that I can commit locally. With a centralized VCS like Subversion, I wouldn't even be able to do that.

lakomen• 3 years ago

I left that platform when M$ took over. Only have an account because of the many stubborn people still clinging to it and I have to tell them when I find a bug.

Especially the forced 2FA.

Whatever will you people do now, that you can't use co-pilot to code for you? It's the end of your career I'm sure. I'm sorry but I can't help being cynical on HN especially.

mparnisari2• 3 years ago

Especially the forced 2FA.

How dare they protect their users?

caeruleus• 3 years ago

Notice that there were issues with Slack notifications only before:

The delayed Slack notifications should be fully processed in about 30 minutes (Nov 03, 2023 - 18:25 UTC)

Exactly 30 min after:

(Git Operations|Issues|Pull Requests|Actions|API Requests|Codespaces|Packages|Actions|Pages) is experiencing degraded performance. We are continuing to investigate. (Nov 03, 2023 - 18:55 UTC)

Might be coincidence, might not be...

bombcar4• 3 years ago

seems to be back now

ChrisArchitect• 3 years ago

Tell HN:

[dupe]

Status discussion over here: https://news.ycombinator.com/item?id=38133297

AndrewOMartin• 3 years ago

What we need is something like github but decentralised somehow, if such a thing is even conceivable.

GitHub was down

Related Stories