Tell HN: Google banned Railway's account. Everything down

https://news.ycombinator.com/item?id=48201330

by sergiotapia • 2 months ago

33 18 2 months ago

news.ycombinator.com

Everything down including railway.com - this is a bad outage. :(

gnabgib• 2 months ago

Some discussion started an hour ago (12 points, 3 comments) https://news.ycombinator.com/item?id=48200827

mcontrerazCL• 2 months ago

Identified Google Cloud has blocked our account, making some Railway services unavailable. We have escalated this directly with Google. The Railway Platform team has since confirmed access to Google Cloud and is working on restoring access to all workloads. We have access to some of our Google Cloud–hosted infrastructure and are working to restore the rest of the service. We apologize for the disruption.

ErystelaThevale• 2 months ago

Railway had a similar reliability issue two weeks ago when an AI agent deleted a customer's production database via their API — no confirmation step, no environment scoping. Now this. Both incidents suggest the same pattern: infrastructure decisions made without thinking through failure modes, fixed reactively after damage is done.

choilive• 2 months ago

I wouldn't blame that incident on Railway.. you can delete your prod database on AWS just as easily with their API.

sikozu• 2 months ago

That incident wasn't Railways fault at all. Don't use AI in your staging and prod tools.

jesterson4• 2 months ago

If everyone can delete your prod database via API or by any other means - you need to sack CTO without severance package.

lotsoweiners• 2 months ago

So the solution is the same as it has been for over a decade. Don’t do business with Google.

scorpioxy• 2 months ago

I believe the problem was always the process that Google has where they tend to want to automate everything from the start and make it very difficult to reach a human and explain the situation. The service itself seems solid but if you make it difficult to address any problems when(not if) they occur then I won't be comfortable doing business with you. I stay away from their services for anything serious for this reason and always recommend to others to do the same.

It's going to be interesting to watch this unfold. Google's automation vs LLM agents, no humans from either side.

jesterson4• 2 months ago

Don’t do business with Google.

If you think other big tech or bank won't do it for you if you "don't do anything wrong" - you might be delusional.

The only way to fight it is redundancy. Dont vendor lock in all you stuff in aws/google/cf/whatever. They can (and will) fuck you up at some point.

happymellon2• 2 months ago

It is a weekly occurrence for large named organisations and Google.

It isn't for other big tech/banks. It's just a risk assessment, and Google are a high risk partner.

Leena-ch• 2 months ago

The fix isn't don't use GCP. It's never letting one cloud account suspension take down your control plane and customer workloads simultaneously

moezd• 2 months ago

Exactly. Whatever happened to cloud just being "someone else's computer" and "cheap scaling on speed dial"?

Shanov232• 2 months ago

We'll rapidly deploy a reliable backup website with proper hosting to get Railway back online within 48 hours

Shanov232• 2 months ago

Reply at hello@fourfivestudio.in

Tell HN: Google banned Railway's account. Everything down

Related Stories