GCP PubSub & CloudSQL are Down

https://status.cloud.google.com/incident/cloud-pubsub/19001

by nojvek • 7 years ago

4 2 7 years ago

status.cloud.google.com

nojvek• 7 years ago

We are experiencing an issue with Cloud PubSub beginning at 2019-05-20 21:35 US/Pacific. Current data indicates that approximately 1% of publish operations globally (15% publish operations in asia-northeast2) and approximately 20~25% of various types of admin operations (i.e. CreateTopic) are affected by this issue. For everyone who is affected, we apologize for the disruption. We will provide an update by Monday, 2019-05-20 22:45 US/Pacific with current details.
Further investigation indicates that approximately 3% pull and 5% publish operations globally are seeing 5xx and 499 errors. Various admin operations like CreateTopic and DeleteTopic are also seeing >50% errors.

This is a pretty huge failure, would be interesting to see the PostMortem

xerxes901• 7 years ago

Our application is calling GetTopic on a whole bunch of topics at startup to ensure that they were already created, so even though the publish error rate was only 1%, the app couldn't start because it kept failing the GetTopic calls :/

GCP PubSub & CloudSQL are Down

Related Stories