Parts of AWS are down

https://news.ycombinator.com/item?id=21327459

by Bob312371 • 7 years ago

33 19 7 years ago

news.ycombinator.com

RDS & S3 is down for me right now in US-East region and apparently some parts of Frankfurt are also down.

There own favicon is timing out: https://awsmedia.s3.amazonaws.com/favicon.ico

wldlyinaccurate• 7 years ago

From https://status.aws.amazon.com/

Intermittent DNS Resolution Errors

We are investigating reports of occasional DNS resolution errors with Route 53 and our external DNS providers. We are actively working towards resolution.

joshuapinter3• 7 years ago

Finally acknowledging the issue.

faCeti0us• 7 years ago

Intermittent DNS Resolution Errors We are investigating reports of occasional DNS resolution errors with Route 53 and our external DNS providers. We are actively working towards resolution.

https://status.aws.amazon.com/

Bob3123712• 7 years ago

Why does the status page say resolved when S3 is still down for parts of US?

DevX101• 7 years ago

Having issues here in US East as well.

joshuapinter3• 7 years ago

Having DNS issues with s3-us-west-2 as well.

joshuapinter3• 7 years ago

We manually updated our `/etc/hosts` file with the static ip for our S3 bucket(s) and that fixed the issue temporarily.

nakodari2• 7 years ago

Glad to hear the issue is on Amazon's side. Our team panicked and tried to find every way to resolve this thinking it was possibly a misconfiguration at our end. I hope it's resolved soon. We're receiving complains from many users who are unable to upload their work due to this issue, I will let them know Amazon is on it.

Bob3123712• 7 years ago

Same here. At the time the status check on AWS was not reporting any issues and there was nothing written about it online. As a result I assumed it was an issue with my account.

I only confirmed it was on their end when I noticed the favicon on the AWS console wasn't loading.

ej12n• 7 years ago

Same here, I was running around like a headless chicken and wondering what I was doing wrong when elasticbeanstalk just wouldn't deploy anything. The fact that we doubt ourselves rather than thinking aws is down is a good damn indicator how reliable they are, luckily it happened during daytime (EST anyways)

champbronc22• 7 years ago

Seeing the same thing

lflux2• 7 years ago

DDos against amazon's DNS servers. One mitigation is to use regional endpoints for bucket hosting if your website relies on serving assets - i e foo-bucket.s3.us-west-2.amazonaws.com instead foo-bucket.s3.amazonaws.com

champbronc22• 7 years ago

Where you see this is DDoS?

smudgymcscmudge• 7 years ago

I'm not OP, but I heard that unofficially from somebody who works at aws.

jared52• 7 years ago

I just got off a chat with an AWS agent:

"We are investigating reports of occasional DNS resolution errors. The AWS DNS servers are currently under a DDoS attack. Our DDoS mitigations are absorbing the vast majority of this traffic, but these mitigations are also flagging some legitimate customer queries at this time. We are actively working on additional mitigations, as well as tracking down the source of the attack to shut it down. Amazon S3 customers experiencing impact from this event can update the configuration of their clients accessing S3 to specify the specific region that their bucket is in when making requests to mitigate impact. For example, instead of "mybucket.s3.amazonaws.com" a customer would instead specify "mybucket.s3.us-west-2.amazonaws.com" for their bucket in the US-WEST-2 region. If you are using the AWS SDK, you can specify the region as part of the configuration of the Amazon S3 client to make sure your requests use this region-specific endpoint name."

lflux2• 7 years ago

From the phonecall with our AWS TAM and support reps

raghava• 7 years ago

status check: Tue Oct 22 22:50:22 UTC 2019

DNS resolution to AWS S3 and RDS is still buggy on other providers (DigitalOcean, etc as well)

https://status.digitalocean.com/incidents/1z3kmlvz69v6

https://developercommunity.visualstudio.com/content/problem/...

MastaX1• 7 years ago

Seeing major issues here as well. All my modules were not working for the majority of the day and all images were not displaying. This is no doubt a Ddoss attack, I just don't see any other reason that the entire east coast AWS DNS servers would just stop responding all of a sudden. If the issue is not fixed by tomorrow our IT guy is gonna have a damn field day implementing all the temporary mitigation's.

Parts of AWS are down

Related Stories