HN user

lomkju

114 karma

https://excloud.in

https://www.linkedin.com/in/arjun-hemrajani-416581ab/

Posts6
Comments25
View on HN
[dead] 4 months ago

Just In RAM prices have crashed got a weekly price update from a vendor for DDR5 64GB RDIMM 10X cheaper than last week!!

Causes TurboQuant and AI Market leader backing out from their RAM purchasing commitment!!

Having being a high scale AWS user with a bill of +$1M/month and now working since 2 years with a company which uses GCP. I would say AWS is superior and way ahead.

** NOTE: If you're a low scale company this won't matter to you **

1. GKE

When you cross a certain scale certain GKE components won't scale with you and SLOs on those components are crazy, it takes 15+ mins for us to update a GKE ingress controller backed Ingress.

Cloud Logging hasn't been able to keep up with our scale, disabled since 2 years now. This last Q we got an email from them to enable it and try it again on our clusters, still have to confirm these claims as our scale is more higher now.

Konnectivity agent release was really bad for us, it affected some components internally, total dev time we lost was more than 3 months debugging this issue. They had to disable konnectivity agent on our clusters, I had to collect TCP dumps and other evidences just to prove nothing was wrong on our end, fight with our TAM to get a meeting with the product team. After 4 months they agreed and reverted our clusters to SSH tunnels. Initially GCP support said they said they can't do this. Next Q Ill be updating the clusters hopefully they have fixed this by then.

2. Support.

I think AWS support always were more pro active in debugging with us, GCP support agents most of the times lack the expertise or proactiveness to debug/solve things in simple cases. We pay for enterprise support and don't see getting much from them. At AWS we had reviews of the infra how we could better it every 2 Qs and we got new suggestion and was also the time when we shared what we would like to see in their roadmap.

3.Enterprisyness is missing with design

A simple thing as cloudbuild doesn't have access to static IPs. We have to maintain a forward proxy just cause of this.

L4 LBs were a mess you could only use specified ports in a (L4 LB) TCP proxy, For a tcp proxy based loadbalancer, the allowed set of ports are - [25, 43, 110, 143, 195, 443, 465, 587, 700, 993, 995, 1883, 3389, 5222, 5432, 5671, 5672, 5900, 5901, 6379, 8085, 8099, 9092, 9200, and 9300]. Today I see they have removed these restrictions. I don't know who came up with this idea to allow only a few ports on a L4 LB. I think such design decisions make it less Enterprisy.

They so need to apply this to Google Maps, in India google maps has gone crazy since past 2 years especially in Mumbai. It seems to give all bad routes always and also all gig workers in Uber, Ola, Swiggy, Zomato are affected by this too.

You can't test everything in production.

Be it an infra change or application. It should always be like:

Staging -> Canary -> Production

If you invest time in making the above pipeline automated it will cause less outages.

P.S we use feature flags but it won't solve many other challenges around infra and application changes.

Some examples: - Changing from Classic ELBs to ALBs (We care about latencies so ALBs served 1% traffic in the start) - Zero downtime single mysql master upgrade. (ProxySql) - New k8s controllers. - New metrics adapters.

Staging should be like a playground to test all edge cases before you go to canary and then to prod.

I think we can do the same things using github actions now. P.S we have self-hosted runners in our enviroment where each step is a docker container too.

We have complex flows where we use Slack, Spinnaker(Webhooks), Terraform, PagerDuty ... and much more.

Is there something better that we can achieve with this?

Can you tell why should I choose fly instead of AWS?

micro-2x shared 512MB $0.000003044 $8 VS t3a.nano 2 Variable 0.5 GiB EBS Only $0.0031 per Hour

I'm missing something? cause seeing the pricing I still feel AWS is cheaper.