Thunder from the Cloud: 40k Cores Running in Concert on AWS

https://medium.com/descarteslabs-team/thunder-from-the-cloud-40-000-cores-running-in-concert-on-aws-bf1610679978

by lhussey • 7 years ago

20 8 7 years ago

valleyjo• 7 years ago

This is pretty awesome. Does anyone know if google or azure can scale to 40,000 cores on demand like that?

I work at DL (although not technical team) and can attest that this is attainable for GCP, and we do so on a fairly regular basis. I cannot speak to Azure, but I expect it too is likely.

robbyt• 7 years ago

I'm building a 160,000 core cluster on GCP right now. It took a week to get the quota increase approved, but GCP can definitely handle it.

spew• 7 years ago

Would love to see another post where this is run on Google and Azure as well and the results are compared.

hobolord2• 7 years ago

what's the cost on something like this?

ahdiamon2• 7 years ago

It cost less than $5000 to run on AWS. More if you include engineering's time.

hobolord2• 7 years ago

per month? pretty amazing

Thunder from the Cloud: 40k Cores Running in Concert on AWS

Related Stories