Figma spends $300k on AWS daily

https://www.datacenterdynamics.com/en/news/design-platform-figma-spends-300000-on-aws-daily/

by handfuloflight • 1 year ago

57 54 1 year ago

datacenterdynamics.com

javierluraschi• 1 year ago

Any chance we can get a high-level breakdown of the AWS services they are paying for?

pgwhalen• 1 year ago

No. The figure is from their S-1 filing and further detail is not provided.

jbverschoor• 1 year ago

On what jeeeeeeezus

Neywiny2• 1 year ago

I don't understand how it takes 3 years to get off the cloud. I'm not a cloud developer, though. The most I've done is run code on free hosts or compute instances. Presumably there's something to the microservices and lambdas and distributed compute that makes this hard. I'm thinking if this was a monolith (like AWS themselves admit is cheaper), they could just run it locally? What a giant waste of money. I'm very glad to start seeing xAAS start to die out. At the end of the day it's just looking like more middle-men instead of how I've always assumed it was intended to be: economies of scale.

However, and missing from this article + discussion so far, is their revenue. If they pay $4/day and make $2 in revenue, that's bad. They pay $300k/day but make ~ $2250k/day in revenue. I don't know what the ratio is supposed to be, but at first blush that doesn't actually seem too bad. I'll let the more qualified take over, I'm struggling to find out how big a % of their total expenses this is.

haiku2077• 1 year ago

The services are the "easy" part; moving data out of a cloud provider is slow and expensive. For a _really_ big dataset it can take months, sometimes years, just to complete the data transfer.

nevon• 1 year ago

A mistake I see commonly whenever someone says to "just move off of the cloud" is that they see the cloud as just a VM provider. If it was, then yeah, moving to another provider wouldn't be such a big deal.

In reality, the cloud creeps into your systems in all sorts of ways. Your permissions use cloud identities, your firewalls are based on security group referencing, your cross-region connectivity relies on cloud networking products, you're managing secrets and secret rotation using cloud secrets management, your observability is based on cloud metrics, your partners have whitelisted static ip ranges that belong to the cloud provider, your database upgrades are automated by the cloud provider, your VM images are built specifically for your cloud provider, your auditing is based on the cloud provider's logs, half the items in your security compliance audit reference things that are solved by your cloud provider, your applications are running on a container scheduler managed by your cloud provider, your serverless systems are strongly coupled distributed monoliths dependent on events on cloud specific event buses, your disaster recovery plans depend on your cloud provider's backup or region failover capabilities, etc. Not to mention that when you have several hundred systems, you're not going to be moving them all at the same time. They still need to be able to communicate during the transition period (extra fun when your service-to-service authentication is dependent on your cloud) without any downtime.

It's not just a matter of dropping a server binary onto a VM from a different provider. If I think about how long it would take my org to move fully off of _a_ cloud (just to a different cloud with somewhat similar capabilities), 3 years doesn't sound unrealistic.

Nextgrid• 1 year ago

Which is exactly why you don't make the mistake of relying on all that.

If you can't run it locally, don't use it unless you have absolutely no choice.

motorest• 1 year ago

A mistake I see commonly whenever someone says to "just move off of the cloud" is that they see the cloud as just a VM provider. If it was, then yeah, moving to another provider wouldn't be such a big deal.

I think it can still be a big deal depending on what's the overall system architecture, where are all data stores, how many services you run, and what constraints you're dealing with.

For example, when you are between two cloud providers, more often than not you will have to replace internal calls with external calls, at least within the migration stage. That as important impacts on reliability and performance. In some scenarios, this performance impact is not acceptable and might require architecting services.

quantum_state• 1 year ago

Cloud is a drug for companies… once get on and become addictive addicted, getting off is almost impossible.

mathiaspoint• 1 year ago

This is why I have a hard rule of just doing everything in the VM for stuff I build. I'm able to move between cloud providers and even self host often with near zero effort because of this.

fifilura2• 1 year ago

I've always assumed it was intended to be: economies of scale

IMO, the value proposition these days is rather to avoid maintenance. I.e. help with up with all the latest patches on your infrastructure.

Neywiny2• 1 year ago

To me it's the same thing. You can pay somebody to care about that, but they might be underutilized for the majority of time so it's not worth it. If you have a service, instead of your security expert being used idk 1/x of full time, they can be y/x where y is the number of contracts. For me and my time we are just way too small to have somebody full-time dedicated. So that's how I think about it

tonyedgecombe• 1 year ago

I'd love to see some real figures on that. My gut feeling is most companies spend as much on AWS experts as they previously did on people running in house facilities but I really don't know.

vidarh11• 1 year ago

I have never had a client that got away with less maintenance because they used cloud.

In fact, those of my clients who insist of relying on cloud, tend to spend far more with me for similar complexity systems. I love taking their money, but I'd frankly rather help them save it, because longer term it's better.

undebuggable• 1 year ago

I don't understand how it takes 3 years to get off the cloud.

Because in fact cloud is not just someone else's computer.

icedchai4• 1 year ago

When you code your entire app to proprietary APIs like Lambda or DynamoDB, it becomes complex to migrate.

vidarh11• 1 year ago

People dig themselves in hard, with reliances on all kinds of proprietary services, and complex relationships.

My experience in helping people do cloud migrations are that companies also often quickly lose oversight over which cloud services they are even still running. Sometimes systems that should've been shut down years ago are still hanging around, or parts of them anyway, like S3 buckets etc. Most companies that use cloud systems underprovision their devops because they think they don't need much for a cloud system (in fact, they typically need more to do it well).

$300k/day for their revenue is very much crazy high.

icedchai4• 1 year ago

During good times, many companies simply don't care about which services or how much. I've worked at several startups where they got a large funding round and the word was "we don't care about cloud costs, just get it done fast / make it scale." Unfortunately, one bad mistake (like storing all data in a proprietary service like DynamoDB) can make this difficult to unwind when things get bad...

PunchyHamster3• 1 year ago

If you use cloud brand queue system, you not only need to run some servers to run the queue, you also need to rewrite all parts of the system that relied on cloud brand queue system into something else.

Repeat it by a bunch of cloud services your app use and you're in for major rewrite.

Cloud offers many, many "good enough" solutions for common problems in development and it is oh so easy to use another cloud api for few cents more

And that's not even touching a bunch of auxiliary services you now need to provide like monitoring, backups, metrics etc.

Usually best strategy is to start move from most expensive parts but if that part is entangled in cloud services it could take forever

andrewstuart• 1 year ago

Utterly bananas.

Seriously the CTO should be fired.

lazystar2• 1 year ago

edit - i read it wrong, removing my comment.

kgc• 1 year ago

Two different companies. Figma is not 37Signals.

lazystar2• 1 year ago

oh, woops. thanks for correcting me, will edit my comment.

samcat116• 1 year ago

Am I the only one that finds that not that crazy?

vidarh11• 1 year ago

It's insane spends like that which are the main reason that reducing cloud costs is the main money maker for my consulting business at the moment.

It's extremely rare to come across people using cloud services where you can't cut their costs by at least 30%, often more than half, sometimes as much as 90%+, and usually it results in reduced devops spends as well.

The interesting thing is that the hardest part of the sales process is to convince people that cloud isn't cheap, because there's a near religious belief that cloud providers must be cost effective in some companies.

handfuloflight• 1 year ago

What percentage of clients do you think don't even need cloud?

vidarh11• 1 year ago

For the vast majority of them, rented servers at a managed hosting provider would beat cloud in price hands down, but for some the difference might be small enough once they've actually optimised their cloud setup that it's a judgement call. I usually recommend clients plan for optimizing their cloud setup first, both to get a fair comparison and also because a lot of the assessments you need to do of your cloud environment to do that will also make it easier to figure out what you'd pay for a managed hosting setup as well (e.g. understanding base load vs. spikes, durability requirements for different subsets of storage etc.)

For some of them, a colo facility would be cheaper, but that's highly dependent on where you want to host it (e.g. I'm in London - putting things in a colo in London is really hard to make cost effective vs. renting servers somewhere with lower land costs; data centre operators are real-estate plays)

However, you can usually make managed hosting/colo even cheaper by sprinkling some cloud in. E.g. a "trick" that can work amazingly well is to set up the bare minimum to let you spin up what you need to handle traffic spikes in a cloud environment, and then set up monitoring for your load balancer so that you start scaling into the cloud environment once load hits a certain level, but use only the managed hosting below that level.

That way, you almost never end up actually spinning up cloud instances, but you gain the ability to run the managed hosting environment far closer to the wire than you could otherwise safely do, and drive your cost per request down accordingly.

gausswho2• 1 year ago

We're discussing Figma's spend here which is a bit of a cherry pick. At their size the savings for on-prem can be substantial. For the vast majority of companies without such heavy bandwidth needs, the additional cost needs to be measured in terms of additional intenance and staffing.

vidarh11• 1 year ago

Savings can be substantial at much smaller sizes and generally devops maintenance/staffing costs goes down if anything.

This is literally my bread and butter, and we make more money off customers who insist on staying on cloud providers, because they consistently need more help.

icedchai4• 1 year ago

Is this production only? Based on my personal experience, many orgs could save a decent chunk just be cleaning up or spinning down dev environments.

vidarh11• 1 year ago

We'd typically look at everything, and you're absolutely right. Dev environments are often also poorly managed and you can find lots of resources nobody knows who are using.

samgranieri• 1 year ago

This 100%. SRE at $dayjob periodically has to ask us to clean up dev and uat kubernetes deployments

mystified5016• 1 year ago

At the startup I work at, we have an AWS instance that only runs our GitLab server. The damn thing runs at 10-15% CPU nearly constantly (because GitLab's founding assumption is that everyone wants to scale to ten million global users) so our spend racks up to $70/month every months.

And yes, that's absolute peanuts for any business. But in my view, spending almost $1k annually for a GitLab server for a team of ten is ridiculous.

We could accomplish exactly the same thing on-prem with hardware we already own. It would take a couple of engineer-hours per year. As long as it's under 40 hours of maintenance per year, we come out day ahead. And over the last two years, I've had to spend a total of maybe 10 hours with hands on this box.

I just don't get the thinking that leads businesses to put unnecessary crap in the cloud. Just save your money, on-prem is cheaper for pretty much everyone not doing a saas or less than a few hundred employees

mdaniel• 1 year ago

My mental model of startups is that they're free of the bureaucracy that would cause you to whine in comments versus Just Fucking Do It. So, what's the impediment to using your time to test your "10 hours" theory?

Also, my experience with GitLab isn't that the thing is hurp-durping because of scaling to ten million, it's that ruby gonna ruby

vidarh11• 1 year ago

I often starts that way, yes. And then you end up accreting more and more.

I usually tell people that if they don't want something on-prem, then at least pick a cheaper provider.

But as long as it's a VM and you don't build in reliance on other AWS services, at least you can move off it as you scale. The real problem comes when you start depending on a single cloud.

Not only do you lose the ability to do an easy move, but you also lose almost all negotiating leverage as your bill increases.

quickthrowman• 1 year ago

Your estimate is off by an order of magnitude unless you value developer time at $21 an hour. I pay union electricians 6x more than that to work for me.

$70 a month times 12 is $840
As long as it’s under 40 hours of maintenance (??)

PunchyHamster3• 1 year ago

I've seen so many people being baited by whole "you will not only not need to buy expensive servers, but also not have expensive ops guys to manage it!" schtick.

Meanwhile your devs put more man-hours than your actual hardware ops teams would ever do into wrangling cloud quirks and spend way more than getting a bunch of servers (even if you include costs of financing them in that) would ever cost.

wg0• 1 year ago

They need cloud storage. Object store. For design files.

Some web sockets with CRDT processes running.

What else?

fulafel• 1 year ago

Bandwidth and networking stuff is famously expensive there

puff_pastry• 1 year ago

This is absolutely bonkers, $110mln/year?? How’re they spending this much??

yogi4D• 1 year ago

It’s wild how someone can invent an idea, force it on everyone something no one asked for then turn around and sell it to us, layering on FOMO to keep us hooked.

Emotional manipulation at its finest.

No wonder nothing feels special anymore. Everything’s automated, abstracted, owned by someone else. If you make things by hand, you have to prove their value meanwhile, platforms like AWS are printing money.

They want us to own nothing. And if “heaven” is real, why aren’t they there already?

PunchyHamster3• 1 year ago

Sir, this is Wendy's

mattbillenstein• 1 year ago

This is basically technical debt of some sort - they may be minting so much money it doesn't matter atm, but eventually, someone will get some wins cutting costs. Probably just the bandwidth is a lot of it.

rkj93• 1 year ago

I see lot of comments questioning why can't they move to another provider. When you start using any of the cloud providers there so many services you start using and then you start building integrations, email notifications, api gateways, and what not. This makes it very difficult to move all the established processes.

vilanye• 1 year ago

This is why I don't use AWS or any "cloud" provider. That is not 100% true. I have 110 GB or so of personal data backed up on Glacier. The bill is 6 cents a month, and they obviously don't charge my card.

It may take longer to start, but owning all your servers is cheaper in the long run.

At the very least, don't lock yourself into one provider. Spread out any services you use.

More than one company has screwed itself over permanently by tying itself to a single service. At the end of the day, it is the extremely poor decision to follow the trend of locking oneself into a single company that dooms it.

You would think people would have learned the lesson to avoid lock-in by now. It is absolutely Darwinian at this point.

Figma spends $300k on AWS daily

Related Stories