How Ahrefs Gets a Billion Dollar-Worth Infrastructure with a 90% Discount

https://tech.ahrefs.com/how-ahrefs-gets-a-billion-dollar-worth-infrastructure-with-a-90-discount-5edd473b2399

by botsbreeder • 2 years ago

65 46 2 years ago

tech.ahrefs.com

CoolCold7• 2 years ago

Nice writing!

My gut feeling says part of the problem is C-levels, like CTOs, in recent time, coming solely from programming experience and its just mentally hard for them to even consider having physical hardware (yet on colocation!) and managing such systems. It's just out of equation.

On the article itself, billions scale payments are astonishing. And how much paying for traffic would add here too.

RicoElectrico• 2 years ago

Arguing with programmers is oftentimes pointless these days. Pragmatism is out of the equation.

I can see why, though. First, there was so much money flowing into IT since 20-ish years that cost was not a consideration both on infra and salary side. Second, connecting all the piping gives an illusion of doing work and feeling "important". You know, we need to be scalable because Google does, and they're successful, right?

CoolCold7• 2 years ago

Arguing with programmers is oftentimes pointless these days. Pragmatism is out of the equation.

I partially agree with you, but my point is - things are even worse at times - arguing happens when there is more than single person involved - CTO may just not even considering/thinking on using hardware, it's a very strong habit to relay on cloudy stuff, atrophy of some sort.

Second, connecting all the piping gives an illusion of doing work and feeling "important".

This happens as well, but I think it's more about "keeping things under control" - humans feel more comfortable when they control or tend to believe they control things, all humans, not just developers. From this perspective "gee, this Cloud has API and Console, I can manage it" gives this assurance -> leads to have feeling of control => preferable solution.

RamblingCTO5• 2 years ago

Running your own data center is just a PITA no one wants to do. Non-C-levels just love to ignore the costs you have when doing such a thing by just comparing hardware costs to the cloud costs. There's always the liability aspect, managing less people, externalizing some complexity to another company etc. etc. All worth a lot of money, at least for me personally. I wanna focus on software, not also having to manage hardware lifecycles, fire stuff, more engineers etc. etc. You deal with enough complexity nowadays already.

I'd say that if you really want to have hardware on your own, you get the right people to do the job for you. CTOs are not managing that on the floor. So I'm not sure it's the real reason.

CoolCold7• 2 years ago

Running your own data center is just a PITA

I believe in the article they clearly say they just use colo, not running own DCs

I wanna focus on software, not also having to manage hardware lifecycles, fire stuff, more engineers etc. etc. You deal with enough complexity nowadays already.

This looks strange to me, to focus on software but not on company goals, one of them quite often is to increase profits/decrease spending. Focus on software may happen on lower level of responsibility from my PoV.

No offense, but for me it sounds like someone having talent for bakery and cooking, decided to open franchise of McDonald's, becoming General Manager of that shop and then suddenly says - I wanna focus on burgers only, not hire/fire workers.

CoolCold7• 2 years ago

I'd say that if you really want to have hardware on your own, you get the right people to do the job for you. CTOs are not managing that on the floor. So I'm not sure it's the real reason.

Not sure I got this point right - if you need to create you own product you will need to hire programmers as well and get right people. Same for PMs, Security, Infra guys as I see it.

mistrial9• 2 years ago

this appears to be survivor bias from two decades of fast money and those that handled it

SkyPuncher2• 2 years ago

In my eyes, physical hardware is something non-growth companies do to improve margins.

When you’re a growth stage company, the flexibility and leanness that comes from the cloud is important. Once your core product and computing demands stabilize, it’s a good time to figure out what makes sense to move to cheaper compute.

nojvek• 2 years ago

Exactly. Dropbox was cloud and then later moved to on-prem to save on storage costs.

Anyone running >1000 servers with >80% utilization should consider going partially colocated.

Comma.ai and Tesla come to mind who run their own training clusters.

If you are serious about compute and cost of compute and storage is ~100s of engineers, then moving off cloud is a wise strategy.

Cloud providers have huge margins. One can run a very profitable business if they get a pie of that margin.

barryrandall• 2 years ago

It's investors and the one-size-fits-all metrics many rely on. They want a company's financial statements to look like the financial statements from other financial instruments (companies) they're familiar with. If your industry has 30% EBITDA, your company needs to have ~30% EBITDA. If your industry is 100% cloud, your company is expected to be ~100% cloud. If your numbers deviate from expectations, they'll demand an explanation. If that explanation isn't your company's secret to success, then it's probably not going to be accepted.

EcommerceFlow• 2 years ago

Fun random fact, Ahrefs is STILL denied a Wikipedia article, even though their competitors like SEMRUSH have one. Wouldn't be surprised if bad actors were involved. I mean hell, I've heard their crawling spider was the 2nd largest on the web after Google.

Super interesting article though, and a nice addition to their prior one.

EE84M3i• 2 years ago

https://en.m.wikipedia.org/wiki/Wikipedia:Articles_for_delet... seems to be the most recent deletion discussion, from 2020

mirochnik4• 2 years ago

First they ignore you...

RamblingCTO5• 2 years ago

This conversation is cringy. Ahrefs is a great tool.

mannyv3• 2 years ago

I think these articles are great.

It's about understanding what you do and what infra is appropriate.

I have to wonder how that would look with a negotiated AWS discount. But they might get beaten to death by incoming bandwidth costs anyway.

CoolCold7• 2 years ago

from AWS/GCP/Azure/you_name_it's perspective, it may be worth to give that 90% discount extra, just to ensure

1. keep shrinking workforce being able to keep doing such kind of setups

2. avoid such writeups with _numbers_ at all

mirochnik4• 2 years ago

Netflix is a flagship example of AWS usage. https://aws.amazon.com/solutions/case-studies/innovators/net... We don't know what are Netflix discounts. Maybe Netflix uses AWS just for free to make others pay. Maybe AWS even pay to Netflix to continue attracting the other guys.

On the other hand, just read this disclosure in the Netflix 2023 annual financial report: "We have architected our software and computer systems so as to utilize data processing, storage capabilities and other services provided by AWS... Given this, along with the fact that we cannot easily switch our AWS operations to another cloud provider, any disruption of or interference with our use of AWS would impact our operations and our business would be adversely impacted. While the retail side of Amazon competes with us, we do not believe that Amazon will use the AWS operation in such a manner as to gain competitive advantage against our service, although if it were to do so it could harm our business."

fragmede• 2 years ago

We don't know what are Netflix discounts

What you do, if you're a VC backed SaaS company with a six or seven figure monthly spend on AWS, is hire away someone from Netflix who does know what their discount is and use that when your next contract negotiation is up with AWS.

mannyv3• 2 years ago

What we've seen with AWS (and we only discounted like 2 services) is that if you commit to a monthly spend amount on a service you can get substantial discounts on that service.

There are EC2 plans as well, which I assume means that you can get discounts off the reserved instance pricing - but you'd have to ask your AWS rep about that.

If you spend > 4 figures on an AWS service you should ask for their discount plan. Nobody wants to say what they get discounted on, because they might lose them (and the contracts are confidential). So you have to ask your rep.

mannyv3• 2 years ago

The reason I ask is that we (who are super small) got a massive discount on cloudfront bandwidth by just committing to a $2k/month spend - a discount that was more than 90% off of list.

For a spend their size they should be able to get a much bigger discount.

It would probably behoove them to ask AWS about it, and to report back.

At some point AWS just wants their capacity to be utilized. Unused CPU and memory are like empty seats on the plane: you might as well fill them up because you're losing money on those empty seats when you take off.

krade• 2 years ago

I don't doubt that physical hardware would be cheaper than AWS at their scale, but their graph seems off: They are showing no difference between on demand and reserved pricing, with reserved pricing often even being more expensive. Are they actually taking the full 3-year up-front cost and comparing it to monthly on-demand pricing?

mirochnik4• 2 years ago

They are showing no difference between on demand and reserved pricing, with reserved pricing often even being more expensive.

That's exactly what is happening when you look at things without depreciation. You pay 3y all upfront. And it's a lot of money: a high step on the graph. On demand will grow steadily and in three years, it will surely be higher than 3y reserved instance. But we are not comparing one instance. We add more ever more powerful servers as they are added in any infrastructure. And we pay upfront. On demand price can't catch up to those all upfront costs on our period of time.

To see it clearly, we fix the infra on the 4th graph. And then on-demand becomes much more expensive. Also, you may look at the last paragraph in the Appendix to see why 3y all upfront is not horizontal between the steps.

Are they actually taking the full 3-year up-front cost and comparing it to monthly on-demand pricing?

Yep.

How Ahrefs Gets a Billion Dollar-Worth Infrastructure with a 90% Discount

Related Stories