Why companies still want in-house data centres

https://www.economist.com/business/2023/10/05/why-companies-still-want-in-house-data-centres

by mfiguiere • 3 years ago

40 89 3 years ago

economist.com

ipython• 3 years ago

http://archive.today/hrC9Z

pm2222• 3 years ago

Simply put: it makes sense. When you are big enough, why not?

adaml_623• 3 years ago

That article is a bit of a mishmash of factors, pros, cons, and trends

Nextgrid11• 3 years ago

If the cloud was truly cost-effective then AWS and its competitors wouldn't be the cash cows that they are.

andrewstuart2• 3 years ago

This is partly true. The reason AWS et al ever grew in the first place was also that managing data centers well and automatically used to be a lot harder, and was assumed to be a whole org's full-time mostly manual job. AWS, Google, et al showed how well this could be done with software, showed the benefits of making it easier to do even for app engineers, and incentives shifted. Tools have significantly improved, though, and we're getting back to the point where it can be done with OSS and a team or three of engineers who really know the stacks involved.

Nextgrid11• 3 years ago

In my opinion, the promised "simplicity" of the cloud never materialized. You've replaced physical hardware that's easy to reason about with opaque, unpleasant and eventually-consistent APIs along unnecessary complexity (VPCs, etc) only designed to vendor-lock you even more into the provider. The staff savings never materialized - every company I've been involved with had at least one "DevOps" engineer, but instead of fighting with hardware he's constantly messing with YAML/Terraform/CDK files.

nine_k3• 3 years ago

The cloud is cost-effective when you start, because you can start using a cheap or even a free plan, and get many pre-configured services.

The cloud is cost-effective when your load is highly spiky, so you can spin up a ton of capacity during the spike, but not pay for it during the lull. (I worked in a company for which this was a principal case.)

If you are a relatively large company and want compute mostly for internal IT needs, especially for sensitive data, it may be cheaper to run your own server room and a private cloud. It won't be cheap in absolute terms though.

thisisnico• 3 years ago

I work as an IT Infrastructure Architect for VMware and Windows Based Enterprise workloads and a lot of the time it's just more cost effective to have the servers in house for many of the workloads. It isn't the case for ALL workloads, for example we move almost all exchange email services to office 365, in combination with Azure AD for active directory service. As far as datacenter space and power usage, a lot of the time the server room is located in the utility room / closets, they already have the space and the building and the power infrastructure. Typically they have vendors that require VM's because the vendor software is not cloud based, many of these enterprise software can consume quite a lot of CPU/RAM/DISK and are very latency sensitive. Having a couple of powerful vmware hosts and an all NVMe SSD SAN is really really nice for performance and speeding up these applications and not paying 4K+$ a month for equivalent hardware in the cloud. You can even argue against the capital investment cost because a lot of this gear you can lease directly from DELL, and have full service contract for 7 years 24x7 4 hour SLA on hardware.

Nextgrid11• 3 years ago

The cloud is only cost-effective if your workload fits into their loss leaders where they're actually giving you reasonably-priced services. Anything else and you're still getting scammed. It may be that the amount is too low to matter and the pragmatic decision is to keep going for the time being, but you're still getting scammed.

The fast scaling/up down is a good argument but you need to proceed with caution:

1) is your application actually structured to be able to do this? Would the overheads of developing your application in this way negate the savings (a fully distributed service with queues/etc has more development/management overhead than your basic monolithic MVC app, which may eat up all your savings)?

You need to do your own numbers here and be pragmatic. You'll have plenty of resume-driven people who see the potential of an engineering playground and will immediately tell you whatever it takes to make this happen, regardless of whether it will benefit the business.

2) how much does it actually cost to run your application at full scale outside of the cloud? I've seen places that do the whole autoscaling thing (and have lots more moving parts as a result) and yet a fully-scaled-up equivalent on bare-metal would cost less than the baseline scale on the cloud, thus removing the business case for it.

Keep in mind that the cloud effectively ate all the hardware improvements we've had in the last decade so switching to bare-metal hosting means you'll need a lot less hardware. A cloud "vCPU" is more like a pocket calculator when compared to a real CPU, and same with storage.

rightbyte• 3 years ago

The cloud is cost-effective when you start, because you can start using a cheap or even a free plan

Is it? The marginal cost of setting up a old computer as a server is essentially zero.

I think the main benefit is that devs can spend $$$ on AWS without doing a procurement. I.e. AWS solves a beurocracy problem not a technical one.

dathinab10• 3 years ago

Depending on what your company does even if it starts out and gets free credits through the AWS startup program it still might not be cheaper.

But it's more flexible and a lot of startups often pivot a view times.

Through if you have high compute and somewhat longer term predictable levels of compute then e.g. renting a rack in a server center and placing your hardware in can be notifiable cheaper (when your book keeping splits the initial buying costs across the years you run the hardware)

And then there is one huge lie about cloud, that it's way less work. It's not. It's a mess. I mean seriously we have a few docent years now of experience about what is user friendly and what is not when it comes to API/interface design and I would argue ~90% of cloud design from AWS is just plain terrible. That costs time, and time is money.

goodpoint4• 3 years ago

It's amazing how people don't see this. Using the clouds is just like renting. In the long term it's more expensive. Cloud operators are not charities.

greggsy• 3 years ago

They also need to renew infrastructure on a continual basis

jeffbee5• 3 years ago

That is actually the perfect analogy. It is exactly like rent vs buy housing discourse, in that many people harbor irrational preferences for buying even when renting is demonstrably cheaper.

VirusNewbie4• 3 years ago

Right and if certain times of the year you need an eight bedroom house for a week, while other times a two bedroom is fine, then renting may be way cheaper than buying an eight bedroom house.

SoftTalker2• 3 years ago

Yet many/most companies rent their offices or facilites, because they don't want to be in the property management business. It may cost more in the end, but it's more predictable and lower risk.

I think the analogy works best for SaaS but not necessarily for PaaS in most cases.

MattGaiser• 3 years ago

Or simply that the benefits of economies of scale are enormous in this area.

goodpoint4• 3 years ago

They are not.

littlestymaar• 3 years ago

The expected argument is probably something like “it's economy of scale”, but in practice it absolutely looks the other way round: Amazon is much less profitable than it is more expensive than the alternatives[1], so if anything their seems to be negative economies of scales.

[1] if you're twice as expensive than a company that has say 10% margin, then with equal costs you should be with a 55% margin. And Amazon is more often than not more than twice as expensive as the alternatives.

jeffbee5• 3 years ago

That is a bogus economic argument. There’s plenty of room in the equation for cloud operators to be profitable offering a product that is cheaper than on-prem.

Nextgrid11• 3 years ago

Which in practice doesn't seem to be happening though. The cloud marketing/misinformation machine is working well enough that there's no point actually becoming cheaper than on-prem since the marketing is successfully convincing people to overpay.

dathinab10• 3 years ago

yes but current AWS and similar mainly win through flexibility and services they provide not on cost for a immutable compute capacity. For that nearly always getting your own hardware is cheaper when you look at a time span of multiple years (but running it in a rented rack in a server center, there are services like that).

next_xibalba• 3 years ago

Economies of scale negate this reasoning. (Among several other factors).

Nextgrid11• 3 years ago

AWS charges 90x more for bandwidth than conventional hosting companies. The economies of scale are in Jeff Bezos' pocket instead of being passed to customers.

renewiltord3• 3 years ago

Can’t get high utilization modern cpu on cloud for cheap. Ryzen 7950x single core very fast and very cheap.

trillic• 3 years ago

You want a consumer desktop chip in your server? Consider the I/O.

Nextgrid11• 3 years ago

If he can buy 2 (or more) consumer-grade chips for the price of one enterprise-grade one and split his workload across them then it still makes more sense to go for the consumer version.

Consumer hardware gets undeserved bad rap despite being a good fit for many of today's "cloud-native" and fault-tolerant workloads.

Don't run critical infra on it that can't be fault tolerant (conventional RDBMS), but for your cluster of queue workers? Go right ahead since the queue is designed to handle failure of any nodes gracefully.

jrockway• 3 years ago

I think it's a great choice for a lot of workloads, especially CI.

renewiltord3• 3 years ago

Some places performance/price matters most. Some places raw performance. 7950x outperforms cloud processor and cheap. But use Epyc in other places. Just empirical optimum.

technofiend3• 3 years ago

I literally built billions of dollars of equipment for my employer deploying new datacenters. Even with a mandate to pivot to cloud-first, that kind of investment still needs to be used. And predictably after suffering enough pain from cloud-preferred thinking, we're settling on a mix, in no small part due to both monetary and migration cost.

candiddevmike• 3 years ago

Are you doing show/charge back for both on premise and cloud? Do different teams manage each environment?

technofiend3• 3 years ago

Are you doing show/charge back for both on premise and cloud?

Yes, you're billed in either case. But it's the difference between blue and green dollars: money that exits the firm is much more strictly budgeted.

Do different teams manage each environment?

Sort of. Knowing how to deploy to AWS vs an internally-hosted kube farm vs managing a fleet of virtual machines are all different skills, not that you can't learn them all. At one time our org structure split those functions but now if you go far up enough the management chain is the same.

xyst• 3 years ago

archive.org - https://web.archive.org/web/20231005135420/https://www.econo...

hoofhearted2• 3 years ago

The author of the article makes a claim about using 5g technology to send the data to the DC; which would mean that it can’t be far away.

Author clearly does not understand wireless networking.

Ubiquiti sells consumer grade wireless dishes that beams a signal between two dishes point-to-point at distances of a mile with speeds of 250mpbs or greater.

I remember when they released there newest dish a few years ago and one of the IT YouTubers did a test of setting up the dish on a basic lan on two different college campus that were miles across town from each other.

It was v1 of the dish, and he got near 1gbps file transfer speeds between the two dishes without much setup or tuning.

dboreham• 3 years ago

Suspect the author is way more confused than that. The 5G distance comment, if it means anything, presumably refers to the distance between the "field-deployed" servers and the 5G tower. That does have to be quite close typically. But the mention of 5G is probably paid content because as you note, anyone with a real purpose would just deploy a fixed wireless p2p link themselves.

hoofhearted2• 3 years ago

It’s not really a 5G tower..

It literally looks like a Wireless Access Point, and can double as one sometimes.

You plug a network cable into a power supply for dedicated power over POE, and another network cable into your LAN. On the other side where you other dish is, you do the exact same. Plug the network cable into a LAN, and boom you will magically have the devices on both sides on the same network.

The dishes just have to a clear line of sight, and you can get some insane distances.

There is a community mesh WiFi network in New York City that uses these exact ubiquiti dishes to beam wireless high speed trunks across the water to the different burrows; and to uptown.

For just $800, you can set up a pair of air fiber dishes, and be able to beam a 2gbps trunk up to 7 miles away.

https://store.ui.com/us/en/pro/category/all-60ghz-wireless/p...

A few years ago, we moved to a larger space down the street; but Verizon couldn’t get fiber installed in time of our move out date of our existing location.

We set up dishes on each building as they had line of sight fortunately. We ran a VOIP network over it for a month without any hiccups. Even in the worst storms, we didn’t see any speed slow downs or packet loss. It was very impressive.

Pannoniae• 3 years ago

The wording is certainly interesting. This article is written as if cloud was a holy grail, with companies still being stuck in the stone-age era of.... owning servers. They also pretend that the cloud is the future for everything, and that on-premise servers are legacy technology.

The following paragraph is the most illuminating: "The decision to build or rent may depend on a user’s data intensity. Consider a firm in America with a medium-sized data centre which thinks its computing load will rise roughly four-fold over a decade. In that case, building its own breaks even in seven years or so, and ends up being 5% cheaper overall than leasing, according to data from Schneider Electric. If the load remains stable, renting is the less expensive option, by a similar amount (assuming a flat cost of capital)."

This assumes that cloud costs are stagnant and reliable. This is a very naive assumption - it can be seen that if the cloud providers think they can get away with cost increases, they will. On-premises works out cheaper in the majority of cases, even including the cost of a sysadmin team.

VirusNewbie4• 3 years ago

I’ve never worked at a small company when on-prem is cheaper, because having to provision for peak load is not cost effective.

jeremyjh• 3 years ago

A lot of small companies have very stable workloads for years. Unless you meant startups?

johnklos3• 3 years ago

That's a sales line. Or do you work exclusively for small companies that are selling high demand online services that're trying really hard to go viral?

The only other time I can imagine is if a company isn't going to exist in six months, such as movie productions, or companies that have so little going on that they don't need more than a $10 a month plan.

Otherwise, on premises or colocation are invariably cheaper when there's any kind of real usage.

safetytrick2• 3 years ago

It just depends on the business, b2b can be very consistent. Some workloads just have predictable behavior.

I worked at a medium sized place where we had plans in place to handle the workload for our entire market on-prem (if we were ever to capture that market).

We realized that horizontal scalability wasn't important to us because we had enough of the market to know that the other half of the market could be supported on our existing solution.

downrightmike• 3 years ago

It becomes cheaper because they will run long after support has ended and cobble together a working server from the pile of broken ones, all the while rejecting getting a new server that could handle and replace dozens.

jjav4• 3 years ago

having to provision for peak load is not cost effective.

That is true, if peak load is much much higher than mean load.

For most companies, that's not the case.

On the cloud you are overpaying for cpu, network and storage by a very wide margin. That makes sense if your peak load is, say, 100x your mean load (and you have sufficienly smart autoscaling to optimize for this).

But for most companies there is not such huge multiplier from mean usage to peak usage. So those are better off with owning hardware.

fatherzine• 3 years ago

datacenter is to information what power plant is to electricity

Economically, scale makes the centralized solution more efficient. That doesn't preclude independent entities running their own stoves, or running their own internal combustion engines, or their own industrial furnaces, but confines those uses to niche areas where direct power generation is more convenient/efficient due to additional constraints.

Politically, centralization means power. Power seeking entities will eventually outright ban independent power generation, for whatever excuse du jour. See NY ban of gas stoves, or EU plans to ban ICE cars. For information, 'to prevent illegal activity' excuse is usually used.

jjav4• 3 years ago

datacenter is to information what power plant is to electricity

Very much not.

Running a generator at home is much more expensive than paying your utility for electricity. But running a server at home is far cheaper than paying AWS if you actualy use the server.

Adiqq• 3 years ago

I don't have much contact with cloud, but it just seems expensive and unstable in practice from business perspective.

Cloud offers standardization and support, so in some cases it might be preferable, because it meets requirements without spending additional time on setup or operations, but eventually cost might become problem, especially if cloud providers will decide to increase prices and you can't easily migrate it to on-premise.

Probably it's best to stay hybrid and be always ready to at least change cloud providers, so you need to avoid vendor lock-in. That sounds like cloud native, so why bother with cloud, if you can have some servers, use them with Rancher and create Kubernetes clusters?

There will be some overhead, but your company controls servers, can easily migrate also to cloud, if needed, provides most functionality that most teams may require and depends only on electricity, internet providers and workers required to operate some data center, so it can always choose cheapest option to operate.

renewiltord3• 3 years ago

Big problem with on-prem is lead time on expansion. Can easily slow down expansion. Hard to project out.

But if you have high utilization stable load, it can pay off to have on-prem. It's still a big risk, though. Cloud has a stable platform to build off. You can trust the 'interface' so to speak. Meanwhile you suddenly have to worry about driver incompatibilities, disk wear, stuff like that.

We have cloud infra and on-prem infra. Big problem with cloud is performance for price.

Even for small company with stable load, think about it. You have to have a guy aware of and thinking of the fact that the Mellanox SFP+ card you have has a driver regression on Linux 6.2. Maintenance nightmare. Better to have cloud. Clear interface to build against, and easy to replicate when problems occur.

Sysadmins do stuff. You can't go sysadmin-free and expect efficient spending. Someone else just ends up sysadmin and frequently someone too high paid so you're losing money on the time they play that role.

jjav4• 3 years ago

Cloud makes economic sense in two scenarios.

For startups that don't have a a clue what they will need because still searching for market fit so it doesn't make sense to provision hardware not knowing what is needed. If you suddenly find market fit and usage grows 100x+ in a few months, you totally want to be on cloud.

And in larger coporations, those that have extremely spiky usage. For example a company that has 90% of their usage at the end of each month, it doesn't make sense to own hardware to meet the peak load if it will be idle 28 days of the month.

Most corporations are not in those two categories though! If you are not, you are vastly overpaying for compute, storage and network compared to owning your own hardware. It is sad that such companies are rushing to overpay for cloud just because it's hip.

Objectively analyze your needs and usage and make a data-driven decision. Don't rush into cloud just because everyone else did.

baz00• 3 years ago

I still want an in house datacentre. Our move to cloud has a 5x cost increase. We need more staff to run it as well (!).

dathinab10• 3 years ago

ah yes -- you need less people -- is probably the biggest lie of modern cloud

through technically it should be that way

so I would argue the current cloud infrastructure is seriously messed up

nikau• 3 years ago

With cloud you can have less staff, but you need to pay more per headcount as they need to know how to manage tens if not hundreds of different services.

Traditional on-prem has silos where the DBA just knows databases and their salary is capped because of that.

I guess in theory you could have a "cloud DBA", but in my experience every business unit wants to keep autonomy over their cloud account/project and do it all in house to get rid of the delays that occur with centralized resources.

jeffbee5• 3 years ago

The main reason is a whole generation of IT executives who don’t want to look superfluous.

dathinab10• 3 years ago

It's paywalled but...

if someone says it's "unlikely" it either is a stylistic method to goat people or they don't understand much about the technology, because there is nothing unlikely about it

nologic012• 3 years ago

The decision to build or rent may depend on a user’s data intensity.

Think of brick-and-mortars real estate (e.g. office space) which for most companies is completely outside their core value proposition hence they would be most inclined to lease. Accurate statistics are surprisingly non-trivial to find but it seems like the company owned segment is circa 20% (versus 80% leasing their office space needs).

Now when you think of the role data centers play in a growing digital economy, for many companies increasingly their value proposition is linked to their digital infrastructure. So that would push the fraction of "owned" versus "leased" substantially higher. This would leave still a eye-wateringly massive amount to be served by "hyperscalers". This is really what the "cloud revolution" (renting somebody elses linux server) is mostly about. There is a certain naturalness to this state of affairs.

But there is the major catch. Imagine if practially all the real estate in the world is owned by just a handful of mega-corps and anybody needing office space needs to rent from them. Companies small and large will quickly realize what the term "rent-seeking" [1] means... when the seller holds all the cards and the buyer holds none.

Rationally a hyperscaler oligopoly would attempt to keep the price of digital infrastructure at just below the ownership cost, though greed and various frictions in migrating back to in-house might temporarily allow milking higher rents.

In any case its a most interesting parameter to observe in these rapidly changing times

[1] https://en.wikipedia.org/wiki/Rent-seeking

johnklos3• 3 years ago

People say cloud is cheaper, but that's based on horrible assumptions such as the idea that companies don't have staff that know how to maintain a server, so cloud costs have to be compared with in-house plus the salary of a systems administrator.

People forget that services in the cloud require technical know how, and that the lack of technical know how can easily result in wildly insane cloud bills.

Really, though, any heavily used service will always cost more in the cloud. Transfer? Sure, bringing Internet to your location might cost an arm and a leg, but it's trivial to find colocation with transfer that's significantly cheaper than anything cloud.

CPU? Definitely. Ancient Xeon cores are a dime a dozen, but they don't compare to even an inexpensive last generation Ryzen. Storage? Absolutely, and the price differences are absolutely ridiculous. Even GPU workloads are cheaper if you have to run them for more than a handful of months.

"But you can scale in the cloud!" Right. You can scale in real life, too, AND you can scale in real life WITH the cloud, if you want, so that's a silly thing to say. It's not one or the other, exclusively.

I have yet to find a single example of a company that isn't doing really silly things saving money by moving to the cloud.

nullindividual2• 3 years ago

You left out all of the other costs of running a datacenter. Cooling, fire suppression, electricity, DR site, etc.

johnklos3• 3 years ago

No, I didn't leave them out. They're just not a significant factor because those costs are proportional to need and demand. A company is either large enough to have enough that those things are aggregated across tons of servers, or they're likely colocating. Also, datacenters don't require all of that to be datacenters.

Nextgrid11• 3 years ago

Your colocation provider handles all that for the flat monthly fee you're paying them for the rack space.

jjav4• 3 years ago

Cooling, fire suppression, electricity, DR site, etc.

No, unless you are particularly huge (google, facebook), you rent space in a colocation facility that handles the real estate stuff (like electricity, cooling, fire). That is the best of all worlds for most companies needs.

sgt1012• 3 years ago

On prem requires a capital investment and a cast iron workload prediction.

If you have a stable business this works. But you may still want to go to cloud if IT is not your business. An on prem infrastructure requires capital and capital can be used to change your core business, which could bring you sufficient bottom line to make cloud costs irrelevant.

If you are trying for growth then a cloud infrastructure enables that.

sgt1012• 3 years ago

Business isn't just about managing costs.

autoexec• 3 years ago

Are data privacy and security really nowhere in the list of considerations? I find it hard to believe that cloud providers aren't mining data and pulling every possible metric they can from usage. AWS alone has been responsible* for countless breeches and data leaks. Wanting to keep your data out of the hands of third parties for the protection of your own data or that of your users seems like it ought to be a factor in there somewhere when deciding to keep your data in house.

* You can argue that it's mostly been the customers who fail to configure the service correctly at fault, and you wouldn't be wrong, but any time that many people are "doing it wrong" the problem isn't just them, its the service they're using (bad defaults, hidden footguns, unclear documentation, etc).

batshibstein• 3 years ago

It's not solely a total cost decision but it's also OpEx vs. CapEx, what can be depreciated based on that, etc.

Why companies still want in-house data centres

Related Stories