The first Oxide rack being prepared for customer shipment

https://hachyderm.io/@oxidecomputer/110635621269494973

by jclulow • 3 years ago

296 192 3 years ago

mbStavola• 3 years ago

Congrats to the Oxide team, this is a massive achievement.

Now that racks are shipping it'd be awesome to see a top-to-bottom look at the hardware and software. They've given a lot of behind the scenes peeks at what they're doing via the Oxide and Friends podcast, but as far as I'm aware there is no public information on what it all looks like together.

supriyo-biswas• 3 years ago

For those not in the know, this is what is being talked about[1]. Congrats to the Oxide team!

Would love to know what kind of uses this is being put to. In this age, everyone only talks about the cloud, with its roach motel model and all.

[1] https://oxide.computer

api• 3 years ago

The demise of on premise and private data centers is greatly exaggerated. Few people here do that because the cloud is great for startups and rapid prototyping. Most on prem beyond small scale is big established companies.

There is a minor trend of established companies reconsidering cloud because it turns out it doesn’t really save money if your work load is well understood and not highly elastic. In fact cloud is often far more expensive.

CharlesW• 3 years ago

FYI for language pedants like me: It’s “on-premises” or “on-prem”. A “premise” is something assumed to be true.

lijok• 3 years ago

Well done and congratulations to the Oxide team ! Very excited to see where this company goes

steveklabnik16• 3 years ago

I am extremely proud of everyone at Oxide. It’s been a fantastic place to work, and finally getting to this point feels awesome. Of course, there is so much more to do…

For funsies, it’s neat to look back at the original announcement and discussion from four years ago: https://news.ycombinator.com/item?id=21682360

runlevel1• 3 years ago

As someone who's worked on on-prem infra automation pretty much my entire career, I'm rooting for you.

vira28• 3 years ago

Super excited to see more companies owning hardware vs renting (aka cloud).

Somewhat related if you are interested on companies which heavily uses on-Prem check out

https://github.com/viggy28/awesome-onprem

pkaye• 3 years ago

What operating system do they use?

steveklabnik16• 3 years ago

A modern rack contains many computers. We use both our own lil embedded OS, Hubris, as well as our own illumos distribution named Helios.

None of this is exposed directly to the customer, though. You run VMs with whatever you want.

71a54xd2• 3 years ago

Why is this better than a normal k8's distribution or just buying vm's from Amazon for someone who doesn't need high security or other boutique features?

slavapestov• 3 years ago

Why do you use Illumos?

nubinetwork2• 3 years ago

None of this is exposed directly to the customer, though

What about security updates and bug fixes? No platform is perfect, after all...

Or what about if (god forbid), you go out of business 5 years down the line... would the hardware be repurposeable? Or would it just become a large paperweight?

mindentropy2• 3 years ago

Pardon my knowledge but isn't this the same as what is being done on the IBM Z Mainframe?

bcantrill16• 3 years ago

Oxide has been discussed on HN a bunch over the last 3+ years (e.g., [0][1][2][3][4][5][6][7]), and while nothing is without its detractors, we have found on balance this community to be extraordinarily supportive of our outlandishly ambitious project -- thank you!

[0] When we started: https://news.ycombinator.com/item?id=21682360

[1] On the Changelog podcast: https://news.ycombinator.com/item?id=32037207

[2] On our embedded Rust OS, Hubris: https://news.ycombinator.com/item?id=29468969

[3] On running Hubris on the PineTime: https://news.ycombinator.com/item?id=30828884

[4] On compliance: https://news.ycombinator.com/item?id=34730337

[5] On our approach to rack-scale networking: https://news.ycombinator.com/item?id=34976444

[6] On our de novo hypervisor, Propolis: https://news.ycombinator.com/item?id=30671447

[7] On our boot model (and the elimination of the BIOS): https://news.ycombinator.com/item?id=33145411

throw0101a2• 3 years ago

Be the outlandish you wish to see in the world. — Not Gandhi

sacnoradhq• 3 years ago

The world is an impersonal euphemism for human culture. All things apart from physics and human nature are dentable.

bch• 3 years ago

The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.

-George Bernard Shaw

tiffanyh• 3 years ago

Wishing you best of luck.

Really curious to see if in 2023, Engineered Systems still have a market in this world of commodity cloud hardware.

Dowwie• 3 years ago

Can you share how software engineers can bring hardware to market? Fabrication, logistics, manufacturing. What should be outsourced? Who did you partner with for what?

Also, congrats.

wmf5• 3 years ago

They have discussed that on their podcast: https://www.youtube.com/@oxidecomputercompany4540

zengid• 3 years ago

I highly recommend folks check out the Oxide and Friends calls on discord, usually on Mondays 5pm PST. More info: https://oxide.computer/podcasts/oxide-and-friends

Disclaimer: I am a fan, not affiliated.

CalChris• 3 years ago

Hubris, Humility and Propolis are open source. Is anyone else using them?

reverius42• 3 years ago

Could be interesting to port Hubris to x86_64 and use it on bare metal commodity hardware -- specifically to run Rust server code, with perhaps a perf advantage over larger OSes?

Though the biggest advantage to Rust-on-bare-metal would have been running all the code in kernel space, which it seems Tasks don't allow for (in order to provide robustness at the OS level).

rcarmo• 3 years ago

Congrats to the entire team. Being a wayward hardware guy somewhere in telcomland that has followed your progress throughout the years (and having listened to pretty much every oxide podcast), I am genuinely happy for you all.

GivinStatic• 3 years ago

Do you have any liquid cooling solutions collaborators?

panick21_3• 3 years ago

Its not liquid cooled.

g0xA52A2A• 3 years ago

Hi, I saw the Oxide Twitter account mention you’d be answering some questions from this thread on this evenings podcast so I hope this gets seen in time. Though firstly I should say congratulations!

Q: Are you shipping Milan racks and if so what does the moving to Genoa entail for Oxide?

bcantrill16• 3 years ago

Yes, the rack is Milan-based (7713P) -- and good question on Genoa; we'll get into that tomorrow!

nickstinemates• 3 years ago

Oxide is such an ambitious project, I am such a fan of the aesthetic and design and of course transparency of all of the cool people that work there.

I'd love to have a rack or two some day!

c7DJTLrn• 3 years ago

I believe that good hardware and software unified into one neat package can steal customers back from the cloud. Especially in the current economic conditions where everyone's looking to save on their server bills. I hope to some day work with Oxide stuff.

anaisbetts2• 3 years ago

Congrats to the team, but after hearing about Oxide for literal years since the beginning of the company and repeatedly reading different iterations of their landing page, I still don't know what their product actually is. It's a hypervisor host? Maybe? So I can host VMs on it? And a network switch? So I can....switch stuff?

steveklabnik16• 3 years ago

I wrote this a while back, does that help? Happy to elaborate. https://news.ycombinator.com/item?id=30678324

anaisbetts2• 3 years ago

Perfect - now get that 2nd paragraph on the landing page somehow!

wcerfgba• 3 years ago

I don't really understand how having a larger minimum purchase unit (entire rack vs rack unit) is a USP. Your comments explain the emphasis on tighter integration across the stack, but it doesn't clearly show why this is a benefit.

What are the problems people are having with existing systems (like Vxrail), and how does Oxide fix those? What stories are you hearing?

noisy_boy2• 3 years ago

Seems like Oxide is aiming to be the Apple of the enterprise hardware (which isn't too surprising given the background of the people involved - Sun used to be something like that as were other fully-integrated providers, though granted that Sun didn't write Unix from scratch). Almost like coming to a full circle from the days where the hardware and the software was all done in an integrated fashion before Linux turned-up and started to run on your toaster.

From your referenced comment:

The rack isn't built in such a way that you can just pull out a sled and shove it into another rack; the whole thing is built in a cohesive way.
other vendors will sell you a rack, but it's made up of 1U or 2U servers, not designed as a cohesive whole, but as a collection of individual parts

What I'm curious about is how analogous or different is this cohesiveness to the days where vendors built the complete system? Is that the main selling point or there are nuances to it?

71a54xd2• 3 years ago

It's like an on prem AWS for devs. I don't understand the use case but the hardware is cool.

electroly3• 3 years ago

It's AWS Outposts Rack without the AWS connection. That is, you get a turnkey rack with the servers, networking, hypervisor, and support software preconfigured. You plug it into power and network and it's ready to run your VMs.

Outposts, too, started with a full-rack configuration only, but they eventually introduced an individual server configuration as well. It'll be interesting to see if Oxide eventually decides to serve the smaller market that doesn't have the scale for whole-rack-at-a-time.

Solvency• 3 years ago

So what was the precursor solution to this before AWS?

shrubble2• 3 years ago

It is a rack of servers, but, every aspect of it is supposed to be engineered to include the full list of things that are needed to make a rack of servers a useful VM hosting setup. So it includes the networking, connection to the service processors which allow you to remotely access each server, other management things, etc.

Once installed, you plug in the network connection(s) and add power, then boot up. Then you can add your VMs and start running them.

freeopinion• 3 years ago

Except that you cut out all the fat of traditional servers. These aren't COTS servers. Imagine if somebody re-engineered an entire rack in the same way they once introduced blade computers. Any part that can be moved from standalone to rackwide should be cut out of individual shelves. So I would expect things like serial buses, video, certain power aspects, etc to all be removed from the individual shelves.

When you cut out all that fat, you can fit a lot more muscle in its place. Then you can arrange everything on a rack scale for airflow, power, redundancy, etc.

The obvious downside is that you are locked into a single supplier for everything in the rack. Somebody please add a link here for Oxide's strategy to minimize this risk for their customers. (And feel free to correct any mistaken over-simplification.) Overall, they have seemed incredibly open to sharing in a way that allows others to play. That might just be because they would die if they didn't, but I think a lot of people are willing to accept that they are pretty all-around awesome folks trying to make the world better. And making money in the process is not a bad gig. We'll see which other vendors jump into the pool.

I wish them great success. But I'm wondering why they haven't coordinated my shipping address for their first shipment. First one's free, right?

panick21_3• 3 years ago

I don't get it. What hard to understand? They are selling servers, specifically in a rack. This has been a discussion in every thread and its pretty clear on their landing page.

greyface-• 3 years ago

Who's the lucky customer?

lwhalen2• 3 years ago

Is there a public price list anywhere?

atdrummond• 3 years ago

What really put me off of them (and as a fan of BCantrill and others there) is this information is highly obfuscated and I was never reached out to the two times I contacted Oxide to find out more info (both times on behalf of an org that could more than pay).

Still think they’ll succeed big but I don’t think they’ve fully dialed in what is important to people who may be able to pull the trigger on a decision like this.

dijit5• 3 years ago

Second this, I was (and am) in a position to pay substantially for such a system and the few times I reached out was met with radio silence.

possibly because I am in Europe and they want to focus on the NA market, not sure.

bcantrill16• 3 years ago

With our apologies, we can't seem to find anything based on either your HN username or the e-mail address in your profile. So sorry that it was somehow dropped or otherwise went into the ether! Would you mind mailing either me (my first name at oxidecomputer.com) or sales@oxidecomputer.com? Thanks for your interest (and fandom!) and apologies again!

sydbarrett74• 3 years ago

Anyone willing to disclose minimum pricing? Are we talking tens of thousands? Hundreds? I hate when people say, 'If you have to ask, you can't afford it.' Please don't be that guy.

brucepink• 3 years ago

If you compare an 0xide rack with a standard combo of Dl380s, Nexus switches, netapp filers and Vmware licences, and look at the specs page - “ 1024TB of raw storage in NVMe” - there’s no way this is tens of thousands and I’d be a bit surprised if it was in the hundreds either.

sgt• 3 years ago

You're saying this rack might cost a million dollars?

wmf5• 3 years ago

Based on the components I'm guessing a half million.

jamiek88• 3 years ago

Based on the components I’d say that’s the build cost not price!

shrubble2• 3 years ago

32x EPYC servers, figure $10K/server on a base config = $320,000 .

Add in what appears to be, 2x 100Gb Ethernet switches, the 100Gbps NICs, cabling, other rack hardware; service processors that allow you to control the servers, whatever amount of NVME drives, etc. Assembly, burn in, test etc.

My guess is that a base config would be somewhere between $400K to $500K but could very definitely go up from there for a completely "loaded" config.

andrewstuart• 3 years ago

I didn’t understand the business opportunity of Oxide at all. Didn’t make sense to me.

However if they’re aiming at the companies parachuting out of the cloud back to data centers and on prem then it makes a lot of sense.

It’s possible that the price comparison is not with comparable computing devices, but simply with the 9 cents per gigabyte egress fee from the major clouds.

If I was marketing director at Oxide I’d focus all messaging on “9 cents”.

fbdab103• 3 years ago

They sell servers, what does not make sense about it? You can argue about the specific niche (big enough to run their own hardware, too small to design their own), but companies need somewhere to do compute. If nothing else, I love their approach to rethinking all of the individual software components in the stack and tossing those things which do not make sense in the modern era.

crote2• 3 years ago

They seem to sell one set of servers, that's the part that doesn't make sense.

Where is this magical company that needs exactly one rack of exactly one type of server? The vast majority of companies needing this much compute will also be interested in storage servers, servers filled with GPUs, special high-RAM nodes, etc. And at that point you'll also be using some kind of router for proper connectivity.

Why bother going for a proprietary solution from an unproven company for the regular compute nodes, and forcing yourself to overcommit by buying it per rack? Why not just get them from proven vendors?

Nullabillity• 3 years ago

The question isn't whether anyone fits into that niche, but why anyone who does would buy this over a plain old off-the-shelf system.

TOGoS• 3 years ago

Anyone want to explain what the heck Oxide is? Based on the comments it sounds like a biodegradable plastic wrap?

steveklabnik16• 3 years ago

I wrote this a while back, does that help? Happy to elaborate. https://news.ycombinator.com/item?id=30678324

rhinoceraptor• 3 years ago

It's a cloud in a rack.

ThinkBeat2• 3 years ago

So they reinvented the mainframe with a prettier exterior?

Their own OS: Check Big tall box: Check Expensive: Check (well no price list i have seen so far) Proprietary hardware at least interfaces. (?) upgrades must be bought from the vendor (?)

Right to repair?

crote2• 3 years ago

Not quite. A traditional mainframe is highly integrated, to the point of allowing hotswapping of CPU and memory. Oxide seems to be a fairly standard collection of x64 hardware, with a proprietary management software sauce.

steveklabnik16• 3 years ago

Just to expand slightly, "proprietary" in the sense of "built by us for this computer," but not in the way that free or open source software developers speak of the word "proprietary," the vast majority of what we do is MPL licensed, to the degree that it is actually possbile.

jclulow• 3 years ago

One of our core product goals is to actually allow relatively hot swapping of individual compute sleds. It's true that each sled is an x86 server, but there's control software managing the whole rack as a unit, with live migration of workloads and replicated storage. The architecture of the whole thing leans towards treating sleds as essentially like CPU and memory (and storage) boards in previous generations of large scale computers like mainframes.

ThinkBeat2• 3 years ago

This is the case for some .

Like the Tandem Nonstop series. (Now part of Hewlett Packard Enterprise) I had a contract with a 9/11 call handler for a while. They had one of them (or possibly more). On display of sorts through a window of bulletproof glass.

The IBM Z series does not allow hot swapping the CPU and I dont believe it ever did. The parts on the Z Series are super easy to replace. All in nice components sitting on a rails to yank in or out.

wmf5• 3 years ago

It's more like reinventing Sun with an uglier exterior but yes.

ahl2• 3 years ago

... but with more inner beauty.

bcantrill16• 3 years ago

Hey now, the Oxide rack is pretty good looking[0] -- and not to take anything away from the likes of the E10K, I personally think the Oxide rack is better looking than the Sun machines of yore. ;)

[0] https://oxide.computer/

__d4• 3 years ago

Finally, a true dark mode server.

siliconc0w• 3 years ago

From a commodity hardware perspective I'm not sure there is much to be excited about but if it's a meaningful better and cost competitive IaaS maybe that is exciting. Also you are probably going to want GPU support which may be hard with their super customized offering.

If I were to do a bare metal deployment I'd look at kubernetes + kubevirt + a software defined storage provider. Then you get a common orchestration layer for VMs, containers, or other distribute workloads (ML training/inference) but don't need to pay the Vmware Tax and you'd be using a common enough primitive that you can move workloads around to 'burst' to the cheapest cloud as needed.

riking2• 3 years ago

you can just follow one of the "kubernetes on ec2" guides and adapt it to their apis

osti• 3 years ago

Does oxide provide some sort of management software like openstack on top of the hardware?

steveklabnik16• 3 years ago

You interact through the rack via an API, yes. I hesitate to say "like openstack" because these sorts of systems are huge, complicated, and what "like" means depends on you know, what you use. But you do get management software, yes.

grrdotcloud• 3 years ago

That's everything they do I do believe.

tpurves• 3 years ago

It looks cool, what primary problem does it solve vs AWS?

pizza• 3 years ago

Ownership and control vs rent and managed setup

sixdonuts2• 3 years ago

Additionally, compared to AWS Outposts which use inefficient commodity hardware and closed source software, the Oxide platform uses integrated hardware and software, and open source software.

dataangel4• 3 years ago

Somebody help me understand the business value. All the tech is cool but I don't get the business model, it seems deeply impractical.

* You buy your own servers instead of renting, which is what most people are doing now. They argue there's a case for this, but it seems like a shrinking market. Everything has gone cloud.

* Even if there are lots of people who want to leave the cloud, all their data is there. That's how they get you -- it costs nothing to bring data in and a lot to transfer it out. So high cost to switch.

* AWS and others provide tons of other services in their clouds, which if you depend on you'll have to build out on top of Oxide. So even higher cost to switch.

* Even though you bought your own servers, you still have to run everything inside VMs, which introduce the sort of issues you would hope to avoid by buying your own servers! Why is this? Because they're building everything on Illumos (Solaris) which is for all practical purposes is dead outside Oxide and delivering questionable value here.

* Based on blogs/twitter/mastodon they have put a lot of effort into perfecting these weird EE side quests, but they're not making real new hardware (no new CPU, no new fabric, etc). I am skeptical any customers will notice or care and would have not noticed had they used off the shelf hardware/power setups.

So you have to be this ultra-bizarre customer, somebody who wants their own servers, but doesn't mind VMs, doesn't need to migrate out of the cloud but wants this instead of whatever hardware they manage themselves now, who will buy a rack at a time, who doesn't need any custom hardware, and is willing to put up with whatever off-the-beaten path difficulties are going to occur because of the custom stuff they've done that's AFAICT is very low value for the customer. Who is this? Even the poster child for needing on prem, the CIA is on AWS now.

I don't get it, it just seems like a bunch of geeks playing with VC money?

dcre• 3 years ago

It is simply false that everything has gone cloud. The whole argument falls down on the first premise. Also nearly everyone who owns their own servers still runs VMs on them.

elishah• 3 years ago

Also nearly everyone who owns their own servers still runs VMs on them.

This strikes me as at least as much of a leap as "everything has gone cloud."

Containers are... kind of a thing. And while there certainly are use cases for VMs over containers, they're comparatively niche.

This product seems as if it'd be a better fit for every real use case I've ever seen if it were a prebuilt kubernetes cluster rather than a prebuilt VM hypervisor.

dataangel4• 3 years ago

In 15 years of professional experience I've never worked at a place that uses VMs on the servers they own. They're just going to run Linux off an image so what's the point? You might want to look outside your niche.

faitswulff5• 3 years ago

~Bandcamp~ Basecamp is doing just the opposite and leaving the cloud, in fact: https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47...

qingcharles• 3 years ago

Many companies are leaving cloud hosting due to spiraling costs. Even well-knowns like 37signals:

https://world.hey.com/dhh/we-have-left-the-cloud-251760fb

It's nice to have options. Cloud good. Self-hosting good. Middle options good.

rzzzt• 3 years ago

My mind also jumped to this post. But can you name another company that did so recently?

electroly3• 3 years ago

Addressing #2 and #3, a "hybrid cloud" architecture can include a site-to-site VPN or direct fiber connection to a cloud. In AWS, Direct Connect data transfer pricing effectively makes your on-prem DC or colocation facility into an availability zone in your AWS Region. Direct Connect is $0.02/GB egress (out of AWS) and free ingress (into AWS), which is a better deal than cross-AZ data transfer within the AWS Region. Cross-AZ within an AWS Region is effectively $0.02/GB in both directions.

This way, you can run your big static workloads on-prem to save money, and run your fiddly dynamic workloads in AWS and continue to use S3.

That said, if a hybrid cloud architecture is your plan and you desire a managed rack-at-a-time experience, AWS Outposts would seem to be the safer pick. They've been shipping for years and they have public pricing that you can look at. I'm not sure that Oxide specifically has an opening for customers who want to keep their cloud. I wish them luck.

https://aws.amazon.com/outposts/rack/pricing/

nighmi• 3 years ago

they're building everything on Illumos (Solaris)

This is an amazing plus in my eyes. Solaris systems are amazing.

sergiotapia• 3 years ago

Cloud was a low interest rate phenomena. I predict a return to metal servers and managed data centers.

oldtownroad• 3 years ago

Part of the appeal of the cloud over on-premise is that on-premise is not just expensive, but hard: oxide’s product isn’t just on-premise hardware, it’s on-premise hardware _that is easy_*. If on-premise was just expensive, it would be so much more appealing — because the cloud is expensive too!

Most every software engineer has worked in an org where spending six figures per month on AWS or GCP is totally normal and acceptable because the alternative, buying hardware, is this awful scary unknown that could be cheaper but could also blow the entire company up. If oxide can solve that, suddenly on-premise becomes much more attractive.

Yes, people are hooked on the cloud, but not because of data transfer… because it’s easy.

* well, the first few deployments might not be easy but that’s true of anything new.

thinkmassive• 3 years ago

100% agree with you

Also, what this person said :) https://news.ycombinator.com/item?id=36552245

alephnerd3• 3 years ago

Not OP, but what differentiates Oxide from VMWare or Dell? I can get pretty competitive quotes from both if I wanted to build my own DC, or if had much more niche HPC requirements I could go with NVIDIA.

bcantrill16• 3 years ago

I'm not sure what "weird EE side quests" you're referring to, but anyone interested in learning what we've done in terms of hardware should listen to the team in its own voice, e.g. in their description of our board bringup.[0]

[0] https://oxide-and-friends.transistor.fm/episodes/tales-from-...

tw04• 3 years ago

* You buy your own servers instead of renting, which is what most people are doing now. They argue there's a case for this, but it seems like a shrinking market. Everything has gone cloud.

This is very much not true and seems to be a result of people in the valley thinking the rest of the world operates like the valley. In the rest of the world I've found mature businesses that bought into the "cloud is the best" quickly started doing the math on their billing rate and realized there is a VERY small subset of their business that has any reason to be in the cloud. Actually one of the very best use-cases of public cloud I've seen is a finance firm that sticks new products into the cloud until they hit maturity so they can properly right-size the on-prem permanent home for them. And if those products never take off, they just move on to the next one. They're willing to pay a premium for 12-18 months because they can justify it financially.

* Even if there are lots of people who want to leave the cloud, all their data is there. That's how they get you -- it costs nothing to bring data in and a lot to transfer it out. So high cost to switch.

And yet company's do it all the time. I think you'll again find mature fortune 500s can do the math on the exit cost vs. staying cost and quickly justify leaving in a reasonable time window.

* AWS and others provide tons of other services in their clouds, which if you depend on you'll have to build out on top of Oxide. So even higher cost to switch.

And as you've seen plenty of people here point out: most of those services tend to be overrated. OK, so you've got database as a service: except now you can't actually tune it to your specific workload. And $/query, even ignoring performance, is astronomically higher than building your own and paying a DBA to manage it unless you're a 2-man startup.

* Even though you bought your own servers, you still have to run everything inside VMs, which introduce the sort of issues you would hope to avoid by buying your own servers! Why is this? Because they're building everything on Illumos (Solaris) which is for all practical purposes is dead outside Oxide and delivering questionable value here.

I don't know of a single enterprise that has run anything BUT VMs for the last decade. Other than Mainframe (which you can argue is actually just VMs in a different name), and some HFT-type applications that need the lowest possible latency at all costs, it's all virtualized. As for Illumos: why do you care? Oxide is supporting and maintaining it as an appliance. Tape has been "dead" for 2 decades. FreeBSD has been "dead" since the early 2000s. It's only dead for people that don't deal with enterprise IT.

* Based on blogs/twitter/mastodon they have put a lot of effort into perfecting these weird EE side quests, but they're not making real new hardware (no new CPU, no new fabric, etc). I am skeptical any customers will notice or care and would have not noticed had they used off the shelf hardware/power setups.

I have no doubt they've done their research, and I can tell you from my industry experience there is a large cross-section of people who want an easy button. There's a reason why companys like Nutanix exist and have the market cap they do - but they could never actually get the whole way there because they got wrapped up in the "everything is software defined!!!". Which works really well until you realize that you're left to your own devices on networking.

So you have to be this ultra-bizarre customer, somebody who wants their own servers, but doesn't mind VMs, doesn't need to migrate out of the cloud but wants this instead of whatever hardware they manage themselves now, who will buy a rack at a time, who doesn't need any custom hardware, and is willing to put up with whatever off-the-beaten path difficulties are going to occur because of the custom stuff they've done that's AFAICT is very low value for the customer. Who is this? Even the poster child for needing on prem, the CIA is on AWS now.

I mean no disrespect but I get the impression you haven't ever worked with a fortune 500 that's outside of the valley. This is EXACTLY what they all want. They aren't going to run their entire datacenter on this, but when the datacenter is measured in hundreds to thousands of servers, they've got plenty of workloads that it's a perfect fit for

dataangel4• 3 years ago

I've never worked in the valley lol

cjalmeida• 3 years ago

A lot of corprotation have a very hard time moving workloads to the cloud due to regulations alone. In some places it's a compliance nightmare, a landing page with "trust me, we're secure" from AWS is far from enough for legal counselors, auditors and the CISO.

ec109685• 3 years ago

Those corporations are dumb. Way more resources are spent making AWS secure than could ever be spent on bespoke security implementations.

rcxdude2• 3 years ago

This is for people who want a turnkey cloud that they own, not rent. Renting is really expensive: you could probably buy a few of these racks a year for the cost of renting the same amount of compute on AWS. You also get a software stack which gives you the tools to manage the hardware without growing your own, with an extremely high level of trust. It's extremely well aimed at the companies which can afford the racks, because operating at that scale you are being fleeced by the hyperscalers.

doctor_eval• 3 years ago

My own experience here was with a rack of servers in a DC running a fairly heavy workload for a number of customers.

Long story short, despite pressure to move from our parent company, there was literally no dimension - services, finance, performance, support - on which AWS was better for our workload. Our DC capex+opex was less than a quarter of the parent company’s opex spend on AWS, despite our workload being much greater - and we needed fewer staff to manage it.

So I for one am really happy to see Oxide producing an on premise private cloud solution like this.

samcat116• 3 years ago

This is super cool. I realize a lot of HN folks might not see the point of this, but it literally saves an entire team of people for companies.

ultra_nick• 3 years ago

Servers with high quality software integration. These provide the same value to businesses as Apple devices provide to consumers. Hopefully they "just work" and eliminate a bunch of Devops distractions.

Most hardware companies have terrible software. If Oxide can handle manufacturing and logistics, then they'll be huge in about 10 years.

Devasta• 3 years ago

I'm convinced that if anything is going to reverse the migration to the cloud, it's Oxide.

Is there anything in the works for using some of this in a homelab? Mainframes (unofficially) have Hercules, be good to see something similar for folks who want to experiment.

syntaxing2• 3 years ago

Maybe I’m misunderstanding the product but wouldn’t hypervisor do the same thing for homelab related stuff? You’ll have to provide your own hardware but shouldn’t be that difficult.

silisili• 3 years ago

In what manner? The vast majority of companies I don't think need or want an entire rack.

So this has to be targeted more at F500, or maybe even to colos?

syntaxing2• 3 years ago

Am I standing this correctly? This is a on premise drop in replacement for your cloud service like AWS?

anyoneamous• 3 years ago

Not unless you have strictly constrained yourself to using vanilla VMs and nothing else.

rvz• 3 years ago

As long as it encourages on-premise self-hosting, this can only be a good thing.

c_o_n_v_e_x• 3 years ago

I LOVE how these guys unit test hardware sub systems before putting them onto whole motherboards.

faitswulff5• 3 years ago

How do they do that?

bcantrill16• 3 years ago

A great question! You can find more details on our approach in our Oxide and Friends episode on the power of proto boards.[0]

[0] https://oxide-and-friends.transistor.fm/episodes/the-power-o...

bastawhiz• 3 years ago

The On the Metal podcast had been one of my absolute favorites. It's clear these folks are exceptional, and they deserve congratulations!

ilaksh• 3 years ago

If I were rich, I would try to buy some big clusters like that now but only if they were full of MI300Xs (and some MI300As). May ten or twenty deadly cabinets. Then (ideally) train a record-breaking multimodal model and (hopefully) take over the planet before they all become obsolete in 2-3 years after tinygrad puts out their ASIC design and Intel and SiFive adapt it for 2nm.

I guess to be truly turn-key it would need to come with a special model or two adept at training and commanding an army of LLM agents.

barelyusable• 3 years ago

I'd not mind frying some meats on a grill and celebrating, if anyone in Bay Area is interested. Having some Oxide folks would be awesome.

mimd2• 3 years ago

The listed items are for last gen CPUs and 2 gen old nics (100 Gb/s). Also zfs rather than ceph. It's hard to consider under those constraints.

Any plans for upgrades beyond single rack deployments and newer gen hardware? And what is the sell case over kube, mesos, or VMware, especially since the hardware and software appear tightly coupled?

wmf5• 3 years ago

It's Crucible not just ZFS BTW.

mimd2• 3 years ago

Thanks for the info.

sixdonuts2• 3 years ago

Congratulations on your release! Great product. This will totally fill our MANY on-premise CI, Security, analytics and Video needs that CANT be moved to the cloud. Oxide will definitely be on our short list for our next hardware/software stack refresh.

The first Oxide rack being prepared for customer shipment

Related Stories