Hetzner GPU Server

https://robot.hetzner.com/order/index/culture/en_GB

249 156 2 years ago

robot.hetzner.com

ecmascript5• 2 years ago

Not really related to their GPU offering but Hetzner is such a great company. I have a dedicated server I rented where I host all of my side projects. For the same price as one shitty VPS I get 8 cores, 64GB ram and a disk of 240GB or something like that.

Just incredible, response times are incredibly low and there is no cap on data. The few times I have contacted support they've been great. The only times I have experienced downtime is when I have stopped or rebooted the servers.

Can't recommend enough, especially if you are inside the EU and care about data privacy. The fact that they're a european company is of great value to me.

If you are new on server provisioning and want to learn how to setup a new server, I wrote a small guide for how to do it for node-projects (works for pretty much any web app if you replace the node.js stuff): https://deployjs.com/ (hosted on that server)

gokhan2• 2 years ago

Had a server with them for more than 10 years. Had they eliminated the setup fee, they would get more of my money for sure for small projects.

ecmascript5• 2 years ago

The setup fee is a one-time fee though, so if you have a server for many years that won't hurt so much. I pay around €30/month for the server I am renting but I know it will last a long time even if I get spikes on any on my many projects. For a 8 core, 16GB droplet/vps at DigitalOcean it will cost you ~$96/month. The performance is better at Hetzner for about 1/3 the cost.

sshagent• 2 years ago

They often have a couple of servers with no setup.

Citizen_Lame• 2 years ago

Ah yes, they are great if you messing about. Not so great if you have crucial business server with them.

ecmascript5• 2 years ago

why not?

rapsey• 2 years ago

We've been running stuff on their platform for years with very little problems. Why not?

withinboredom• 2 years ago

I haven't run into a single issue for years, what do you mean by that?

elric2• 2 years ago

Care to elaborate on that?

I've been using their services for some 20 years at various scales, and the only complaints I've had are that their shared mailservers occasionally get blacklisted.

That, and the UI of their KonsoleH could do with a bit of a refresh.

happylion0801• 2 years ago

Signed up from an Asian country and during signup they asked me to upload a passport copy page.

I thought that was weird (I mean no other cloud provider asked for it but okay). I uploaded anyway because I thought it’s in EU so they at least have better privacy laws so they may end up deleting it later from their systems but I immediately got rejected. Not sure why no reason specified. So now they have my passport copy page without me having an account.

Oh well, seems like either they have an overly aggressive spam filter that gets false positives or they don’t really like accepting signups from Asia

lakomen2• 2 years ago

Hetzner is pretty hostile towards foreigners.

And don't you dare suggest in their forum that foreign tech is better than German, instaban.

Fuck Hetzner

deltaknight• 2 years ago

I had a similar experience very recently (albeit within Europe). Did you try their manual verification process? It was available to me after two failed automatic attempts, and it basically had me send an email to Hetzner support asking if they needed any more information. I got approved manually the very next morning.

Maybe not the most customer-friendly experience, but I figure half an hour of signup weirdness is okay for a solid after-signup service in the long term, especially as that half an hour is likely to deter a decent chunk of fraud.

lofties• 2 years ago

Had the same experience. Contacted support for manual verification, was online the next day with Hetzner. Machine has been running without interruption for a year now.

EVa5I7bHFq9mnYK4• 2 years ago

Same story from the US. I think they mostly worry that you are trying to avoid paying VAT.

pmontra3• 2 years ago

Which server do you have on Hetzner that I missed? Dedicated servers start at 37.30 Euro per month now there, the AX-41 couch is pretty good and has no setup fee. On the other side I'm paying OVH about 12 Euro per month for two small VPSes and they are even too much for me, performance wise.

e12e2• 2 years ago

Have a look at the server auction? Tbh, I would limit my dedicated server rentals to server/ddr ram setups (which is a little more expensive).

But if you just need some cpu (to transcode some video, crack some passwords, what-have-you, run a game/videochat server) - consumer cpu models give you unmetered gigabit or 30tb/month 10gbps. So can definitely be worth it vs a VPS.

ecmascript5• 2 years ago

I have a dedicated server that I got on their auction page for 30 euro per month. That gives me 64GB of ram, 8 cores and some disk space.

Sure, you can get a cheaper VPS, but given a VPS with the approx same perf it would cost much more.

thatwasunusual• 2 years ago

Ditto.

Been using them since 2012 or something, and have never had a problem (that wasn't my own fault).

V__• 2 years ago

Note: For now for existing Hetzner customers only.

greatNespresso• 2 years ago

Is Nvidia RTX 4000 solid for training? Alternatively this card seems listed for around $1800 so at $180/month on Hetzner, it may be even cheaper to buy it, depending on the expected usage.

Ayesh• 2 years ago

It's with ample RAM and the latest Intel CPU. Assuming the whole thing draws 100W, that's also including 72kWh of electricity.

bluedino2• 2 years ago

I'm not sure what the exact 'rule' is here, but you wouldn't use a contraction in this sentence.

It is, with ample RAM and the latest CPU.

Aerroon2• 2 years ago

Why does this GPU cost so much? It seems to be like an RTX 3060 Ti/3070 with more VRAM and less power draw. Does that really justify the price tag?

kookamamie3• 2 years ago

Because it's "enterprise".

HPsquared• 2 years ago

Like Apple, they use RAM for market segmentation.

petercooper5• 2 years ago

NVIDIA's licensing. Note that the 3060/3070 is a "GeForce" whereas a 4000 is a "Quadro". Then look at the GeForce driver EULA: https://www.nvidia.com/content/DriverDownloads/licence.php?l...

No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.

I could never figure out why they allowed blockchain processing, given how wasteful much of it was and how it impacted card supply for years, but there you go.

stefan_• 2 years ago

Datacenter customers don't have options, gamers do.

rmbyrro• 2 years ago

You have to discount the price of a similar server without the GPU, tough.

But even so, it's still quite expensive. I have the impression Nvidia charges more from data center customers. So, for cards available to retail consumers, it's not a fair cost comparison.

With only 20GB, RTX 4000 has limited use... I was expecting they'd be offering something with at least 60GB at this point.

kookamamie3• 2 years ago

No, not a great GPU for training. Consider something like A5000, A6000 or their Ada variants - if you want to go full "data-center", A40, L40, L40S are solid picks for small-scale training.

brucethemoose2• 2 years ago

Are you going to be training constantly?

TBH renting a big GPU (A100 or better) and getting the run over with quick is usually best. VRAM is everything.

christkv• 2 years ago

How is the gpu for inference or tuning ?

SushiHippie2• 2 years ago

FWIW, on the server auction, you could get GPU servers since a while:

https://www.hetzner.com/sb/#additional=GPU

Looks like currently there are only GTX 1080s on the server auction, I don't know if they normally have other GPUs available as well.

alberth• 2 years ago

Nvidia RTX 4000 SFF Ada

From the OP link, the GPU-Server GEX44 server line includes RTX 4000.

And the server only cost €184/month.

SushiHippie2• 2 years ago

I meant if they have other GPUs normally available in the server auction. As far as I understand the servers in the server auction are mostly custom built ones for customers and once they stop renting them they land on the server auction for others to rent.

The one with the RTX 4000 seems to be a standardized line of products.

aborsy• 2 years ago

The price of a brand new Dell precision PC with a core i9 13900 CPU (24 cores), 64 GB RAM and 2X2TB NVMe is probably around 2.5k$. It costs tens of Euros per month in electricity to run this PC. The hardware is good to go for some 3 years, before running into various issues (newer CPUs becoming much more power efficient, firmware updates getting less and less frequent etc). Consider also the cost of components failure, time wasted in dealing with hardware, etc.

On hetzner it all costs less than 100$ per month. Doesn’t this imply that it’s better to rent than to buy?

throwaway114603• 2 years ago

Yes, the volume discounts and economies of scale they can get (while you can't) are insane.

singhrac2• 2 years ago

One of the simplest is that data center energy pricing is just simply different from residential energy, because they often get wholesale (plus small markup) rather than residential rates.

schroeding• 2 years ago

If you pay European (German) energy prices, this very well may be the case, in my experience.

For some Hetzner servers (especially from their "Server Auctions"), for domestic customers the cost of electricity alone, disregarding the hardware cost, would sometimes be higher than the rent Hetzner wants.

elric2• 2 years ago

They build their own servers using some custom components. Economies of scale probably don't really factor into the pricing of their GPU stuff (unless I'm seriously underestimating their GPU customer base), but they do factor into the peripheral stuff like PSUs, fans, hard drives, etc.

jsheard• 2 years ago

There's some videos out there showing off their custom motherboards, it's an odd mixture of standard-ish consumer ATX but stripped down to the absolute bare essentials and with some components rearranged for better airflow in racks chassis they also build themselves: https://youtube.com/watch?v=V2P8mjWRqpk

Another video showing the custom racks and other infrastructure: https://www.youtube.com/watch?v=5eo8nz_niiM

wongarsu2• 2 years ago

Hetzner is not only able to get better volume discounts, they also spend a lot of engineering time bringing down costs, and have decades of experience doing so.

They offer some Dell Servers for those that really want them, but most of their servers have a custom mix of consumer hardware, server hardware and in-house hardware (for example they use their own racking system), optimized to minimize lifetime cost in a datacenter. For example most servers use datacenter SSDs, but consumer CPUs.

dist-epoch• 2 years ago

Hetzner is buying ASUS consumer AM5 motherboards, but with the whole CPU + memory block rotated so that the memory sticks are horizontal instead of vertical for better airflow:

https://www.youtube.com/watch?v=V2P8mjWRqpk

cobertos• 2 years ago

Do you have more details on their own racking system? I couldn't find anything on this.

YetAnotherNick2• 2 years ago

It's definitely weird that cloud costs which is supposed to be commoditized is differing by 2 orders of magnitude. Hetzner/OVH seemed to have solved the hard part of making the service cheaper but just can't solve the easier task of making the platform usable. Why do I need to upload documents for verification. Why can't I try server for 2 hours right after signing up for the service.

mike_hearn• 2 years ago

Those are part of making it cheaper. ID verification is an anti-fraud measure and ensures that bans stick, which in turn means they don't have to invest so heavily in heuristic anti-fraud and "spam filtering" type work. A delay on provisioning means they are provisioning hardware JIT in some cases, meaning they need less idle float capacity, which in turn drives down costs.

Scotrix2• 2 years ago

Have been in deep contact with data centre and rental server offerings in the past, there is a crazy large amount of fraud going on, e.g. 2 hours for free means 2 hours free resource to DDOS.

delfinom2• 2 years ago

You can thank crypto for ruining the internet for us.

EVa5I7bHFq9mnYK4• 2 years ago

Sure you can. I rented a server and after two weeks decided I don't like it, and got refunded all the money, including the setup fee.

brudgers2• 2 years ago

Doesn’t this imply that it’s better to rent than to buy?

TANSTAAFL

1. Physical hardware is a useful abstraction.

2. At the end of a three year cycle, you can sell something you own. (This has associated costs of course).

3. Rents can go up. Terms and Conditions can change. Credit cards can expire or be cancelled. In other words, renting introduces a significant dependency.

4. You are your own most important customer. Statistically, you are not The Clouds's most important customer. When The Clouds has a fault, it gets resolved based on its business model.

Engineering decisions are specific to a specific problem and all of them come with tradeoffs.

Dalewyn• 2 years ago

2. At the end of a three year cycle, you can sell something you own. (This has associated costs of course).

This is huge and often overlooked or undervalued. Renting means you have nothing when the contract ends, buying means you have something for your money spent.

k8sToGo2• 2 years ago

You forgot that Hetzner also has spare parts.

chrisweekly• 2 years ago

TANSTAAFL

"There ain't no such thing as a free lunch"

(I had to look it up, figure I'll save someone else yhe trouble)

tiffanyh• 2 years ago

Yes.

This is what you expect prices to be when a vendor is buying in bulk.

Cloud vendors should be like "Costco" (buy servers in bulk and can pass along those savings & this is what Hetzner does)

The trick AWS (and other cloud vendors have done), is charge "Uber Eats" pricing for Costco items.

fragmede• 2 years ago

That's a really clever analogy, but is it really a trick? UberEats delivers the food to my door but if I go to Costco I have to make my dinner myself. Sometimes life happens and UberEats isn't just a convenience thing, it's the only way I'll manage to feed me and my crew. I could run a pubsub queuing system and a database myself, but if I don't have to do that, it frees me up to focus on the tasks I'm really trying to accomplish instead.

whizzter2• 2 years ago

It seems a bit like car leasing, providers gets a bit of discounts for their volume purchases and customers pays a premium (that is still affordable) compared to just taking the entire deprecation cost themselves. The provider then still has something worth a fair bit of cash that can still generate income (through resale).

So if you look at a 2 year horizon and want to keep cutting edge (or prefer operating expenditures to capital expenditures) then it's cheaper whereas looking at a 5 year horizon the capital expenditure will pay off.

Hetzner is very bare-bones compared to regular clouds (a friend who was after performance but wasn't prepared for this ran into some issues when a disk died), so you need some procedures for backup/replication in place but if your procedures are in place then you can save big compared to the regular clouds.

chpatrick• 2 years ago

There's definitely a tradeoff when it comes to maintenance effort. You need to make sure your RAID and monitoring works and if a disk fails you need to message support to replace it. In general I've had a great experience with them.

treffer• 2 years ago

Last time I checked the GCP prices vs. hetzner servers there was a factor of 10 in monthly cost. So I cant say this is a general rule of thumb. It is true though that hetzner can provide superb performance per $. Potentially below what you can do (unless you buy racks of hardware).

Also keep in mind that hetzner is mostly a beowulf cluster. See e.g. this press picture from their newsroom page: https://cdn.hetzner.com/assets/Uploads/IMG-0546-91.jpg

They do have dell servers, too. But don't expect to run on one of these nodes unless the machine type has Dell / PowerEdge in their name. This means the management capabilities are minimal (but usually enough). A fully licensed management card can do way more.

Their renting pricing is ridiculously cheap though. Hardware failure is more common than on cloud providers (see cluster hardware), support has been helpful in the past. Overall IMHO a good choice if you are price sensitive. But even then consider development vs. deployment vs. running (server) costs. Servers might make up less of the total cost of a service than you expect.

rthnbgrredf• 2 years ago

To be fair, companies that merely launch VMs on GCP constitute a small fraction. GCP truly excels when you leverage its object storage, BigQuery, managed postgres database (which starts at 7$/month), and serverless solutions for cloud-native applications. Our company operates around 500 services, with billing per second, and a significant number of them are scaled down to zero when not in use. If you need a GPU for a batch processing task involving 10,000 images, you can simply activate a VM equipped with a high-performance GPU for an hour, pay for that duration, and then shut it down. At Hetzner, you're required to pay for a whole month upfront, regardless of whether you need the GPU for just an hour each day.

Therefore, I'd argue that if you require continuous, raw computing power, Hetzner is indeed cost-effective. However, the cost-effectiveness elsewhere really hinges on your usage patterns.

rstuart4133• 2 years ago

On purchase price alone you are usually better off not renting. Given a server generally comes with 5 years warranty, you are about 50% better off on those figures.

However, the killer isn't the purchase price. You have to put it somewhere with a lot of bandwidth, UPS, generators and cooling. To use your Hetzner example, you can't get co-lo for less than $119. That's 14U, which is more than you need for one server of course, but then there are usually charges on top of that.

So yes, for small installations it's hard to beat renting. You need to be using several racks for the equation to tilt towards purchase.

petercooper5• 2 years ago

I've just ordered one to ditz around with. We're early in the process of deploying our own internal server for Mixtral work, but it'll be interesting to see how this performs (in raw terms, almost certainly worse, since we can roll out a 4090 without getting into trouble – but Hetzner has a better connection and handles the maintenance, so..) Order has been accepted but not deployed yet. I'll update when I have some initial inference numbers.

petercooper5• 2 years ago

I got it after a few hours. You need to install drivers/CUDA yourself, but all very straightforward. Unfortunately due to having 20GB of VRAM, I'm limited to mixtral:8x7b-instruct-v0.1-q2_K but it runs fine, generating at about 40 tokens/s (65 tok/s for eval). As per official specs, it's running maxed out at 70W (being an SFF card).

(I've now tried running the Q4 mixtral which is 26GB. 18GB is on GPU, 8GB through CPU. Gets about 11 tok/s.)

petercooper5• 2 years ago

To compare, running Ollama with Q4 Mixtral on my 24GB 3090 Ti locally (with 22.5GB of the 26GB model on the GPU) I get 14 tok/s generation, so the Hetzner server and the 4000 really aren't bad at all (notably my 3090 draws 70W during inference also).

syntaxing• 2 years ago

I never deployed it professionally but wouldn’t getting a Mac Studio be a better run for your money if you’re only inferencing?

petercooper5• 2 years ago

Possibly. It's not a bad idea! The performance is pretty good in our initial tests, though cloud platforms do scale further in terms of concurrency.

Given how quickly things are changing, using more generic hardware (i.e. PCs and Nvidia cards) or leasing cloud based services might make more business sense in the long term, so investigating fallbacks with more control, like this server, are worthwhile experiments anyway.

Further, while I've been an Apple superfan for a couple of decades now, their recent attitude towards developers stinks, and I'm loathe to give them any more money than I need to right now, so I might be migrating away from the platform in any case.

gregoriol• 2 years ago

It's a good price at ~0.30$/hour comparing to Paperspace's RTX 4000 with 30GB RAM at ~0.56$/hour.

Are there many other providers for such environments?

pacohernandezg• 2 years ago

vast.ai, runpod.io, tensordock.com, ... I have just ordered a GEX44 from Hetzner. :-)

edgoode• 2 years ago

If you want to check them all out in one place, we built a platform for that at https://shadeform.ai

binarymax• 2 years ago

The GPU product page: https://www.nvidia.com/en-us/design-visualization/rtx-4000-s...

20GB RAM, can fit 13B param models, and maybe some quantized larger models.

ramesh31• 2 years ago

Does every European tech company have some kind of committe that ensures their customer facing website looks like a decrepit ASP page from 2004?

I mean what explains this? It's a billion dollar corporation and I am having serious reservations about whether I would even enter my credit card info on this site.

lakomen2• 2 years ago

It's true. And it breaks on mobile when you use the menu.

delfinom2• 2 years ago

Hetzner's site looks fine to me. Highly readable and to the point unlike modern web shit where you have to scroll through 5 miles of marketing animatinos.

rsynnott• 2 years ago

I mean, see also AWS; for a very long time their console looked like something from the dark ages. I think they have largely spruced it up now, though I'm not sure it helped _usability_ too much, and may have actually made it worse.

system2• 2 years ago

I see the same thing on HN posts. 1999 vibes everywhere.

Shorel2• 2 years ago

I was going to seriously disagree with you, and then I looked at their source code.

It's awful.

I would still buy from them because their reputation trumps any concerns.

hipadev234• 2 years ago

Just make sure any workloads you place on Hetzner are treated as spot instances and outputs are immediately sent off network. The reliability of their systems and customer support is very low. Hetzner is a 5-star or 1-star company. Are you willing to risk your business on the flip of a coin? My own experience [2] was definitely a 1-star.

[1] https://www.trustpilot.com/review/hetzner.com

[2] https://news.ycombinator.com/item?id=39075608

tomschwiha• 2 years ago

I've been with Hetzner for around 10 years and didn't had a single larger incident so far. Support also answered promptly - didn't had the need to request a lot support so far. Running 2-3 servers and a few cloud instances.

gizmo• 2 years ago

Reliability and customer support are unbelievably good. 99.995% uptime for the past 10 years. Hetzner will happily customize your servers however you like, and at their low prices you can have hot standbys for everything and still save money.

whizzter2• 2 years ago

My friend went onto Hetzner for the performance/price, unfortunately the machines drive died within 2 months iirc. Seems like if you're buying on auction you'll get a wiped drive and if you're unlucky it's been written to a lot already, don't remember the specifics but he really wasn't a happy customer afterwards.

I think the last sentence you have there is pertient though, maybe don't go for Hetzner unless you can also afford to have a hot-spare (but for the performance/price you probably can compared to any public cloud). In my friends case he was boot-strapping but needed the CPU so he was really penny pinching.

dr_faustus• 2 years ago

Trustpilot is a scam which encourages bad reviews to force companies to buy their paid plan to “manage” those bad reviews and to feed it good reviews.

hipadev234• 2 years ago

Hetzner is also a fraudulent operation with zero support in my personal experience. But as you can see in this thread alone, any negative feedback is magically silenced. It sucks. Maybe we go back to webhostingtalk?

dmaa2• 2 years ago

It looks like a good value for money, but immediately after registering, I get this reply

"After reviewing your updated customer information, we have decided to deactivate your account because of some concerns we have regarding this information. Therefore, we have cancelled all your existing products and orders with us."

Someone has a similar experience? I've tried it twice, filled everything truthfuly, valid credit cards etc. and no still the same.

stanislavb• 2 years ago

I haven’t faced this issue myself, however, lots of people from India and some other countries have this experience.

I’d assume it’s something to do with collecting recurring CC payments from Indian cards. That’s inherently difficult.

dmaa2• 2 years ago

That would also be my guess, but I live in western Europe.

breakingcups• 2 years ago

There's a manual verification process if you contact support. Obnoxious that it's needed, but works for many people.

belk• 2 years ago

I got this when I first registered, contacted support and it was lifted in a day or two.

lobito14• 2 years ago

Hetzner has one of the worst customer support I've ever seen. Get really, really, faraway from this company.

nubinetwork• 2 years ago

After the exchange rate, that's $270 CAD a month... after a few months, you might as well just buy a raptorlake and run it locally.

zymhan• 2 years ago

Welcome to Cloud Computing.

throwaway815232• 2 years ago

It's a GPU system, not just raptor lake.

pinetroey• 2 years ago

A few months ago I tried to register an account, but I was denied.

I don't know why, never got a response...

I went with ovh.

ThinkBeat• 2 years ago

I need some help:

I have no idea where the NVIDIA RTX 4000 SFF belongs in the hierarchy of speed and suitability for "AI" computing.

How does this compare to a NVIDIA A800 40G? (if that is still considered good, I can't keep up)

To me the RTX part of the name sounds more on the consumer side but I am probably wrong about that as well.

renewiltord• 2 years ago

They had a strange setup process. I had to scan my passport and stuff. However, I did have a VPS with them that had uptime greater than a decade and that I killed only because the underlying hardware was going away. That was some 7 years ago or so.

kristianp• 2 years ago

It's running an RTX 4000 SFF Ada, 20GB GDDR6. Interesting that they've gone with a single-slot, low power card. Doesn't pull enough power to mess with their cooling/power design, I guess. Not a lot of graphics RAM to do much with LLMs.

That card is about $1500: https://www.amazon.com/dp/B0C2KMXQYG

yread• 2 years ago

It's a pity you can't commit to 3 year rental for a discount

Hetzner_OL• 2 years ago

Hi there, Thanks for helping to spread the news about our new GPU server, the GEX44. Please keep in mind that it’s not like we don’t want you to open an account – it’s more about safety & security. In general, we take extra precautions when it comes to verifying accounts and accepting new customers, as in these cases the potential of abuse of our ToC are higher. As a reliable hosting partner, we always take the time to listen to our customers and will do our best to find a solution for each case. So feel free to reach out to us if you need further assistance. --Katie

thijsvandien• 2 years ago

A reliable partner does not close accounts without any kind of recourse. Denying that under the guise of fraud prevention is just security through obscurity. I hope these practices become illegal sooner rather than later. Never before have I been treated as a criminal as much for trying to do legitimate business, and it’s quite the experience.

gn4d• 2 years ago

Do we get a discount for your MITM?

Marketing still doing damage control, I see.

Hetzner GPU Server

Related Stories