Nomad can do everything that K8s can

https://mrkaran.dev/posts/nomad-k8s-showdown/

by madmax108 • 3 years ago

36 19 3 years ago

mrkaran.dev

fesc• 3 years ago

I stopped reading when the solution for DNS was AWS and the solution for stateful sets was AWS EC2.

This is just a very weird comparison. Seems like the author doesn’t understand why people run K8s.

Of course they also “managed” several k8s clusters using… wait for it… AWS.

I’d say, when you are all in AWS, then fine don’t use k8s. But you also don’t need Nomad then in most cases.

And if you do need nomad on AWS then that’s fine as well, but it’s not comparable to k8s in general.

OJFord2• 3 years ago

I think that was confusingly written, but what it means is 'k8s recommends not using it for database, but in Nomad is fine' - the EC2 instances are just nodes like any other, but 'tagged for db'.

And if self-hosting Nomad on AWS is (allegedly) so much easier than operating managed k8s (EKS), isn't that more damning rather than less?

The all-in AWS answer I suppose would be ECS. I quite like it but I think for anything beyond small, more than a few services part of the same system, I can understand wanting to run Nomad (or use EKS) instead.

GauntletWizard2• 3 years ago

You can run databases that way in K8s as well. Nobody does, because it's a terrible abstraction for high availability of conventional databases. The operator pattern does the opposite of help, creating more moving parts.

The problem with running databases yourself is that it's hard to run databases, and having dedicated physical hardware works, until it doesn't. Nomad make it easy to run dedicated physical hardware for it. As soon as you have a failed EBS volume with your DB on it. Failed EBS volumes still happen frequently, though we've mostly obviated the problem by using managed dbs.

ranting-moth2• 3 years ago

To whoever wrote this, I have this unsolicited advice: being obnoxious about your competitor is very off-putting to the reader.

Putting down your competitor is a very fine art if you don't want to come across as a douche. Done correctly it can be hilarious though. But this is just a bull in the china shop.

capableweb• 3 years ago

being obnoxious about your competitor

As far as I can tell, the author of the article doesn't work for Hashicorp (Nomad) nor Google (Kubernetes) so not sure why you're giving this unsolicited advice?

ranting-moth2• 3 years ago

That's OK.

edude032• 3 years ago

I’ve always hated this style of argument in tech where someone says X can do what Y does as an argument against using Y (or for using X) because it always missed *why* people use Y in the first place.

See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code)

Or you can do machine learning in JavaScript so python is useless (you can but it’s a much smaller ecosystem)

Etc etc.

You can pick whatever tools you want for whatever reason but please give me a better reason than it *can* be done.

worksonmine• 3 years ago

See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code)

Isn't it usually the other way around? "This would be safer if written in rust".

Or you can do machine learning in JavaScript so python is useless

Said no one ever.

The author also elaborates why (according to him) X is preferable over Y (mainly simplicity), and even if he didn't there's value in comparing tools if one is on the fence of switching but afraid of loosing compatibility. Entire websites exist with that sole purpose.

edude032• 3 years ago

> Or you can do machine learning in JavaScript so python is useless
Said no one ever.

I want to reply to this point because I have some relevant & interesting experience here. I've done a fair amount of consulting for companies who have a scenario that goes like this:

- founder starts business in their free time, in a language they know well, as a monolith.

- founder gets a few customers and decides to scale the business

- founder decides they need to use "AI" to solve some business problem

- founder hires a person who is great at solving that particular problem.

- that person wants to use python (of course they do) but the monolith is not in python so various tensions ensue.

- the founder (or someone else on the engineer team) inevitably says "you can do ML in $LANGUAGE_WE_CHOSE, why use python? We didn't chose python because $REASONS_PYTHON_SUCKS.

You'd be surprised how often the language they chose is javascript and how often the reason not to use python is that they can't share code between the client and server. So unfortunately that example wasn't pulled out of nowhere

sharts• 3 years ago

To be fair, it seems a great number of folks use K8s because they can and/or because Google.

And if that criteria is good enough to justify its use then surely it's enough to justify the use of an alternative.

Frankly, the majority of software houses running K8s really don't need to.

Semenoto2• 3 years ago

Companies run k8s because it abstracts away infra.

People like k8s because it's fully open source and community driven.

pseudocomposer2• 3 years ago

I use K8s because if I don’t like what my deployment costs or how my cluster is supported, I can easily pick it up and take it to another provider or even host it myself (though I don’t really want to self-host).

nunez• 3 years ago

I wish more places ran Nomad. I haven't used it myself but it seems so much simpler than Kubernetes (and, like the author, I have extensive experience operating Kubernetes clusters) and HCL is a dream compared to drowning in OpenAPI docs and Golang structs to figure out how the hell to define the MySuperCoolCR resource that x.y.x-k8s.example.com/v1beta4 offers.

McSwag• 3 years ago

I’ve been using it in production for many years and I agree that it’s vastly simpler than kubernetes, by many orders of magnitude. I’m currently migrating off of it because I reached a point where I needed more enterprise-y features for not a ton of money such as rbac, federation, multi-tenant isolation, resource quotas, and less-often mentioned, hireability (it’s easier to hire for since more people have had exposure to it). HashiCorps enterprise products are also notoriously expensive.

Kubernetes has a lot of merits and open source community support which is attractive which means out of the box integrations are vast.

That said, Nomad builds in a lot of sensible default behaviors that are way way closer to production ready with significantly less effort. Many things in Nomad, with Consul and Vault “just work” like you’d expect. Some of it seems like magic.

Kubernetes on the other hand does almost nothing for you to make your custom app production ready out of the box. This means there’s a ton of room for error and incorrectly configured applications. CNIs, Ingress, PodAffinity, PodDisruptionBudgets, Services, HPAs, ServiceRoles, Secrets management are not defined for you and must be exactly correct or orchestrated in clunky and obscure ways, or a ton of effort needs to go into filling these gaps. “Just use EKS/GKE” quickly becomes a list of subsequent decision trees and you need to hope yoy made the right decision because if you don’t your choice will be deprecated in 6-9 months.

The two config options for Kubernetes are Helm/Kustomize Yaml. Both are great for quick POCs but at production scale are completely inferior to HCL and generally an absolute nightmare to work with on the day-to-day.

If I’m a team of one in a startup, I’d choose Nomad.

Nomad can do everything that K8s can

Related Stories