Kubernetes as a platform vs. Kubernetes as an API

https://aws.amazon.com/blogs/containers/kubernetes-as-a-platform-vs-kubernetes-as-an-api-2/

by mariuz • 3 years ago

66 45 3 years ago

aws.amazon.com

ianpurton• 3 years ago

What is kubernetes?

For me it's a platform to deploy apps without tying myself into a particular vendor.

spyremeown• 3 years ago

One of the things I hear the most about Kubernetes is the complexity. Honestly, it's not that complex from an user perspective. It's not much more complicated than Docker, really. It's also super nice to just write two yaml files and have everything running nicely with rollouts, auto-scaling, node pinning etc. It just works and I never have to even think about the internals (although I took the time to learn most of them).

Adding: heck, I'll run a single node k3s on my computer (which takes one (1) command) instead of using docker-compose just because I like it better than compose.

ninkendo• 3 years ago

Yeah most of the (IMO) legitimate complaints about the difficulty of k8s are about how hard it is to correctly set up and administer. (Networking setup, making ingress work, cloud service integration, etc.) Especially when you get into permissions/namespaces and trying to get that right for a decent size organization and trying to configure things for least privilege. Simply using it isn’t that hard IMO.

danpalmer2• 3 years ago

Kubernetes is complex if you don't need what it provides.

However if you do need what it provides in _some way_, Kubernetes provides a pretty nice, coherent way of doing it.

bravetraveler• 3 years ago

I find consuming K8s to be like this - it's great, provides facilities for common everyday things

I find building K8s (well) to be wildly different. CNIs, storage classes/providers, yada yada. For a 'solution', you have to build a lot of it.

It's in a weird spot to me, I like/enjoy it - but only because I have other people helping me tend to it

edit: Think of a typical 'cloud' -- it's typically made of compute, storage, and networking.

My 'thing' about K8s is it's hyper-focused on the compute part, touted as complete, but (understandably) lacking in terms of storage/networking OOTB

Being able to easily build (development) clusters perpetuates this a bit more than I care for. They let you ignore the parts of production-worthy K8s, worrying only about the workload.

MuffinFlavored• 3 years ago

Could you share those 2 YAML files? I feel like it's not hard to do k8s YAML the wrong way and miss out on the auto-rollout, auto-scaling, etc. features

ilyt2• 3 years ago

It moves most/all of the complexity on ops side. or in "whatever automation thingy you use to deploy it". Yes, easy on dev, but compared to "just docker" or "a VM with some systemd service running your app" it is hideously complex

mac-chaffee2• 3 years ago

Amazon publishing a blog that blatantly argues for vendor lock-in? Who would have guessed?

One of the big benefits of Kubernetes is the cloud-agnostic API for running pretty much any application. CustomResources can help blur that boundary and make it easier to use some managed services, but actually arguing in favor of that approach for everything is a pure advertising tactic.

It's no secret that managed services are several times more expensive than equivalent EC2 time. Sometimes it's worth it. But to throw Kubernetes into the mix just to run some controllers that deploy those services will pointlessly add to your monthly bill. Running any Kubernetes cluster is not cheap, even if it's just a bunch of controllers.

joshribakoff• 3 years ago

It’s arguably an extreme option and one that doesn’t even exist today

How is it blatantly arguing for this thing when it literally calls it extreme and something that doesn’t exist? I think a more fair criticism would be the fact they don’t call out vendor lock-in as a tradeoff

to throw Kubernetes into the mix just to run some controllers that deploy those services will pointlessly add to your monthly bill.

I think you have a valid point it may be dubious but the article does outline purported reasons it could make sense for some people. I dont think there is a “right way” and a “wrong way” here, rather there is the way “you would do it” and “the way they’re proposing as an option”

mac-chaffee2• 3 years ago

The author admits it's extreme, but is clearly trying to normalize it:

Someone defines Kubernetes as “a well-designed and extensible API with programmable reconciliation logic, that happens to come with a container orchestrator built in.” I like that definition.

cmeacham98• 3 years ago

Disclaimer: I work for AWS but had nothing to do with this blog post (I'm seeing it for the first time with everyone else here).

I think this is an unfair summary of the post. Of course, using Kubernetes to orchestrate other AWS services is going to be a go-to example on the _AWS_ blog, but there is plenty of vendor-agnostic software doing similar things: DNS Records[1], Databases[2], even using Kubernetes CRDs to deploy Kubernetes[3].

The idea of using Kubernetes as an API to orchestrate external resources doesn't inherently lock you into any single vendor.

1: https://github.com/kubernetes-sigs/external-dns 2: https://github.com/kubedb/operator 3: https://cluster-api.sigs.k8s.io/

vbezhenar• 3 years ago

I'm running self-managed kubernetes cluster.

Overhead is as follows (/month):

1 x load balancer: $20

3 x master servers: 4GB RAM, 2 CPU, 15 GB SSD: 3 * $20 = $60

also on every worker server around 15% of RAM is dedicated to kubernetes pods (kubelet). So slightly higher price for RAM and CPU.

So basically overhead is like $100/month. Or $50/month with one master server which is not very reliable but acceptable for small workloads. If your project runs on $5/month VPS, Kubernetes is pure overhead. If you're paying $500/month, Kubernetes is something to consider.

rektide• 3 years ago

I also ran a single instance k3s node on my personal $8/mo 3gb vps, and it took up around 400MB. Ran some workloads, a postgres operator. Ingreds via traefik on the node. I think my overhead was like $1.20/mo. Did I have fancy HA? No. But it was an easy way to manage workloads & learn skills. Works fine.

bryanlarsen• 3 years ago

You're paying 4x as much for your self-managed cluster as I am for a vendor managed cluster. I have a single-node digital ocean cluster without a load balancer.

steveBK1233• 3 years ago

Maybe I'm ignorant, but why would you pay $1200/year for the compute/storage/ram equivalent of a 5 year old iPhone?

dopylitty• 3 years ago

As I was reading this I thought it was calling for using the K8s API as an abstraction over all types of backend compute vendors. That would actually avoid vendor lock-in. Set up one control plane somewhere and use it to manage your AWS, Azure, GCP, on-prem VMWare etc.

That would mean you need the K8s API for your control plane but not necessarily that you have to run a K8s cluster or clusters as long as the API layer is the same. Maybe the control plane is a managed service too and you just interact with it.

I'm a bit skeptical of these single pane of glass type solutions since the abstractions always end up being somewhat leaky. The abstraction turtle stack is so high at this point you have to know 11 different technologies just to start a process on a Linux box.

But I get the operational appeal of having a central control plane rather than a million control planes all with their own AuthN/AuthZ, logging, custom YAML DSL, APIs, security configurations, integration points etc.

Chmouel• 3 years ago

I think that's the actual goal of kcp: https://github.com/kcp-dev/kcp

gladiatr72• 3 years ago

There's an old apache project: libcloud (python) that tried to generalize various cloud service apis. It was.. not good.

mreferre• 3 years ago

[ Disclaimer: Author of the blog here ] I knew this blog was very polarizing so this is a fair comment. I have called out advantages and disadvantages in the first scenario ("The pros of this approach is that the customer is decoupling heavily from the infrastructure underneath thus enabling better portability of their entire platform. The cons of doing this is that the customer is signing up for a high amount of work.") and, in retrospect, I should have called them out explicitly for the last scenario (I have just assumed readers could infer they would be the opposite). I did mention towards the end that "customers [that can benefit from the third model ] are coming from an Amazon ECS background, they are all in into Amazon ECS and AWS" but I take to heart the suggestion that I should have been more explicit. Thanks.

linuxftw• 3 years ago

There has been a lot of prior work in this space. There's crossplane, the Terraform Operator, and a ton of roll-your-own solutions for tackling this vary problem. Alibaba also was early to have the concept of a 'virtual node' where you can deploy seemingly infinite number of pods, they'll handle all the orchestration below.

I was reading the article thinking "Yes, finally, some code or use to implement interacting with AWS without a 3rd party lib" and then I got to this section at the bottom:

In this post, we showed you the flexibility of Kubernetes. It’s arguably an extreme option and one that doesn’t even exist today (the Amazon ECS ACK controller isn’t available but please thumb up this roadmap proposal if you are intrigued). If anything, it proves that Kubernetes can be a lot of things to a lot of different people and defining it remains a work in progress (at least in my head).

As usual, Amazon is playing catch-up in actually orchestrating their services in a non-AWS-centric way. Very deceptive to create a blog post and not disclose it's not real software at the end.

coredog64• 3 years ago

The ACK exists and allows management of AWS services. What doesn’t exist is the ECS-specific ACK.

IceWreck• 3 years ago

The last one (Kubernetes is my control plane but not my data plane) is completely stupid. Instead of that why not use terraform or equivalents ?

Paying for EKS just to use its API to provision and manage other AWS resources is an interesting idea, but not something I'd use in production.

jhoelzel4• 3 years ago

I love how conveniently on-premise kubernetes is missing. Or self-managed in the cloud.

The whole point of kubernetes is to not think about your servers anymore because they in essence can be replaced at any time. This would only provide absolute vendor lock-in.

Of course you can use any external service there is (like a database) but you should never put all your eggs into one basked and trust only one hoster. The principle remains the same: its just someone elses computer.

Storage and networking are solved in a lot of places at a fraction of the cost and that is really just the beginning., but also the whole point of the exercise: to be able to move your workloads from wherever to wherever

awill3• 3 years ago

>The whole point of kubernetes is to not think about your servers anymore because they in essence can be replaced at any time.

Isn't that also the case for VMs?

pixl97• 3 years ago

How are you orchestrating the configuration of your VMs?

Kubernetes as a platform vs. Kubernetes as an API

Related Stories