Obscure challenges supporting Kubernetes 1.22 in Pixie

https://blog.px.dev/k8s-122-challenge/

by htroisi • 5 years ago

15 6 5 years ago

blog.px.dev

I’ve been encountering similar challenges dealing with old Kubernetes library versions, CRDs, etc. Is this a problem mainly with the Kubernetes project or just Go dependencies? It’s quite frustrating, and I always discover some new set of voodoo to handle it.

philkuz• 5 years ago

I'm not quite sure the flavor of your problems, but I imagine some of them were caused be the confusion between v1beta1 and v1 apis. Users would continue to use beta versions of APIs despite the availability of the GA APIs.

Hopefully this becomes easier with the beta lifecycle policy: https://kubernetes.io/blog/2020/08/21/moving-forward-from-be...

Unfortunately this also means you'll see similar issues in the short term.

One way to avoid these might be to opt for only GA k8s APIs for your infra.

dilyevsky2• 5 years ago

Fluffy operators like etcd-operator that do nothing but create some pods for you is incredibly overused concept in k8s ecosphere. I’m convinced most of these projects are spun up for resume padding of the creators. The fact they were able to replicate them using built in k8s controller is another proof of that

kobalsky• 5 years ago

coreos archived etcd-operator

doesn't surprise me.

this operator really tarnished coreos in my eyes when I learned that the it worked only with volatile storage, so if for example, your power backup failed and every node went down, you lost your all your etcd data until your last manual backup.

it felt like it was a half-assed attempt of a proof of concept and it was somehow being used in production systems by a lot of people, including the guys in the article.

however, and to be fair, I don't know if it evolved much after our experience, although the fact that it was scrapped doesn't bode well.

luckily it didn't resist much testing so we didn't waste much time with it.

dilyevsky2• 5 years ago

it felt like it was a half-assed attempt

So like 90% of k8s operators then?

zzyzxd• 5 years ago

The article described their issues related to using deprecated software (etcd-operator and nats-operator, both archived by upstreams), which rely on some k8s beta APIs that deprecated 2 years ago. The result shouldn't be surprising at all.

Unfortunately, both operators chose to define their CRD inside of their code (rather than as a separate yaml) and the latest releases still used the beta version of the API.

They made it sound like a bad thing, but I don't think it is necessarily true in all cases. It is convenient to define CRDs outside the operator code, but just like putting ORMs in your CRUD app, embedding CRDs in code gives developers better chance to ensure that their code is compatible with a specific API version. If I say my code only supports v1 API, I can't help you if you run it with v2 API and it gets broken.

Obscure challenges supporting Kubernetes 1.22 in Pixie

Related Stories