Google Kubernetes clusters config checker tool

https://github.com/google/gke-policy-automation

by mikowhy • 4 years ago

127 33 4 years ago

github.com

game-of-throws• 4 years ago

A similar tool is Popeye. It works on any cluster, not just clusters hosted on GKE.

https://github.com/derailed/popeye

mikowhy5• 4 years ago

Our focus is on Google Cloud and Google Kubernetes Engine specifics but I love Popeye and K9S from derailed!

Bayart• 4 years ago

I've used Popeye because it comes shipped with k9s, which is the best k8s "dashboard" in my opinion. I really like it but it tends to bug out a bit on my terminal.

mikowhy5• 4 years ago

they put all the best to k9s: ncurses style of UI + vi style of key bindings

rpadovani• 4 years ago

Given that is specific for GCP, and developed by somebody within Google, why not including it directly in the webconsole?

nova22033• 4 years ago

This is not an officially supported Google product.

How does this work? It's developed by google but it's not officially supported?

vaughands• 4 years ago

It's likely someone's 20% project.

kyrra• 4 years ago

(googler, opinions are my own).

Google actually publishes most of the docs that Googler's have to follow when open sourcing software. See:

https://opensource.google/documentation/reference/releasing

This specific line you're asking about is talked about here: https://opensource.google/documentation/reference/releasing/...

mikowhy5• 4 years ago

The tool is developed by Google Cloud engineers working in professional services. We create such tools to quickly help our customers (and often ourselves) in some particular challenges. Sometimes such initiatives are merging with a core product. The disclaimer is there to say, that this tool is maintained by the community, not the official product support.

mikowhy5• 4 years ago

That is good point but also long way ahead. Sometimes such tools are merging with a core product. The tool and policy was created by Google engineers that are not part of GKE product development teams.

For now I suggest to use the tool in a scheduled, serverless manner and configure evaluation output to Security Command Center. By that, processes will be fully automated and the results will be visible in a webconsole (as findings in Security Command Center).

pdimitar3• 4 years ago

As many people predicted years ago (I can't claim to know k8s that well, in fact I suck at it), eventually we'll go full circle and k8s config will just become its own specialized programming language.

Maybe we should stop moving these things so agonizingly slow and through the path of natural (and did I mention slow as molasses) evolution and just skip to the endgame that most of us know will inevitably come?

Apparently not.

I'd like to see Google being more brave here. They are on the forefront of k8s in many ways (or so it seems, maybe I am wrong?). Just make a specialized programming language with a good compiler / linter and let's all collectively be better for it.

cyberpunk• 4 years ago

Why do you need a 'specialized programming language' when kube for most people, is just waiting for various objects via api?

I mean, sure, ATM it can be a bit painful to generate your 'Deployment' object in json or yaml or whatever, but then it's just a post to the kubeapi and it's done.. You can do that in any language you want..

What am I missing?

pdimitar3• 4 years ago

You're missing the part where people do this periodically and can never truly remember all the gotchas so they make small mistakes with time that at one point make your cluster fall over with cryptic error messages. I've seen it (but as I said above I am by no means a k8s pro).

That's why there are these linters.

Having a small super-specialized language that catches errors before "compiling" your configuration to an YAML will help hugely.

phailhaus• 4 years ago

There's no reason to create an entirely new language. Since k8s manifests can be represented in JSON, you can use existing templating languages like Jsonnet [1] in order to generate it for you. All we need is an official library.

[1] https://jsonnet.org/

ninkendo• 4 years ago

There was a tool called ksonnet for this (jsonnet + a default library for k8s configs), but the company behind it got bought and it's no longer maintained: https://github.com/ksonnet/ksonnet

vbezhenar• 4 years ago

JSON is terrible to write by hand. YAML is terrible too. There should be an official and sane language which will map to YAML 1:1. We don't write Java code in YAML for a reason.

solatic2• 4 years ago

https://github.com/google/gke-policy-automation/blob/main/gk...

What's the point of requiring the control plane to be locked down to authorized networks (IP address ranges)? Isn't Google responsible for DDoS protection, enforcing authentication controls (i.e. logging in with a Google account in the right Google group), patching the control plane ASAP for any security vulnerabilities?

If you have a VPN, if you have heavy-duty network monitoring on your VPN endpoint, sure, limit it to the VPN. For the rest of us? Is every startup running GKE without heavy-duty VPN / network monitoring fundamentally insecure? That doesn't sound right to me. Security is supposed to be a spectrum, and it seems like black-and-white automated config checkers like these are more likely to provoke arguments internally ("but the tool said it's bad!!") than to help reach a nuanced understanding of why tradeoffs are made. No?

no_circuit2• 4 years ago

Likely security in layers. Why expose your control plane to attacks directly from the internet if you don't have to? Cuts down login attempts noise in logs since anything would have to be coming from the VPC. Other than initial setup of a bastion -- that's the tradeoff -- sounds like less to worry about for a small shop or a startup. Same for Cloud SQL or any other managed service.

Ironlink• 4 years ago

I wish they would reuse the pattern of Cloud SQL where you can get temporary access without manually handling the Authorized Networks setting. The Cloud SQL API lets you exchange your API access token for a short lived TLS client certificate. This is done client side by things like cloudsql-proxy[1] and the cloud-sql-jdbc-socket-factory java library[2]. This way, I can access my Cloud SQL instance from my IDE, even though my list of authorized networks is empty.

I feel like the gke-gcloud-auth-plugin cloud do something very similar.

[1] https://github.com/GoogleCloudPlatform/cloudsql-proxy

[2] https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket...

solatic2• 4 years ago

But setting up a bastion has its own issues. Now I need to monitor and patch it regularly, which with a 100% Kubernetes shop means I need a completely separate setup for it. How do I protect my bastion against DDoS?

Managed VPN services have their own costs and worries. Don't want to accidentally pay for tunneled Netflix or YouTube? More configuration and maintenance.

What kind of attacks from the Internet need to be worried about here? Basically just zero-days? But that's true for any bastion or VPN as well, and again, Google is managing the control plane so they're empowered to roll out the patch faster than a small customer could.

_joel• 4 years ago

Because you don't need to, plus you're covered against 0days. If you're creating infra on GCP then I'm sure it's not too much of an effort to use Google's osn Cloud VPN or create a bastion interface to ssh/wireguard into (only needs to be very small).

mikowhy5• 4 years ago

The point is to limit the attack surface, following zero trust and least privilege principles. This particular rule follows also GKE product networking best practices [1]. However we understand that not all rules apply to all environments. That is why it is possible to exclude selected rules in a tool or provide set of own, custom policies. If Security Command Center is used, it is also possible to mute findings there.

References: [1] https://cloud.google.com/kubernetes-engine/docs/best-practic...

pid-1• 4 years ago

security = onion

sg1234• 4 years ago

Would you feel more comfortable if you had an in cluster agent that dial into a Google Managed API and receives command from it ? This way your control plane can be private and you don't have to handle Managed Authorize Networks.

zxspectrum1982• 4 years ago

Red Hat has a similar tool called Red Hat Insights Advisor for OpenShift, which they provide as a free service with OpenShift subscriptions, so you don't need to install anything:

What is new in Insights Advisor for Red Hat OpenShift https://www.redhat.com/en/blog/what-new-insights-advisor-ope...

https://console.redhat.com/openshift/insights/advisor/recomm...

Google Kubernetes clusters config checker tool

Related Stories