Spinkube: Running WASM in Kubernetes

https://www.fermyon.com/blog/introducing-spinkube-fermyon-platform-for-k8s

by ridruejo • 2 years ago

106 44 2 years ago

matei_radu3• 2 years ago

Hi, everyone! I'm one of the maintainers, I'd love to chat about the project and about how and why we built it.

whalesalad2• 2 years ago

Why is anything special or unique required to run WASM on kube? A container is a container, no? Serious question. To me the benefit of kube is that the api is a container and allows for any container to run there.

eptcyka2• 2 years ago

With WASM, you get better isolation than a regular virtual machine, you can be more granular with scheduling and the attack surface is far smaller than a regular VM. When compared to namespaces containers, you don’t need to rely on the kernel attack surface being tight for security. And you get to intercept all syacalls ala gvisor with less complexity. The downside is interaction with specialty hardware and performance.

triplechill2• 2 years ago

Hey SpinKube maintainer here. You're right that you could just run your Wasm (or Spin app) directly in a container and there often are reasons you might want to do that. But, SpinKube executes WebAssembly using a containerd-shim which means we avoid the overhead of starting a container and rather just directly execute the Wasm.

https://www.spinkube.dev/blog/2024/03/13/introducing-spinkub... is a really good blog post on Spin that talks about this.

matei_radu3• 2 years ago

Hey, one of the SpinKube maintainers here!

You are correct that there is nothing extra required if what you want is to run Wasm inside a container.

However, that comes with a few disadvantages, primarily stemming from bundling your wasm runtime with each container image — and while your wasm workload is portable, the runtime itself needs to be compiled for the specific OS and architecture you need to execute on.

SpinKube comes with a containerd shim (or plugin) that executes Wasm workloads directly, so you can continue to integrate with the Kubernetes API (either by using regular pods with a runtime class, or the Spin application manifest), but not get the overhead of a full Linux container for your lightweight wasm component.

kodama-lens• 2 years ago

Is there anything that makes spin stand out?

From what I've read, it seems much more complicated then it has to be. I've been running crun with wasmtime enabled since about 1 year in my private test clusters. No shim, no operator, no runtime class manager. I just have to set one extra annotation to the pod.

Only problem is that most k8s distros ship with runc. So there is no out of the box experiance yet, but it is a super efficient setup. Your approach seems more managed/enterprise ready, but also a more complicated. I don't want to be rude, I just want to get what additional value I would get from this since there a so many small differences in the WASM runtime world and I'm not super familiar with most of them

matei_radu3• 2 years ago

Thanks for the question!

For context, the shim is just an implementation detail of how containerd runs workloads; some come pre-built by default on some systems (like runc), and others you configure (like kata-containers, or in this case, the Spin shim); and some Kubernetes distros already have built-in shims (or will have soon), like k3s.

Your setup seems to work really well for you (and as a side note, I'm curious to learn more about the kinds of workloads you are running) — I'll note that you can also set the runtime class on a regular Kubernetes deployment / pod spec and you can run the workload like this with the Spin shim (for reference https://github.com/spinkube/containerd-shim-spin/blob/main/d...).

For Spin, we focused on the end-to-end developer experience, and on building a set of tools that take advantage of the benefits you get from Wasm, while integrating with a lot of cloud native tools — while allowing you to more easily build APIs and event-driven functions.

Our goal with SpinKube is to integrate with how organizations run workloads on in their infrastructure, and to make building, distributing, and running Wasm as seamless as possible.

Happy to dive into details for any of the points above, and would love to learn more about what you are running as Wasm.

Thanks!

plopz• 2 years ago

Did you solve the 32bit and no multithreading problems of wasm or are those still issues?

scubbo2• 2 years ago

I attended a talk by these folks at KubeCon last year - really great stuff. Sadly I'm not in a position to do anything with WASM in my professional life (and I have other priorities in my personal projects), but this for me is the "next big thing", more than any AI fluff - it will go unnoticed because it's low-level and transparent, but I'd bet paycheques on it having a huge impact in the coming years. Seems like a really exciting thing to be a part of.

willbush• 2 years ago

Hey I was there too :)

haolez• 2 years ago

Why? It's just an (allegedly) faster and lighter weight applications runtime on Kubernetes. For many applications, this will be less convenient than proprietary serverless stuff.

Ae you saying this is bigger than AIs that are so smart that they can do software development?

ElevenLathe• 2 years ago

I can't comment on Spinkube specifically, but I'd bet on serverside WASM-based orchestration, and WASM in general, being a big part of the next generation of IT stacks. If somebody can get the UX right, the "write once, run anywhere" promise of Java may finally come true.

jauntywundrkind• 2 years ago

The thing that interest me about wasm is having ultra lightweight small processes, which happen to tap into very large pools of already resident libraries in a safe sandboxed fashion.

There's been huge huge strides in low-profile elegant serverless platforms built around v8 isolates. We can get all those wins and more - in an way more scaled out more multi-tenant way - with wasm. If it lives up to it's aspirations, which it seems on target to do.

Luke Wagner's WasmCon talk iterated through the core attributes of what makes wasm wasm and what those attributes unlock. The ability to host much much much more with a muchower profile, with extremely short-lived sandboxed runtimes, is incredibly enticing. The app-server model could finally be disrupted.

I have no patience for "bigger than AI" apples to oranges comparisons. Wasm is orthogonal, has it's own features & distinctions. The direct run off comparison shirks both of their value.

russell_h• 2 years ago

In this context WASM is more like a slower container than an application runtime. The potential advantages being:

  * Far better isolation than a normal Linux container
  * If WASM takes off, maybe architecture-independent artifact distribution
  * If WASM takes off, maybe improved ability to call from one language into another

As a Kubernetes container runtime WASM today is probably most comparable to gVisor, with the somewhat huge caveat that it prevents you from just running normal Linux software.

pjmlp6• 2 years ago

It is basically Java and .NET application servers with YAML and extra complexity.

gdsdfe• 2 years ago

I believe so too wasm is going to change the game in really profound ways

Traubenfuchs• 2 years ago

How would I capitalize on this?

Is there any specific language or framework I could learn to stay competitive?

dns_snek• 2 years ago

Theoretically, any language that can target WASM, like Rust, Zig, or even C. I think the plan is to eventually support higher level languages with garbage collectors, but I don't know how much progress has been made on that front.

I haven't played with it enough to know which one provide the best experience at the moment, so I'd be interested to hear suggestions from others who have used it more extensively, especially with newer parts of the ecosystem like the component model, interface types, WASI, and generating ergonomic bindings to other languages (e.g. JS <> WASM)

spankalee• 2 years ago

This is a little hard to understand without more context. What's a "Spin WebAssembly application"?

edit: Seems to be this: https://developer.fermyon.com/spin/v2/index

rohitghumare• 2 years ago

Fermyon and Spinkube are doing a great job with this. WASM FTW!

pjmlp6• 2 years ago

Now all the complexity of Java and .NET application servers, with YAML spaghetti, and distributed systems infrastructure configuration.

Progress is wonderfull! /s

rahen• 2 years ago

How does this compare to krustlet.dev?

_w1tm• 2 years ago

Just FYI but krustlet is dead and the community has moved on to approaches where the container runtime handles the difference instead of having special nodes for it.

algorithmmonkey• 2 years ago

Krustlet implemented the Kubelet API rather than the containerd-shim API. That means that with containerd shims, you can use the default K8s kubelet, containerd, and extend your containerd runtime with a shim for executing Wasm. Kubelet is much higher in the stack and requires much more to implement it and everything below it in the stack.

FooBarWidget2• 2 years ago

Why are WebAssembly binaries smaller than native?

dragonelite• 2 years ago

They probably mean smaller than containers which package a slimmed down OS plus the executable binary.

giovannibonetti• 2 years ago

This reminds me of Lunatic [1], an Erlang-inspired runtime for WebAssembly. Unfortunately it seems like development stalled some months ago.

[1] https://lunatic.solutions/

ushakov2• 2 years ago

As far as I know, they're working on flawless.dev now

boomskats• 2 years ago

So interesting seeing this hit the front page while being sat in an auditorium in Barcelona at wasm i/o listening to one of the original creators of Krustlet, Taylor Thomas from Cosmonic, compare and contrast literally this 'wrapped in k8s' approach to the 'alongside k8s' interop alternative.

Great talk, highly recommend watching it on YouTube if you're into this kind of thing. No link but I imagine it'll be up pretty soon.

Spinkube: Running WASM in Kubernetes

Related Stories