Show HN: Lunatic – Actor System for Rust/WebAssembly

by bkolobara • 6 years ago

141 47 6 years ago

the_duke2• 6 years ago

Erlang/OTP inspired actor systems that run Webassembly code are a brilliant (and potentially way superior) alternative to microservice madness.

Until now adopting Erlang or Akka meant restricting yourself to a single language ecosystem, but Webassembly maturing and more and more languages gaining support means that you can still use a heterogeneous ecosystem while benefiting from easy sandboxing and isolation, almost instant startup, actor snapshotting/migrations, a unified way of inter-service communication ...

All while avoiding the complexities of grpc, building Docker images, Kubernetes, service meshes, "serverless" frameworks, ...

WASM running in the browser pushes languages to adopt it. Even Microsoft is really investing into dotnet support via Blazor [5].

Also check out wascc [1] [2], which is similar, also written in Rust, quite a bit further along , has some very cool concepts around security and code signing, and supports multi-node systems. (not affiliated)

There is also a spec for distributing WASM via OCI (aka Docker) images, which is useful here. [3]

Some Webassembly proposals like interface types [4] will really help out in this context as well. Sadly wasm has not quite been evolving at the pace I would have hoped for since 1.0. Progress is there, but from the perspective of an enthusiastic early adopter it can seem glacially slow.

[1] https://wascc.dev/

[2] https://www.youtube.com/watch?v=vqBtoPJoQOE

[3] https://github.com/solo-io/wasm-image-spec

[4] https://hacks.mozilla.org/2019/08/webassembly-interface-type...

[5] https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor

macintux• 6 years ago

single language ecosystem

The BEAM supports at least 20 languages.

indigo747• 6 years ago

And you can always feel the Erlang in them (though that's not a bad thing).

the_duke2• 6 years ago

All of which are tiny niche languages, except Elixir...

I actually think think BEAM gaining the ability to run WASM would be a smart development.

It would make using high performance C/C++/Rust code much easier than the NIF/Ports hassle, with much better sandboxing and scheduling control for the VM.

If they also add WASM actors on top it could expand the interest significantly.

OTP has decades of tuning and improvements that will be hard to match in other projects like this.

__jem• 6 years ago

"Supports" and "ready for production" are two entirely different things.

xiphias22• 6 years ago

I just checked, and I don't know any of them....to tell you the truth, I haven't even heared of them, except Elixir which I heared is a modern version of Erlang.

Snitch-Thursday2• 6 years ago

OTP inspired actor systems that run Webassembly code are a brilliant (and potentially way superior) alternative to microservice madness

I very much agree with this. While Actor model systems have not been commonly encountered in the circles I've run in (we don't need no stinking XML Web Services, we're going to microservices wooooo), this confluence of (Actors + Rust + Wasmtime) seems perfect for the age of increasingly-parallel CPUs as Moore's Law comes in for a landing.

I hope these kinds of efforts start to represent the dawn of a beneficial trend that starts us down the curve of excitement, cargo culting, then depression, then plateau of standard utility.

This trend doesn't have to get as bad as the OO craze of the 90s, but it needs to grow large enough to get people to stop reinforcing the OO model out of baby-duck syndrome or inertia.

ReactiveJelly4• 6 years ago

TIL https://en.wikipedia.org/wiki/Baby_Duck_Syndrome#Baby_duck_s...

justsomeuser2• 6 years ago

All while avoiding the complexities of grpc, building Docker images, Kubernetes, service meshes, "serverless" frameworks,

I don't think this will happen to be honest. The WASM process still has to run somewhere, it's probably going to be wrapped up into those systems.

Also all of the code written against existing system interfaces for IO etc will not be compatible with the WASM runtime, which means you'll have to run WASM + something else too (and the easiest path is probably going to be to use those "ops" systems you mentioned).

ReactiveJelly4• 6 years ago

The advantage is that you have _one_ Docker container for hosting your wasm host, and they can use some clever communications inside there, instead of having many containers which have to find each other and communicate using only basic networking primitives.

At work I have a Rust web server with basically two parts - What humans see (HTML), and what computers see (API)

The API rarely changes, and I was thinking of splitting them into two services so that I could iterate on the human side without worrying about breaking the API side.

But I don't want to do that because it means extra management overhead in Docker, extra build steps, etc. And the Docker image is unwieldy from having all of Debian packed in it.

So 1 Docker image with 2 Wasm servers in it, is less overhead (of ops) and less risk. I should still use testing to prevent the API breaking, and I still have to figure out a deterministic way to get Wasm images into the server, and measure the performance overhead of Wasm, but it's a tempting option.

dragonwriter• 6 years ago

Until now adopting Erlang or Akka meant restricting yourself to a single language ecosystem,

"Erlang" itself is a single language, but the BEAM/OTP ecosystem isn't usable only from that language (Elixir is fairly widely used, and there are others with less-broad use).

But, even funnier is saying that about Akka: Akka is a JVM library built in Scala and usable from almost any language on the JVM (so, in addition to its "native" Scala you also have Java, Kotlin, Ruby [by way of JRuby], Clojure, and...well, listing languages with JVM implementations that could use Akka could go on virtually forever.)

nsonha2• 6 years ago

I could not find any tutorial on running akka on golang, kotlinjs or scala native. I'll let you know how well it runs on python once I rewrite my whole infrastructure to use jython instead

tester756• 6 years ago

Even Microsoft is really investing into dotnet support via Blazor [5].

I think it's kinda underestimation because Blazor seems to be one of the top "implementations" of support for WebAssembly

ignoramous• 6 years ago

See also, Bastion: Highly-available distributed fault-tolerant runtime in Rust https://news.ycombinator.com/item?id=22403713

nahuel0x3• 6 years ago

This is the way forward. A wasm-instance-per-actor architecture enables:

- Actor prioritization

- Actor pausing

- Killing actors at arbitrary points

- Actor serialization / migration

- per Actor GC (internal to them)

- Multiple languages

- Inter-actor/host protected shared memory in addition to message passing

Has the potential to be better than Erlang and much, much better than Go.

dnautics• 6 years ago

Has the potential to be better than Erlang

what's the overhead of a wasm instance?

ReactiveJelly4• 6 years ago

Author claims 4 KB for a stack page here: https://news.ycombinator.com/item?id=25163677

whalesalad• 6 years ago

I have neen trying to think of a way to leverage Cloudflare's workers/durable objects as a poor man's Erlang cluster... I think this might be it!

Snitch-Thursday2• 6 years ago

I've been pushing away potential suitors to learn Actor-based systems, contenting myself with researching patterns to use with F# MailboxProcessor while I wait for Gleam (the proglang that is basically statically typed Erlang) to mature enough for this newbie to be able to jump in.

But the concept of rust-based Actor frameworks (that compile to WASM!?) is very, very tempting.

bkolobara5• 6 years ago

I'm a big fan of Gleam! I will put "compiling Gleam to WASM and running on Lunatic" on my "if I find time for it" list.

bkolobara5• 6 years ago

Hi! Author here. I wrote more about the motivation behind Lunatic if you are interested, check it out: https://kolobara.com/lunatic/index.html#motivation

uglycoyote2• 6 years ago

Thanks, because from the github page I had no idea what "Actor" meant, though I was interested to know more. Reading the code sample did not give me any better idea what was meant by the word "Actor"

It would be great if every github readme had a "motivation" section at the top. Why not link it there?

bkolobara5• 6 years ago

Thanks for the suggestion! I have added the link to the Readme.

octopoc• 6 years ago

Nice work! How do you pass complex objects between actors in Lunatic? Do you have to pass a byte array, or is there a more sophisticated approach where you don't have to serialize and deserialize each message?

bkolobara5• 6 years ago

Thanks! Currently the heaps and stacks of each actor are completely separate and it's impossible to pass anything without copying it. In the future I would like to add an option to share buffers for efficiency reasons.

nahuel0x3• 6 years ago

just a question: how much memory a wasm process in lunatic consumes as minimum?

bkolobara5• 6 years ago

You need 1 page (4 kb) for the stack at minimum.

elcritch2• 6 years ago

Fantastic project! The ability to transform serial compiled languages to run atop an async engine is great.

The examples, though, remind me more of Go routines and CSP rather than actors. They appear to be passing channels around to communicate which is very similar to but different than a mailbox open to any other process. Is that just a temporary approach? One of the best aspects of BEAM is OTP and supervision trees (linking processes) alongside flexible process naming. I'm not sure how channels would mesh with that. Granted this looks early stage and maybe you're planning on building mailboxes or process links or similar.

Also, you might consider providing support for a binary message passing format like Message Pack or CBOR. Excited to see where Lunatic progresses!

dmarlow• 6 years ago

I would love nothing more than for dotnet to adopt something like Erlang's actor system as well AOT compilation to native. I don't think I'd ever leave my house again.

gameswithgo• 6 years ago

I believe AOT is now easier/better with .NET 5 so take a look.

And there is: https://getakka.net/

joshsyn• 6 years ago

why do you think its better with .net 5?

reubenbond• 6 years ago

Have you seen Orleans? https://github.com/dotnet/orleans

staticassertion• 6 years ago

Pretty slick, reminds me of Erlang in that Actors are just functions. I've always thought this made stateful actors a bit weird since they have to recurse. I like the wasm approach though.

cercatrova• 6 years ago

Any comparison with actix(-web)? I know the latest version of actix-web doesn't use the actor model, but actix itself still does.

bird_monster• 6 years ago

It seems I've been in my "ignore all new tech stuff" hole for too long when I open a project link and don't understand most of the value/code. It seems very interesting though, I am not knocking the project at all.

So, I write code that compiles to WASM, and then pass that code as a process to Lunatic? Is this pattern intended to be a generation beyond microservices/containers, where rather than writing a "service" that gets put into a container and then managed by a container orchestrator, you're creating processes that get managed by a process manager? To get rid of the container overhead? If that's true, isn't it just all turtles because Lunatic will have to run out of containers (and presumably those will have to be managed/orchestrated)?

ReactiveJelly4• 6 years ago

I don't think the point is to reduce overhead. It may actually increase it, as long as the overhead for AOT wasm is more than 0. [1] As I understand it, containers have almost no CPU overhead. I have no citation for that.

Lunatic, to me, looks more like a web browser hosting many JavaScript sandboxes. But instead of JavaScript it's wasm, and instead of a web browser, it's only a host process, like node.js. (I don't know if node.js has a concept of running many side-by-side JS actors. Maybe it does)

So you have 1 Unix process hosting many Erlang-like actor processes.

I'm writing a little bit out of my depth, but I think the huge difference is permissions.

Sure, this all might be possible with containers. But Docker, the only container system I've used, is this huge thing. You have to install it, there's a daemon, it hooks into systemd probably, usually you need to be root unless you do everything just right. It requires kernel support for lxc, so I can't run it on the crappy little ARM boards I have to program.

Lunatic, like a web browser hosting JS sandboxes, is _just_ a Unix process. No root, no permissions. I think it will even work on Windows and OS X.

So yeah, if you're talking about web services where you can already manage many Docker containers on a Linux host no-sweat, it might not add much. But if this can be deployed on arbitrary clients with no root and no Linux kernel dependency, it's a very fun toy that must be useful for _something_.

[1] You may find this cool: https://hacks.mozilla.org/2020/02/securing-firefox-with-weba...

Firefox has a C++ font shaping library. Starting in FF 75, they compile the C++ to webasm, then AOT compile the webasm into native code. Resulting in a memory-safe sandbox for C++. There is some overhead, and it looks like a lot of set-up work. They don't give exact percent numbers on runtime overhead.

I don't know if Lunatic is doing JIT or AOT. I'll generously assume that one day they'll do whatever is faster. Part of the point of wasm is to beat JS performance, so I think at least compared to Electron or Node.js (JS sandboxes in a Unix process) it should be competitive.

elcritch2• 6 years ago

It's more about fine grained control of errors. WASM/Lunatic could bring many of the benefits of Erlang application design to any wasm friendly language. The creator of Elixir has a nice writeup of the Actor mentality and k8s: http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...

nsonha2• 6 years ago

how does this compare to other "green thread" approaches? Like go channel?

Show HN: Lunatic – Actor System for Rust/WebAssembly

Related Stories