Lunatic is an Erlang-inspired runtime for WebAssembly

by sts153 • 4 years ago

231 99 4 years ago

kotxig6• 4 years ago

I wish they had called it something else. I hate to be that guy but you know, it's not that hard to be maybe just a little bit more thoughtful.

https://www.bbc.com/news/magazine-17997413

Quarrelsome4• 4 years ago

Is it still that offensive? The linked article discusses a peak of the term around reform of 19th century "lunatic asylums" where the phrase was in common use and had dark connotations but aren't all those people dead now? Is the term still somehow problematic and if it is, doesn't that imply that words like "insane" have the same problem?

kotxig6• 4 years ago

I mean, I had to think about it. Whether it's hugely offensive or not isn't necessarily the right question here. With the whole English language to pick from there are probably better choices that are less ambiguous.

pessimizer2• 4 years ago

As far as I can tell, it's not offensive, and was never offensive. What can be offensive is using it as a term to refer to the mentally ill, especially in law - and it's really more antiquated than offensive even there. We have more specific and thoughtful terminology for varieties of mental illnesses now, that have nothing to do with the moon.

So if you have a law on the books governing how to treat lunatics, or how lunatics should be treated, it's a dumb old law because there's no official test for lunatic. It would be just as stupid if you had laws about the "wacky" or the "nuts."

The overwrought nature of the concern about seeing the word as the name of a piece of software is just a symptom of the current zeitgeist.

jamil7• 4 years ago

It's not offensive as much as it's kind of odd sounding, and would feel awkward pitching to someone else.

danpalmer2• 4 years ago

Agreed. While it might be debatable whether it's explicitly offensive, it's at least a bit of a grey area and it immediately stood out to me as a poor choice of name.

Whether you agree or not with the use of the term, names are part of communication, and communication that people are going to take issue with (rightly or wrongly) is going to have an impact on a product/project/business, and should probably be reconsidered, unless the point is to cause controversy or dogwhistle certain values.

This name for example would probably rule out use at many companies, and I suspect many would choose not to publicise it with things like conference talks.

lolinder4• 4 years ago

The problem is that in the age of the internet, it's very difficult to find anything that someone, somewhere won't be offended by. Every time an organization or project gives in and alters its innocently-chosen name because a tiny minority takes offense, they further normalize outrageous outrage.

It's definitely a fine line to walk. There are names that originated in an offensive context, and it's easier for me to see the objection to those. But "lunatic" hasn't been commonly used in its offensive sense in nearly a century, so I don't think it's fair to stoke outrage towards a project that started in the 2020s and chose it for extremely innocent reasons.

elteto2• 4 years ago

It is not used in a context in which it is offensive. It’s just a name. Maybe they just like looking at the moon a lot.

sli• 4 years ago

This logic implies that one could name a project after an explicit slur and it being a project instead of a directed insult makes it no longer a slur. That doesn't track at all.

bheadmaster5• 4 years ago

Everything is "offensive" these days - anyone can claim anything is "offensive", and because it's subjective, it's impossible to even be proven wrong.

The very word has lost all real meaning.

kotxig6• 4 years ago

definition of "lunatic" - "a person who is mentally ill (not in technical use)."

I didn't say it was offensive. If it was called "autistic", the word is not offensive but... why? It's just a really strange choice.

goodpoint3• 4 years ago

Everything is "offensive" these days

Spare us the BS. Society is very much able to form opinions on what is widely considered offensive and what is not.

Case in point: legal proceedings for libel, threats and so on. Turns out society still gives meaning to words.

it's impossible to even be proven wrong

You've been proven wrong right now.

yamtaddle2• 4 years ago

Someone on here recently asserted that ditching the colloquial use of "crazy" had significant mainstream support. I pushed back on people even noticing that it might be a problem, let alone actively choosing not to use it, being anything like a norm outside tiny niches of terminally-online Web users.

Sure enough, sensitized to it, I heard an NPR host and a Chipotle ad use it in the colloquial sense within the next week. And those are just the ones I noticed, and that were very-public rather than in private conversations.

This stuff's not mainstream and normal people don't care a bit. It's not even caught on in groups worried about impressing the word-police crowd (major advertisers and NPR both qualifying, I should think).

eigenrick• 4 years ago

The origin of the name comes from the fact that it started as a Lua project. In Lua (Portuguese for moon), everyone loves any "moon" related names or puns for their project. As such, "Lunatic" fits right in.

As someone who has family members with mental health issues, I don't find the term offensive at all.

I, for one, by default think more of someone who's obsessed with the moon, I guess it's because I never read that article, so I've never been told to be offended by it.

All that said, I do hope that the authors understand that they might be fighting an uphill battle with regards to adoption.

lolinder4• 4 years ago

The article you link to discusses "lunatic" being offensive when used in law in its original sense to refer to real people with real mental health problems. Whether that is offensive (and I think it probably is) is a completely different question from whether its modern, colloquial use is offensive.

The word dramatically declined in popularity through the 19th century but started climbing again after ~1995[0]. The recent increase in usage is driven entirely by a much milder, colloquial meaning of "wildly foolish"[1].

I think being offended because our great-grandparents used the word literally is a bit silly.

[0] https://books.google.com/ngrams/graph?content=lunatic&year_s...

[1] Meriam Webster identifies the older meaning as "dated": https://www.merriam-webster.com/dictionary/lunatic

macintux3• 4 years ago

Interesting. A huge part of the Erlang value proposition is that the language and runtime are designed to complement each other, and both have significant constraints imposed by the language design.

Taking half of that equation, and supporting any language, feels like you’re missing a big part of the benefit.

pessimizer2• 4 years ago

Elixir, LFE, etc. seem to work fine for the people who enjoy them.

https://github.com/llaisdy/beam_languages

LFE is Virding's himself, and has been around for longer than Erlang has been popular: Robert Virding - LFE - a lisp flavour on the Erlang VM (Lambda Days 2016)

https://www.youtube.com/watch?v=Br2KY12LB2w

Edit: Implementing Languages on the BEAM - Robert Virding - London Erlang User Group

https://www.youtube.com/watch?v=qm0mbQbc9Kc

weatherlight4• 4 years ago

I think you're missing the point. All these languages are functional programming languages, have structural pattern matching and share the same types. The types of languages you can develop on the beam are very restrictive. For example, you'll never get Ruby or Java to work on the BEAM VM.

let's look at garbage collection.

How does tradition stop the world, Mark-and-sweep garbage collection work on the BEAM? it doesn't really.

Each process on the beam has its own garbage collection algo, because processes can't share memory, garbage collection on one process can happen without disrupting any other process.

https://www.erlang.org/doc/apps/erts/garbagecollection

jeremyjh• 4 years ago

Those languages have the same semantics as Erlang in many important respects. For example, there are no mutable variables in any of those languages. They all have the same concurrency model, the same memory management etc.

klohto2• 4 years ago

How exactly is Erlang complimenting BEAM in a way that Elixir, for example, doesn’t?

macintux3• 4 years ago

The other languages that work with the BEAM are all roughly equivalent. I failed to be explicit, but as weatherlight indicated, languages like Ruby will never work on it.

spinningslate• 4 years ago

I read the "Erlang inspired" more narrowly. Specifically, it's about supporting large numbers of efficient, isolated processes. So there's a single, well-understood concurrency primitive: the (lunatic) process.

Having written concurrent code in Erlang, C# and Python, I'd view above as a win. A big win. Concurrency decisions in Erlang are simple: if things need to run concurrently, spawn them in different processes. If they're sequential, put them in a function. That's it.

In C# and Python it's nowhere near as straightforward. Do you use OS processes? Threads? Async/Await? Each has different strengths; each also has significant weaknesses.

So: a new runtime that aims to bring Erlang's simple, powerful and scalable concurrency model to any language that targets WebAssembly is, at the very least, interesting. It doesn't need to have all the wider benefits of OTP, or the symbiosis of Erlang and the BEAM. (Though that's great too obviously).

Quarrelsome4• 4 years ago

hang on, what part of the web assembly stack is this competing with exactly? I'm likely just very ignorant on how Web Assembly works but I don't understand exactly how it fits. Do browsers have pluggable hooks for interpreting web assembly that you can swap out for this or smth? Or are we talking about running WebAssembly on the server here?

chairmanwow1• 4 years ago

My understanding is using web assembly isolation to execute arbitrary untrusted code in a different (non-browser) env

Quarrelsome4• 4 years ago

So like what node.js is to javascript in the browser? For whatever reason I had mistakenly thought that web-assembly was purely a front-end browser thing.

bkolobara6• 4 years ago

I'm Bernard, the creator of lunatic. Great to see us back on HN and happy to answer any questions!

We have been busy pushing out a lot of new stuff. Recently we released a web framework [0] for Rust building on top of lunatic, and a live-view library [1] for crating full stack apps in Rust without JS. For now we are staying close to the Phoenix LiveView implementation, where everything is rendered on the backend and just a diff is sent over websockets to the frontend. But a big advantage of Wasm is that it runs in the browser too, so we hope to leverage this part to move some of the logic directly into the browser, like form checking, or offer a way to create offline-first experiences with the backend running as PWA in a service worker.

[0] https://github.com/lunatic-solutions/submillisecond

[1] https://github.com/lunatic-solutions/submillisecond-live-vie...

titzer• 4 years ago

I'm curious about this part:

This allows it to decide on a per process basis what syscalls are going to be forwarded to the operating systems.

1. Which system calls are being referred to here?

2. Does Lunatic follow WASI's capability-based filesystem access?

bkolobara6• 4 years ago

1. You can limit almost anything, file system usage (fopen, write, ...), networking i/o, how much memory can be used, message sending, the capability of spawning child processes, etc.

2. Yes

mr_luc3• 4 years ago

Ooh, LiveView pattern implementors! I have a question -- I've been tinkering with a custom Phoenix LV client in my 'shop time', so I wanted to ask: are you doing your diffs on a string-based representation, like they do in the internal Phoenix LV protocol? Or are you sending the DOM patches?

In the phx protocol, their diffs depend on having split the render into a tree of 'statics', 'dynamics', 'components', and on each change, that structure is patched, and needs to be recursively zipped to a string, parsed into a DOM, and patches computed against the previous DOM.

I've been playing with a new client as a way of understanding it, and it's definitely a certain amount of work! Fortunately computing 'what needs to change to reconcile 2 html-like DOMs' is a project that's been wrapped into libraries a few times in a few languages (phoenix relies on other people having done that too, via the 'morphdom' library in JS, and Dockyard recently open-sourced an ergonomic Rust library I'm using in my client).

So -- it's a lot of client work, but it optimizes for bandwidth. Cool.

But I've wondered if there could be any benefit to optimizing for granularity of patches, and simpler clients -- by sending 'just the patches', ie the list of DOM operations that cause the desired end state. My intuition is that the bandwidth 'cost' would be prohibitive, because the list of patches required to perform the morph could be quite large for a change that appears small in 'what string has changed' terms. (maybe harder to debug, too? lots of trade-offs)

I just don't have the energy to try it that way too, hehe, on the off chance that my intuition is off base and it'd actually help improve things (simpler clients, and maybe the bandwidth hit wouldn't be too bad?); it'd require deep server changes as well, when up until now I've only been writing a client. So I was curious what approach y'all are taking.

Prior/related art in the Rust space: I know there's a non-Phoenix liveview project, 'dioxus', with a generic DOM liveview implementation that several clients have been created for, by making a state machine that operates on that list of generated Patches; however I don't know what that one sends, whether DOM patches directly, or Phx-style diffs that need to go phxDiff -> zippedUpString -> dom -> morphPatches -> applyToDom.

bkolobara6• 4 years ago

We take the Phoenix approach of splitting the DOM into static and dynamic parts. In fact, we even re-use the pheonix.js library on the frontend, so we just implement the backend part that speaks the same protocol as Phoenix.

I don't think that sending DOM patches directly would have a significant higher bandwidth cost, but it would have higher "implementation" cost for us. That's why we choose to just use an existing "battle tested" library.

quechimba3• 4 years ago

I have implemented a VDOM in Ruby which I use to send patches to the browser. It works really well.

I'm currently playing around with signals as in SolidJS. That would result in more granular updates, but it's not as easy to implement.

jtwebman• 4 years ago

Has anyone written an OTP library on top of it?

bkolobara6• 4 years ago

We expose some OTP constructs in the lunatic Rust library directly, like supervisors (https://docs.rs/lunatic/0.12.1/lunatic/supervisor/trait.Supe...), gen_server (https://docs.rs/lunatic/0.12.1/lunatic/process/trait.Abstrac...) and others.

VikingCoder• 4 years ago

Neat. Good luck. I'll paste here the request I keep making again and again:

I want to teach something like Tailscale about my contacts, so they are allowed to communicate with my services (through their services).

Then I want to run an app inside a strong sanbox (maybe Lunatic?) that has capability-based APIs to do things kind of like ZeroMQ for sending messages TO THE CONTACTS. I don't want the API to look like a tcp/ip or udp connection. I want the app to think it's sending a message to a contact I've given it permission to talk to, via some interface that Lunatic itself provides.

If this was running in a strong sandbox, I think this would be an awesome way to develop and use simple federated apps. If something like Mastodon existed on top of this, I'd think it would be really secure and much easier to tell people to stand up their own node...

And I want the whole thing to feel like Sandstorm.io, in that self-updating way, with an app store... But Sandstorm.io is like an Oasis... I want a Federation. And I feel like a Rust-language-implemented WebAssembly sandbox that uses CONTACTS (not TCP) and Messaging, over something like Tailscale for connectivity and encryption, and then a Capability model for storing locally, or talking to a DB, or making external HTTP requests depending on the service, etc...

I want a WebAssembly thin client running on my phone... And a WebAssembly server running on my PC at home. And the server at home runs applets that THINK they're running on my phone (like XWindows, basically). Talking to other applets on my server, which talk to other applets on YOUR server, which talk to applets on your phone. I type a message on my phone, lots of Tailscale happens, and you see the message on your phone. And in my head, there's basically no boilerplate in the code I write. It just looks like Contacts and strongly-typed Messages.

That's the thing I keep wanting.

Kinrany• 4 years ago

This is a very cool vision but it sounds like a lot of research, and they probably have many lower-hanging fruit to pick before that!

johnisgood• 4 years ago

How are you going to implement hot reloading, or when can we expect it to be ready?

bkolobara6• 4 years ago

Hot reloading is always a pit tricky. The programming model needs to explicitly support it. In lunatic it will consist of 2 parts:

1. Stuff inside the vm that lets you keep the process id and message queue, but replaces the code running.

2. Higher level libraries that allows you to preserve state when the code is reloaded and potentially do state modification so that the newly loaded code can continue operating on the existing state.

This is fairly similar to how Erlang does it and I think it's a good approach for most use cases.

Nican• 4 years ago

Wow! I really like this project. I have been loving actor systems, and your project brings a lot of interesting concepts together.

I do not have much of anything to say, but I did come up with an idea. One thing that Fuchsia (operating system) does is that parents are able to lock down the permissions when spawning childs. For example, if a child is spawned with no I/O permissions, that child will have all those permissions removed, and any child that spawns from that will have the same restraints.

Could Lunatic support something like this on the far future?

EDIT: I just read the FAQ and looks to be heading that way. Awesome.

bkolobara6• 4 years ago

Thanks!

You can already do this in lunatic. For example, in the Rust library you can create a configuration (https://docs.rs/lunatic/latest/lunatic/struct.ProcessConfig....) with specific permissions (e.g. no i/o) and then use the spawn_config function to spawn children with it. Children will automatically inherit the configuration from their parents.

Lunatic is also a bit of a meta-system. We expose a lot of the vm stuff directly to the running wasm instances. So you can in a running wasm modules embed other modules or dynamically load them, then spawn processes from them. At the same time you can use the configurations to limit capabilities of this, potentially untrusted, modules.

toastal3• 4 years ago

Was interested, especially aiming to by language agnostic, then immediately not when I see all communications are exclusively via proprietary platforms.

Choosing proprietary tools and services for your free software project ultimately sends a message to downstream developers and users of your project that freedom of all users—developers included—is not a priority.

— Matt Lee

This means if Discord or Microsoft (parent of GitHub) or Twitter (I guess) decide to delete or ban your account, you are by proxy banned from the project, Lunatic; this means Lunatic is not in control of the moderation of its community. This also requires all users must give up personal data to the corporations in order to participate. This also further drives a wedge between those that prefer the spirit of open source not just in the style of communication, but those that want it in the tools used to create it as well.

This project isn’t the only or first offender, but it’s frustrating seeing this pattern pop up over and over and it needs to be called out every time. People care about their privacy more than ever. Interest in OSS has never been higher.

We just witnessed yet another exodus off Twitter for federated plaforms like Mastodon and other platforms because folks don’t want to deal with the whims of megacorporations. Everytime Microsoft or Microsoft GitHub make another questionable move, projects seek refuge on Codeberg/SourceHut/GitLab, or self-hosting. There are a non-negligible amount of folks that don’t want this.

ghqst• 4 years ago

How often do people get banned from GitHub for reasons anyone would disagree with? (besides projects that are targeted for DMCA, but you seen to be talking about personal accounts being unable to participate in the community of an open source project)

cestith2• 4 years ago

It is MIT licensed so one could host their copy or fork entirely elsewhere. It's available as a Rust crate, which doesn't seem to have the issues that concern you. The Discord part is a little harder to address. Contributing back to the code upstream could be difficult. Projects have moved before when corporate hosts proved problematic for either the maintainers or enough users, such as the exodus from Sourceforge. I have no doubt if Discord, Github, or Twitter become issues the project could move elsewhere, possibly to a private Gitea and to Mastodon. Right now, though, the maintainers are probably happy to use these services and not worry about maintaining production servers for it.

toastal3• 4 years ago

Use them, sure. Suppport them, sure. But require them, nah. Vendor lock on them, nah too.

83457• 4 years ago

If they need a theme song.

https://youtu.be/uqUa_G1h3pw

Defletter2• 4 years ago

Does Lunatic have a starter / example project?

taldridge• 4 years ago

There's some things: - https://github.com/lunatic-solutions/chat - https://github.com/lunatic-solutions/submillisecond/tree/mai...

Defletter2• 4 years ago

My apologies, I mean in terms of an AssemblyScript template / example project that takes advantage of https://github.com/lunatic-solutions/as-lunatic

LAC-Tech• 4 years ago

From the FAQ:

Each process has its own stack, heap, and even syscalls. If one process fails, it will not affect the rest of the system. This allows you to create very powerful and fault-tolerant abstraction.

This is also true for some other runtimes, but Lunatic goes one step further and makes it possible to use C bindings directly in your app without any fear. If the C code contains any security vulnerabilities or crashes, those issues will only affect the process currently executing the code. The only requirement is that the C code can be compiled to WebAssembly.

So as I understand it, an Erlang process can "crash" from an Erlang perspective and it's fine (and even encouraged!), but if it crashes from a system perspective (segfault or some such), it takes down all neighbouring processes as well - usually an issue when using foreign libraries.

With this set up, one of your erlang processes can segfault and it's fine?

I have to admit I'm not across using WASM as a general purpose backend.

peoplefromibiza2• 4 years ago

but if it crashes from a system perspective (segfault or some such), it takes down all neighbouring processes as well

there are three different types of crashes in Erlang.

First there are processes `spawn`ed with the spawn primitive. If such a process crashes, there's no risk for any other process in the system, not even exceptions can bubble up (or be intercepted for that matter).

The second type is a linked process, a process whose faith has been linked to their children.

In that situation if a parent process dies, intuitively all the children processes die with it too.

If a child process dies, it brings down the parent too, and if the parent process is a child itself, recursively until the top Application process.

It sounds risky until you realize that this mechanism allows parent processes to intercept child processes dying and act accordingly (restart one, restart all, ignore it, etc.)

That's how supervision works in Erlang.

Finally, if a process crashes when running as NIF (Native Implemented Function) or, to put it simply, a native library written in C (or that can export the same C calling convention) injected into the VM, it can bring down the entire VM.

syrusakbary• 4 years ago

Damn, it Wasm runtimes week! haha (Wasmer first, then WasmEdge and now Lunatic!)

Good to see Lunatic here, they have been doing great work on their runtime. Props to Bernard and team :)

thingification3• 4 years ago

Idly dreaming here, without enough understanding to even know how much this makes sense in this context, of:

- Erlang-style concurrency (which I haven't tried so don't understand but hear good things about always)

- With "platform" level capability security...

- ... and language-level capability security

- But with a sound and expressive type system like F# or similar

- Oh, and garbage collection

Lunatic is an Erlang-inspired runtime for WebAssembly

Related Stories