Writing a DSL in Lua (2015)

https://leafo.net/guides/dsl-in-lua.html

by flykespice • 1 year ago

93 26 1 year ago

neilv• 1 year ago

This is a nice example of one way to implement DSLs in a language that doesn't support them well.

IIUC, they're using Lua's language's function application syntax, combined with the ability to introduce a different namespace environment for function lookup, and then they're evaluating the tree of function calls at run time.

In languages that support doing this at compile time, through compile-time evaluation or compile-time syntax transformation, you can make run time faster. For example of syntax transformation, https://www.neilvandyke.org/racket/html-template/ shows an example of expanded syntax, where it's rendered some of the angle brackets and coalesced that with (escaped) HTML CDATA.

If you wanted to extend that, to support chunks of traditional HTML angle-bracket syntax inline with code (without using string literals), like in React, you could define a Racket reader to parse those chunks and transform them to parentheses syntax above.

wahern• 1 year ago

This is a nice example of one way to implement DSLs in a language that doesn't support them well.

It's not a coincidence that Lua supports such syntax for such use cases.

alberth• 1 year ago

I've always wanted to recreate a Python like syntax for Lua, this might be a good guide for that.

stdbrouw• 1 year ago

Around the time when CoffeeScript was popular, there was also https://moonscript.org/ for Lua. Made by the very same prolific Leaf who authored this DSL tutorial.

isr• 1 year ago

Yup, and it's still in active production use (itch.io, for example).

There's even a fork of moonscript, called yuescript, which has a speedier parser, and includes macro's. Effectively, it's moonscript-2.0, and it's bundled directly into the dora-ssr game engine (the same dev is involved with both).

Of course, the "big dog" in the compile-to-lua space is fennel. The great thing with all of these options is that they don't have a runtime component- it's all lua semantics, all the way down. So you can mix & match from project to project, as your whim takes you - and all the while you're really just gaining more experience in building lua solutions (while not having to fight against some of lua's syntax)

moonlet• 1 year ago

While I’m surprised to see they aren’t writing about https://www.inf.puc-rio.br/~roberto/lpeg/ it looks like they’ve covered LPEG in a previous post, and their chosen method this time around is neat anyway!

dan353hehe• 1 year ago

I love Lua for being able to do things like this.

I was building a bunch of html pages for an htmx frontend and a golang back end, and got really exhausted from using the builtin `html/template` library in golang. I kept trying to build reusable components, and it just didn’t work as I wanted it to.

I ended up doing this exact thing as mentioned in this blog post. https://github.com/DBarney/Glua Granted this is just for me, and for prototyping. I put it on GitHub so I wouldn’t loose it.

Writing html got so much easier this way:

    local function Component(data)
     return div{"my name is", data.name}
    end

    return function(params)
      html{
        head{title="this is my page"},
        body{
          h1{"This is my small page I wrote"},
          p{
            "some content",
            "more content"
          },
          div{"page has some dynamic values",params.count},
            Component{name="daniel"}
        }
      }
    end

Edit: formatting

arccy• 1 year ago

you should try https://pkg.go.dev/maragu.dev/gomponents and do it all in go

dang• 1 year ago

Writing a DSL in Lua (2015) - https://news.ycombinator.com/item?id=22223542 - Feb 2020 (41 comments)

roxolotl• 1 year ago

Lua is a much cooler and more useful language than I think it, publicly, gets credit for. I know it’s widely used in many industries but so many times I’ve read an article like this and thought that it could be even more widely useful.

I’m currently writing a just for me static site generator in Fennel and it’s sad to see just how many powerful packages are abandoned. A lot of it is because they are mostly complete and part of why I like it is lack of churn but a last update on 18 months ago is very different from 10 years.

I want to host knitting patterns with my site generator and techniques like this make it super easy to build a little pattern dsl. It’s stuff like that which makes me wish it was more widespread.

epcoa3• 1 year ago

Lua is a much cooler and more useful language than I think it, publicly, gets credit for.

I think it’s gets about the right amount of credit. It isn’t obscure by any means, openresty is used quite a bit in odd places, so many games have used it. PyTorch (well what became it) was originally written in it (it was just torch but basically LuaTorch)

It doesn’t get more adoption because its warts and limitations really start to show once pushed into more into a “real” language. It doesn’t have the browser monopoly. It’s also quite slow, and LuaJIT doesn’t always magically fix that.

Yliaho• 1 year ago

LuaJIT doesn’t always magically fix that.

Not to mention there are some significant performance issues on aarch64. Love2D, a popular game framework, opted to not use JIT on macOS for this reason.

https://github.com/LuaJIT/LuaJIT/issues/285

aeadio2• 1 year ago

Lua without JIT is one of the fastest interpreted languages. In fact it might just be the fastest. LuaJIT is also one of the most advanced JIT compilers out there, frequently beating V8 and others.

There are very valid complaints about Lua. It lacks useful quality of life features. Some of its design decisions grate on users coming from other languages. Its standard library is barebones verging on anemic, and its ecosystem of libraries does not make up for it.

But after using Lua in many places for well over a decade, I’ve gotta say, this is the first time I’ve heard someone claim it’s slow. Even without JIT, just interpreter to interpreter, it’s consistently 5-10x faster than similar languages like Python or Ruby. Maybe you’re comparing it to AOT systems languages like C and C++? That’s obviously not fair. But if you put LuaJIT head to head with AOT systems languages you’ll find it’s within an order of magnitude, just like all the other high quality JITs.

wruza2• 1 year ago

You can learn about various opinions in this thread: https://news.ycombinator.com/item?id=42517102

so many times I’ve read an article like this and thought that it could be even more widely useful

Lua is not widely used because it doesn’t exist. Lua {5..1-4,JIT} do exist, but these are different languages between which you and others can easily port code sometimes. It’s sort of cool in a moment, but if (when) you live through a couple of versions, that just gets old. It’s not immediately visible, so you’ll see reviews from 5 stars to zero at all times.

aa-jv• 1 year ago

Actually, Lua exists all over the place, in all kinds of amazing applications - but because Lua is an embeddable VM, you mostly don't know about it that much. Its not exposed because - when you use Lua professionally, you tend to bury it deep in the problem it solves for you.

gustavopezzi• 1 year ago

Lua's LPeg library would also be a great option here. A couple of years ago I taught an interpreters class together with Roberto Ierusalimschy and we used LPeg for lexing and parsing. The end result was great. Even if you're not using PEGs for your implementation I would recommend spending 30 minutes and looking at LPeg.

jFriedensreich• 1 year ago

trailing closure syntax looks similar but is probably even a bit nicer. I just wish js would get this feature so we could stop using jsx.

mjmas• 1 year ago

MicroLua for the RP2040 has a nice dsl for the PIO assembly with some special use of methods: https://github.com/MicroLua/MicroLua/blob/main/lib/pico/hard...

    function pio_timer(_ENV)
    public(start):
        set(y, 0)
        mov(y, ~y)
    wrap_target()
    delay:
        pull()
        mov(x, osr)
    loop:
        jmp(x_dec, loop)
        mov(isr, ~y)
        push()
        jmp(y_dec, delay)
    end

flykespice• 1 year ago

Was that really achieved with vanilla Lua or MicroLua has some language extensions?

SyrupThinker• 1 year ago

While I can't speak for MicroLua, this can indeed be accomplished with vanilla Lua.

There are two things in play here:

First, the _ENV variable here is special.

It is implicitly used to lookup any identifier that has no visible binding, locally or in the surrounding scopes.

Thus a script just containing print("Hello world") is really _ENV.print("Hello world").

Usually _ENV is just implicitly defined to be the global environment (available as _G), but it can be overridden within a lexical scope to a custom value.

Second, the jump labels just make use of the alternate function call syntax of object:method(args), which is equivalent to object.method(object, args).

The whitespace is non-significant, which allows it to be written like that.

In combination with metatables you can use _ENV to track variable reads, calls etc. within a function, which you can (abuse) to create DSLs.

You can get an idea of whats possible by just tracking what we can intercept with _ENV. Adding the following code after the pio_timer function, and running the script with Lua 5.4, already gives us quite a bit for just the first three lines of the function.

local loggerMetatable = {} function loggerMetatable:__bnot() print("Called bnot on " .. rawget(self, "name")) end function loggerMetatable:__call(...) local name = rawget(self, "name") print("Invoked " .. name .. " with " .. #table.pack(...) .. " args") -- Return another logger table to visualize variable interactions return setmetatable({ name = name .. "(...)" }, loggerMetatable) end function loggerMetatable:__index(key) local name = rawget(self, "name") print("Accessed key " .. key .. " on " .. name) -- Return another logger table to visualize variable interactions return setmetatable({ name = name .. "." .. key }, loggerMetatable) end

local pio_env = setmetatable({ name = "_ENV" }, loggerMetatable) pio_timer(pio_env)

-- Output:

Accessed key public on _ENV

Accessed key start on _ENV

Invoked _ENV.public with 1 args

Accessed key set on _ENV.public(...)

Accessed key y on _ENV

Invoked _ENV.public(...).set with 3 args

Accessed key mov on _ENV

Accessed key y on _ENV

Called bnot on _ENV.y

Invoked _ENV.mov with 1 args

[...snip...]

binary132• 1 year ago

It never fails to strike me as odd how polarizing Lua is. Some people (like me) find it really pleasant and fluid to use. Other people get hives from things like 1-based indexing and global-by-default. The intense complaints are just so silly, especially when the “better alternative” presented is typically Python, which is one of the worst languages I’ve ever used. Lua’s syntax and semantics are so much smaller and cleaner that it’s mindblowing how simple it makes rather complex and custom abstractions. When I use Lua I feel like I’m working on another level of abstraction than in other languages, almost a lisp, but with just the right amount of convenient / quality-of-life features that it makes everything trivial. Multiple inheritance, mixins, type constructors are all trivial classes, if you want them, or parts of them, or to do your own thing entirely, or you can choose none of it at all and merely build a very primitive C-composition scripting API for your library, and yes embedding languages inside it is therefore also trivial. I’ll never understand the mindset that leads people to hating it.

“But bro, Python has C libraries!!”

and?

Lua is built out of C semantics and has a beautiful C API

if I want native libraries, I’ll just take a little time to write a trivial wrapper for precisely as much functionality as I want from the library

Writing a DSL in Lua (2015)

Related Stories