Making your own programming language is easier than you think (but also harder)

https://lisyarus.github.io/blog/posts/making-your-own-programming-language.html

by atan2 • 3 months ago

8 2 3 months ago

lisyarus.github.io

jdw64• 3 months ago

I enjoyed reading this and gave it an upvote.

I also tried building my own programming language, and I think the difficulty depends a lot on how far you want to take it.

At first, when I used C as the transpiler target, it felt relatively manageable. But once I tried to support a different backend, the difficulty increased dramatically. It is now a project I work on seriously with LLMs, and my impression is that making a language is both easier than expected and harder than expected.

vrighter• 3 months ago

"Lua (or any other JIT-compiled scripting language for that matter). That's a standard choice, but it turns out that it's really hard to sandbox it."

This is a sign the author didn't even try it properly. Lua is one of the easiest languages to sandbox. You can choose not to load the dangerous libraries into the environment in the first place, or you can load them and set up a different global environment table that you specify only for any untrusted code (not by prepending text to the untrusted lua code). The only thing you really need to do is to never accept untrusted bytecode and always load untrusted scripts by compiling lua source code, so you can be sure the bytecode is valid. Or you could even very easily spawn a lua state for each separate untrusted lua code.

When I want a scripting language that's easily sandboxable, lua is the first one I reach for.

"Lua is a highly dynamic language that knows nothing about c pointers"

That's why you have both lightuserdata and userdata. Where you can set up a metatable from the C side, which can't be overridden from the lua side. It was honestly one of the easiest languages to integrate (try embedding python, and then tell me how much hair you have left).

And if you use luajit, you also get niceties like native integer support in lua (before lua 5.3, but with sane bitshifting), borderline frictionless FFI (in most cases, just providing the header file is enough). And by frictionless I mean you can literally manipulate c structs directly from luajit, without having to write any translation code (again, luajit parses the header itself). LuaJIT literally satisfies all of the design goals.

Making your own programming language is easier than you think (but also harder)

Related Stories