Lua in the NetBSD Kernel [pdf]

https://fosdem.org/2013/schedule/event/lua_in_the_netbsd_kernel/attachments/slides/278/export/events/attachments/lua_in_the_netbsd_kernel/slides/278/kernel_mode_lua.pdf

by profquail • 13 years ago

68 43 13 years ago

fosdem.org

AlexanderDhoore5• 13 years ago

I was at this talk @ fosdem. Pretty cool stuff.

He said he considered java (there have been attempts to write drivers in java) and python. Python didn't make it because of high ram usage (java wasn't any better). He didn't look into ruby, but "not for any particular reason". Probably because embedding ruby is quite hard compared to python or lua.

Very interesting talk!

pm90• 13 years ago

Do you know if that talk was recorded, by any chance? I checked here http://video.fosdem.org/2013 but couldn't find it there

AlexanderDhoore5• 13 years ago

It wasn't recorded. At least not officially. Fosdem has so many rooms, I think they just give up at some point. It's a shame though.

Edit: Just checked and there are videos for only one devroom. And there were around 30...

klrr• 13 years ago

How does this really work, is it possible to write such stuff as drivers with a garbage collected language?

pjmlp• 13 years ago

Yes.

There are operating systems written in system programming languages with GC. Check Native Oberon and A2 for desktop operating systems used at Zurich's university.

Smalltalk was originally developed as OS. Only the so called primitives were done in Assembly, with the remaining parts in Smalltalk itself.

Sun played around with Java in Solaris for device drivers and Squawk is a VM that runs directly on hardware with device drivers written in Java, as an example.

rwmj2• 13 years ago

Also OpenMirage, written in OCaml straight to the "hardware" (where "hardware" is a Xen virtual machine). http://openmirage.org/

cpressey• 13 years ago

As someone who once threw together a Lua kernel module for FreeBSD 4 just for fun, I approve of the idea -- although I don't know how useful it is.

Being able to rapidly prototype kernelspace things is probably the main advantage. I'd think twice about actually running garbage-collected code in the kernel in production. And the slide that touts that it's easy for users to modify -- yes, well -- you could easily argue that it shouldn't be easy for users to modify what's in the kernel :)

AlexanderDhoore5• 13 years ago

Indeed, prototyping functionality is the main point here. The whole system isn't even part of the default install and can be turned on/off.

Easier is always good. It's still far from easy though.

Nelson69• 13 years ago

I've seen this on an embedded product that has been sold for years.

The license is liberal, it's remarkably easy to work with Lua, is can basically call any of your C functions. Great for debugging stuff, prototyping, certain kinds of monkey patches. Great for doing unit test like experiments too; you can have some Lua put the system in to a particular state and then do your experiment.

KMag• 13 years ago

Yea, I agree that it's neat, but probably of limited practical value.

If you need something like this, it's probably best to either host a kernel in userspace (like UML), or else have a lightweight kernel interface that allows you to do your heavy lifting in userspace. I've never done any Plan9 driver development, but my understanding is that Plan9 has userspace equivalents of most of the internal driver API calls, so that a driver module can almost seamlessly be developed as a userspace driver and then (if it turns out to be performance critical) run as an in-kernel module in production.

Once you've got good userspace APIs for the kernel APIs you need to use, then all kinds of rapid prototyping possibilities open up.

wildchild• 13 years ago

Could anyone explain, why Lua is so special?

AlexanderDhoore5• 13 years ago

It's very small and uses very little memory. It's designed to be embedded. And has a liberal license (MIT).

deweerdt• 13 years ago

It's not the language itself. It's its implementation. Embedding a Lua interpreter in a C program is straightforward. See the 'Python' and 'Java' slides in the OP: it's all about integration.

Slackwise2• 13 years ago

Take JavaScript, remove all the wonky stuff, and now you've got Lua. Then ask yourself, why has JavaScript become popular outside the browser?

ams6110• 13 years ago

Javascript is popular outside the browser not because it's a great programming language, but because it gets us back to where we were before the browser became the dominant app delivery platform: it's now again possible to write your entire client-server app in one language.

vor_4• 13 years ago

Small, fast, and easily embeddable in C programs. It's been popular in videogames for years but hasn't seen much usage outside that industry, so it's always interesting to see it used in different ways.

eric_bullington3• 13 years ago

It's very small, fast, and uses little memory. It's easy to learn and easy to interop with C. It's got two great implementations, with one (LuaJIT) probably holding the top spot among all dynamic, garbage-collected language currently in existence, measured by performance (although the creator -- `mikep` on HN -- is very careful to avoid making this sort of claim).

It's also got one of the coolest GUI toolkits around in IUP. If you've ever wondered by a cross-platform GUI toolkit that uses native widgets has to be so huge and complex, IUP's what you're looking for. Unfortunately, IUP doesn't yet have a native OSX driver, but you can use motif graphics driver to build a program that will run on OSX. Like Lua, IUP's in portable C, so a number of other languages have built bindings, notably go and perl. Like Lua, IUP has a very liberal license license (MIT), so you can use it for commercial projects without worrying about dynamic or static linking.

arundelo• 13 years ago

Correction: LuaJIT's creator Mike Pall is http://news.ycombinator.com/user?id=mikemike .

snogglethorpe4• 13 years ago

IUP seems cool in theory, but in practice, I've never even been able to compile/install it, despite trying multiple times over the years! Unfortunately the build system seems to be mostly of the "Works great on the author's system!" sort... ><

I always try a bit to hack around the issues, but the problems have always exceeded my "oh this is too annoying, I'll try again next year" threshold (part of the problem being that it's an unusual build system, not just a few Makefiles, so it's less familiar).

snogglethorpe4• 13 years ago

This is a tweaked/updated version of a stackoverflow answer I wrote (see end for link) to someone wanting to know what language they should use for embedding:

Lua, and Lua's standard implementation(s), capture a wonderful balance of features/smallness/speed/usability, particularly for embedded use.

Lua:

1. Is very small, both source and binary, an order of magnitude or more smaller than many more popular languages (Python etc). Because the Lua source code is so small and simple, it's perfectly reasonable to just include the entire Lua implementation in your source tree, if you want to avoid adding an external dependency.

2. Is very fast. The Lua interpreter is much faster than most scripting languages (again, an order of magnitude is not uncommon), and LuaJIT2 is a very good JIT compiler for some popular CPU architectures (x86, arm, mips, ppc). Using LuaJIT can often speed things up by another order of magnitude, and in many cases, the result approaches the speed of C. LuaJIT is also a "drop-in" replacement for standard Lua: no application or user code changes are required to use it.

3. Has LPEG. LPEG is a "Parsing Expression Grammar" library for Lua, which allows very easy, powerful, and fast parsing, suitable for both large and small tasks; it's a great replacement for yacc/lex/hairy-regexps. [I wrote a parser using LPEG and LuaJIT, which is much faster than the yacc/lex parser I was trying emulate, and was very easy and straight-forward to create.] LPEG is an add-on package for Lua, but is well-worth getting (it's one source file).

4. Has a great C-interface, which makes it a pleasure to call Lua from C, or call C from Lua. For interfacing large/complex C++ libraries, one can use SWIG, or any one of a number of interface generators (one can also just use Lua's simple C interface with C++ of course).

5. Has liberal licensing ("BSD-like"), which means Lua can be embedded in proprietary projects if you wish, and is GPL-compatible for FOSS projects.

6. Is very, very elegant. It's not lisp, in that it's not based around cons-cells, but it shows clear influences from languages like scheme, with a straight-forward and attractive syntax. Like scheme (at least in it's earlier incarnations), it tends towards "minimal" but does a good job of balancing that with usability. For somebody with a lisp background (like me!), a lot about Lua will seem familiar, and "make sense", despite the differences.

7. Is very flexible, and such features as metatables allow easily integrating domain-specific types and operations.

8. Has a simple, attractive, and approachable syntax, which can be important if you intend to have end-users write scripts.

9. Is designed for embedding, and besides its small size and fast speed, has various features such as an incremental GC that make using a scripting language more viable in such contexts.

10. Has a long history, and responsible and professional developers, who have shown good judgment in how they've evolved the language over the last 2 decades.

11. Has a vibrant and friendly user-community.

Stackoverflow link: http://stackoverflow.com/questions/7919763/lua-vs-embedded-l...

_dps• 13 years ago

Let me add my favorite hobby horse to your point 4: LuaJIT's C FFI is a tremendous and undervalued advantage compared to any other language/implementation I've used. Its core value proposition:

0) It can parse C header files and automatically construct bindings.

1) You can write Lua code that works directly on C data types with C memory layout and is JIT compiled. It makes it very easy to achieve performance within ~2x of C while still maintaining the expressiveness and extensibility of Lua. And if you miss writing OO C++, you can easily add methods to your C structs and they will be JIT compiled. See here http://wiki.luajit.org/Allocation-Sinking-Optimization and search for "Point Class with FFI cdata Struct" and then "Point Class Benchmarks".

2) Because the memory layout matches C data types, it is trivial to swap out a Lua implementation of a core operation with a C implementation if that becomes necessary.

3) It is easy to write cross-platform systems code without conditional compilation because the native calls through FFI can all be constructed dynamically through Lua (so instead of #ifdef WIN32 all over your socket code, you just have a single line checking for ffi.os == "windows" and then if necessary call the winsock initializer as ffi.C.WSAStartup(foo,bar)).

lemcoe9• 13 years ago

Oh my god, I may have fallen in love with that PowerPoint theme...

profquail• 13 years ago

I think it's one of the standard themes for Beamer (LaTeX package for creating presentations):

http://www.hartwork.org/beamer-theme-matrix/

vor_4• 13 years ago

Lua sometimes drives me nuts, but it's such a humble little language that you can't hate it for too long. There are alternatives I would have preferred, like the Lua-inspired Squirrel (http://http://squirrel-lang.org ), but Lua is the most well-known and is written in plain C, which is likely appealing to a BSD project. I'm interested to see if this gains any popularity and if the other BSDs show interest.

StavrosK2• 13 years ago

How does it drive you nuts?

rwmj2• 13 years ago

Arrays based on 1. The odd and slightly under-documented stack that you have to use when interfacing it with C. Global variable scope by default. Tables (.. are both good and bad). Not standard named operators like '=~' and ':'.

Having said that, it's not a bad language. It has an excellent community, who really went to great lengths to help me writing the Lua libguestfs bindings.

http://libguestfs.org/guestfs-lua.3.html

vor_4• 13 years ago

It's annoying building a class system by hand. I've got C functions I wrote to do it for me, but I wish I didn't have to.

There are certain behaviors I dislike even more. One example is a historical quirk in which all non-nil or non-false expressions are true. This means inconsistent behavior like:

while 0 do print("Loops forever") end while not 1 do print("Does nothing") end

and:

while 1 do print("Loops forever") end while not 0 do print("Does nothing") end

I also don't like how undefined variables return nil. This can cause a host of subtle errors, including accidentally creating sparse arrays if you make a typo in a variable name, which will mess up the behavior of the length operator.

The syntax, which is what most people complain about in Lua, really doesn't bother me. It's the sometimes unconventional behavior of the language and its minimal feature set.

pasbesoin• 13 years ago

Your link is borked. It should be

http://squirrel-lang.org

P.S. No criticism meant. Just a friendly correction. :-)

fafner• 13 years ago

Similar to this SnabbSwitch is a virtual Ethernet switch for Linux written mostly in Lua (using LuaJIT).

https://github.com/SnabbCo/snabbswitch/wiki

markwong• 13 years ago

will the backward compatibility of Lua be a matter?

Lua in the NetBSD Kernel [pdf]

Related Stories