Love for LuaJIT

http://google-opensource.blogspot.com/2010/01/love-for-luajit.html

by whyenot • 17 years ago

75 45 17 years ago

google-opensource.blogspot.com

whyenot2• 17 years ago

Mike Pall's annoucement on Lua list: http://thread.gmane.org/gmane.comp.lang.lua.general/62316

mbrubeck3• 17 years ago

LuaJIT 2.0 is an amazing achievement. On the programming language shootout's "flawed benchmarks", LuaJIT outperforms almost every other dynamic language (including JavaScript V8, Python, Ruby 1.9, JRuby, VisualWorks Smalltalk, CPython, PLT Scheme), and is competetive on many benchmarks with highly-optimized language implementations like Sun's JVM, and GHC:

http://shootout.alioth.debian.org/u32q/which-programming-lan...

Mike Pall wrote LuaJIT and now LuaJIT 2 as essentially solo efforts. Thanks to Google and the other sponsors for helping him continue!

sandGorgon• 17 years ago

it would be interesting to see LuaJIT as part of LuaTex.

Super fast desktop publishing FTW !

lsb• 17 years ago

I was abusing a footnote package, sorting footnotes alphabetically per page, and that took about a second a page on my laptop, but for everything else disk seeks take the majority of TeX's time; once you LaTeX something, and cache the font files in memory, it usually takes under a second or two for anything under 50 pages.

silentbicycle2• 17 years ago

In my experience, the 1.x releases of LuaJIT have typically made whole programs (not just benchmarks) run 2-3+x faster than standard Lua. Without any other changes. :)

I haven't tried LuaJIT 2 yet, but I'm eagerly waiting for it to get ported to amd64.

joeyo2• 17 years ago

The fact that luaJIT is written in assembler makes me wonder how practical an amd64 port. :-\

mikemike5• 17 years ago

For an apples-to-apples comparison you'll want to look at the single-threaded performance:

http://shootout.alioth.debian.org/u32/benchmark.php?test=all...

aaronblohowiak• 17 years ago

Spoiler: LuaJit compares even better when comparing single-threaded performance.

Edit: replaced "performs" with "compares", eliminating ambiguity around competitive performance vs clocktime performance

dchest2• 17 years ago

We use Lua internally at Google

Interesting! Anyone know details?

yannis• 17 years ago

Just a guess google power meter? http://www.google.org/powermeter/index.html

whyenot2• 17 years ago

Joshua Haberman gives some details here: http://article.gmane.org/gmane.comp.lang.lua.general/62321

Our Lua usage isn't too widespread at the moment; it's really one infrastructure project in particular that uses Lua to allow user-defined functions to run within a tightly controlled container. Lua was the best choice, because of its low overhead, fast execution, and the ability to set limits on execution time.

Unfortunately I cannot be more specific, since the project is not public.

haberman3• 17 years ago

Man, you only have to say something once and then the Internet replicates it for you. :)

teamonkey• 17 years ago

Interesting to read this. I thought Google only used C and JVM-based languages as a company policy. Perhaps that's still the case except for this one example?

toisanji• 17 years ago

it makes me wonder why languages like ruby are so slow.

mbrubeck3• 17 years ago

Ruby might just need a better implementation with more time spent on optimization. http://maglev.gemstone.com/ is going in this direction.

But also, Lua was always designed with cleanness and simplicity of implementation in mind. Its developers have strong backgrounds in both theory and engineering, which you can see in their interview in the book Masterminds of Programming. They've been very conservative about the design of the language. In contrast, Ruby feels much messier and more "evolved" - look at its Yacc grammar[1], for example, compared to the grammar of JavaScript or Lua[2]. Even the Ruby lexer is full of weird edge cases. Much of MRI is like that, and for years it was the only "spec" for the language; now its decisions have been reverse-engineered and inherited by all other implementations.

[1] http://blog.nicksieger.com/articles/2006/10/27/visualization...

[2] http://www.lua.org/manual/5.1/manual.html#8

[By the way, I've used Ruby for years and like it at lot, while I've written only a few lines of Lua code and like it but am not deeply familiar with it.]

dchest2• 17 years ago

Also, "The Evolution of Lua" — an interesting paper that shows how they came to this fast and somewhat elegant language: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.... [PDF]

derefr• 17 years ago

It'd be interesting to see how fast Ruby would run "cached" (i.e. with the AST already assembled in memory,) to see how much of the time is really spent in the parsing and lexing phases.

haberman3• 17 years ago

Also, Ruby has much more complicated rules for method dispatch. Lua just has functions, and if you want to implement any more complicated dispatch, you have to do it yourself. Once I implemented Ruby-like dispatch on top of Lua, and it was terribly slow:

http://blog.reverberate.org/2009/02/09/disappointment/

To make a long story short, porting my existing Lua project to use my Ruby-like method dispatch resulted in a 4x slowdown.

stcredzero• 17 years ago

Ruby might just need a better implementation with more time spent on optimization...But also, Lua was always designed with cleanness and simplicity of implementation in mind

Ease of implementation has a multiplier effect with Open development effort. Not only do you get more of "more eyeballs," new contributors can ramp up faster. There tends to be a better effort/ fun ratio.

Even the Ruby lexer is full of weird edge cases.

Actually, the lexer sets state that changes the operation of the parser and vice versa.

mikemike5• 17 years ago

Some of Ruby's more excentric semantics are really hard to optimize away by a compiler. It takes a much greater effort to compile Ruby source code to machine code of similar quality than for Lua. Since time and money are not infinite resources, especially for compiler development, you'll get worse results in practice.

So, yes, language design does impact performance.

haberman3• 17 years ago

> Some of Ruby's more excentric semantics are really hard to optimize away by a compiler.

Any examples in particular? Besides name resolution and method dispatch (and callcc, but I think that may have even been removed from Ruby 1.9) none of Ruby's semantics seem all that different from Lua. Am I missing something?

I guess there's also throw/catch and exceptions.

blasdel3• 17 years ago

Ruby was just designed and implemented with all the attention paid to sugary syntax/semantics and very little thought given to how well it could be implemented. MRI is a really poor implementation of a (formerly) poorly-specified, highly irregular, unorthogonal language.

MRI's garbage collector is abysmal, use Phusion's REE fork to get one that at least doesn't mark every single object on every collection and actually bothers to perform them in normal situations. It didn't do any desugaring at all until 1.9, and YARV only has bytecodes for the most primitive accessors, assignments, stackops, jump/condjump, and returns; with optimizations for -, +, <. <<, and some regexes. Your normal code just gets passed through verbatim.

Every once in a while I'll be reading docs or sitting in irb and suddenly scream at the ceiling, spin in my chair, and get the urge to throttle a Japanese person.

sjs• 17 years ago

No need to wonder, have a look at the source code for them both and it will become clear exactly why there is such a performance difference. Lua gives you raw C numeric types which makes a big difference right there. Other people have mentioned a few other good reasons too (no costly method dispatch).

I haven't looked at LuaJIT yet but considering how small it is I'm sure it meets or exceeds the quality of the code behind Lua.

chancho2• 17 years ago

> Lua gives you raw C numeric types

Standard Lua (the one from PUC-Rio) doesn't really, it uses boxed numeric values, as I imagine ruby (and almost every other dynamic language) does as well. "Boxed" meaning that each number is stored on the heap with some kind of tag, and when you pass such a value to a function you're really passing a pointer to it and that indirection costs cycles and puts pressure on the cache. LuaJIT's most significant (IMO) feature is that it stores numeric values directly, like C does, so when you call a function the actual bits representing it go right on the stack, not the address of where the bits can be found, and it uses some cool tricks of floating point representation to prevent the garbage collector from trying to follow these stack values as if they were pointers. This (combined with the trace optimization) is why when you write a numeric for-loop in LuaJIT it runs (nearly?) as fast a C.

cpr• 17 years ago

http://www.macruby.org/ -- Apple's supporting an OSS project to use LLVM for real compilation, getting Ruby within spitting distance of Objective-C.

Of course, the fact that it's tied to the Objective-C runtime means it's not portable as-is, but at least it's a proof that Ruby need not be slow.

kilowatt• 17 years ago

It's been on my todo list to take some time to grok luajit's internals for awhile. Awesome to see Google parting ways with some cash!

This sort of makes me want to try to write a Python->Lua compiler.

fortes2• 17 years ago

Anyone using Lua for web apps? What's the current state ofthe ecosystem? Anything good, stable, and easy to deploy?

lh• 17 years ago

That's something I've also been wondering lately, since the Kepler and Orbit projects haven't seen activity for ages (at least according to their webpages).

I found this Sinatra clone for Lua called Mercury (http://github.com/nrk/mercury), but it lacks documentation and does not seem to be very mature at the moment.

fortes2• 17 years ago

Yeah, I've poked around a bit as well, but didn't find anything that seemed like it was getting traction.

I was excited when I heard that mod_lua was going to be included with Apache by default [1] ... but I haven't really heard anything since then.

[1] http://news.ycombinator.com/item?id=452396

mascarenhas2• 17 years ago

LuaForge's ftp server is broken since the big LuaForge crash a few months ago, so those pages are badly out of date. Browse http://github.com/keplerproject for more recent information.

jawngee• 17 years ago

Can't wait for the x64 version.

Love for LuaJIT

Related Stories