JIT-optimized Ruby can outperform a C extension

https://railsatscale.com/2023-08-29-ruby-outperforms-c/

by mooreds • 3 years ago

169 106 3 years ago

pmontra• 3 years ago

TL;DR

This is about pure Ruby code optimized by JIT vs manual optimization by writing C extensions (native code). There is Ruby code around those extensions but

The JIT cannot know what the native code will do to parameters (possibly mutate them), it cannot know what type of Ruby object will be returned, if the native code will look at the stack, or if it will raise an exception, etc. Thus the JIT is forced to have a conservative approach when dealing with native code, essentially forgetting much of what it had learned, and falling back to the most conservative implementation possible.

Hence the JIT often makes pure Ruby code run faster than the code with C extensions.

I think I heard the same argument many times with other JITed languages, interpreted or compiled.

xxs4• 3 years ago

Any managed language - java and go included, pays for trips to external, non-intrinsics calls. The standard way to deal is large portions/buffers to amortize the costs. This has been the case since the late '90s

andrewstuart• 3 years ago

"X outperforms C/C++/Rust/Golang".

Only if the code design in C/C++/Rust/Golang is bad.

nvy• 3 years ago

For the time being. But C on x86 hasn't been a good model of what the CPU is actually doing for a while now and eventually we'll have a better way to harness the computing power available to us.

starbugs9• 3 years ago

C was never intended to be a model of what the CPU is actually doing.

This is purely an optimization issue and the article compares apples to oranges for clicks.

throwaway17_173• 3 years ago

I want to be optimistic and agree that ‘we’ll have a better way to harness the computing power available to us’, but at this point it’s been years or maybe decades of propping up the x86 ISA while actual operation of the computing device drifts further away. I certainly understand the desires of processor manufacturers to want to be able to maintain an ISA and then be able to implement a machine that can change underneath it to realize performance gains. However, this has left the status quo utterly entrenched and I can not see what would make them change this behavior.

I really would get onboard with an alternative ISA that was more in line with actual architectural features of modern processors and I often wonder why we could not have an extra compilation (jit or aot) step from x86 to some other format. But the trend seems to be going the opposite direction for all the large scale processor manufacturer/designers toward a more abstracted interface at the lowest level.

I’ve seen various comments that basically boil down to “you can’t change the ISA because then C/C++/Rust programs would loose performance in benchmarks in the short term”, and this resistance only reinforces maintaining current trends. So… what are you assuming happens that prompts the shift toward a better way to harness computing power?

anonymoushn6• 3 years ago

immintrin.h seems tough to beat.

mgaunard2• 3 years ago

C is a pretty good model in part because all ISAs were designed with C in mind.

anonymoushn6• 3 years ago

Golang's compiler is not great. The main way to achieve acceptable performance in golang is to use their weird asm dialect.

umvi• 3 years ago

Golang's compiler is so fast though that I can iterate much faster so I'm willing to accept slightly worse performance than C++. C++ I waste so much time just waiting (esp. if I'm using a lot of templatized header only libraries)

Kiuhrly1• 3 years ago

Somehow I can't find a source for this but it's well known that Go prefers compilation speed over runtime performance. Doesn't help that they implement the entire compiler stack themselves so it definitely misses optimisations that other compilers can already do.

Also, I've never heard of anyone using assembly for performance optimisations. It would never happen where Go in commonly used (server programming), maybe if you're doing graphics or something nonstandard?

anta40• 3 years ago

Perhaps gcc-go is better?

josephg4• 3 years ago

As well as differences in code gen, as I understand it Go also has worse tail latencies than servers written in C/C++/Rust because of the GC.

intelVISA• 3 years ago

Yeah it's not fair to put Go in the same bracket as fast langs tbh.

It's a lang to ship MVPs quickly without much thought & cheaper devs - not a high perf one.

Patrickmi3• 3 years ago

If this was 4 years ago I would say “fair enough” but now Go 1.21 ? This claim is no longer Valid, just like saying CGO is slow, just because one language uses LLVM or GCC doesn’t mean they have state of the compiler

Mawr3• 3 years ago

Not at all. The main way is to reduce pressure on the GC by reducing allocations, which is a fairly straightforward process thanks to good support from the tooling.

Past that, you either call out to C via FFI or give up and rewrite in C/C++/Rust. Nobody really bothers with the asm.

josephg4• 3 years ago

(Edit: my mistake, I misread the comment I was replying to assuming it claimed these languages themselves were equivalent.)

Why would the performance of these languages be identical? These languages aren’t the same in lots of ways.

Golang has a GC and gives you much less control over memory layout than any of the other languages you mentioned. It also has a very different concurrency model, which will vastly change the performance characteristics of multithreaded code.

Rust is limited to llvm, while C can be compiled with many other compilers - some of which produce better code in many cases. (Eg icc)

Rust has much better noalias analysis - which allows the compiler to make optimizations that aren’t available in general purpose C or C++ code. (Since nobody adds noalias to all their variables in C, for a variety of reasons).

Unlike the others, C doesn’t support monomorphization (without disgusting macro tricks). Monomorphization cuts both ways with performance - since it bloats executables (and thus hurts the branch predictor and cache) while generating more efficient code.

These languages aren’t the same. There’s lots of reasons they will perform differently both at the limits of what we can optimize and in the average case of everyday code. It’s wild that you would consider them to all be the same.

PartiallyTyped3• 3 years ago

Rust is limited to llvm, while C can be compiled with many other compilers - some of which produce better code in many cases. (Eg icc)

There's a rust front-end for gcc as well.

starbugs9• 3 years ago

I think this is a somewhat misleading title.

It does not outperform C.

This article appears to compare a suboptimal native C implementation to a JIT optimized version of some similar Ruby code. The JIT only outperforms that particular hand written C implementation in this case.

Still, great that the Riby JIT can do this now, but the title would be better phrased as ‘Ruby JIT in some cases outperforms suboptimal hand written C code‘. That wouldn’t be good clickbait though.

fasterik• 3 years ago

Yeah, it appears that in the original benchmark an off the shelf C parser outperformed an off the shelf Ruby parser. Then they wrote their own parser in Ruby, which was faster. But for a fair comparison, you should also write the same parser in C and compare it to that.

It's possible that writing it in Ruby is preferable if it avoids FFI overhead and can do more aggressive inlining. But this doesn't seem like an apples to apples comparison.

kazinator• 3 years ago

Ruby outperforms C that is a Ruby extension, solving a problem by wrangling with Ruby types.

When you write a run-time for a dynamic, high level language, and then library functions in C which use the API's and objects of that run time, the resulting code is reasonably fast, but never as fast as solving the problem using direct C idioms.

There are ways to make improvements in that type of code, but they tend to be tedious to code. For instance, you can use split the code into cases by type, and then use faster, more type-specific routines.

Sometimes it's possible to get all the inputs out of the objects, work with lower, less encapsulated level C representations to do the bulk of the problem, and then put results back into the run time's objects again. Or do that partially at certain steps.

In the TXR Lisp mapcar implementation, alloca is used to create a native C array of stack-allocated iterator objects. (For the parallel iteration over multiple sequences: mapcar is an N-ary function!). The element values are pulled from all these iterators and stuffed into a stack allocated args structure destructively; that same structure is re-used for each call to the projection function.

That's not as fast as it could be. Beyond that it would be possible to separately handle the case when there is only one sequence argument, and to split that by type, and other such approaches.

It's faster, though, than if it allocated a dynamic vector of dynamic iterators, and/or if it built a dynamic list of arguments in each iteration which were applied to the function as a list.

VeejayRampay• 3 years ago

Hacker News is going to nitpick at the bold claim of the title instead of focusing on the fact that it's a well written technical article that underlines some neat tricks that allow a rather slow, highly dynamic language such as ruby to provide great performance under some circumstances, which is all that engineers should care about

anonymoushn6• 3 years ago

The fastest parser in the post works at 35MB/s which is ~100x slower than necessary, improved from a baseline of ~1000x slower than necessary.

starbugs9• 3 years ago

Critizising the title is not exactly nitpicking. The title is important. And it’s misleading.

I disagree that this article is well written. It is hard to identify what is compared exactly, at least as someone not deeply familiar with the ruby ecosystem. The article doesn’t do a great job of explaining the caveats and limitations of the comparison either.

coldtea5• 3 years ago

Critizising the title is not exactly nitpicking. The title is important.

Only if one doesn't bother to read the article. Else the title is just an insignificant detail, or, at worse, a redundant piece of promotional material attached to the article.

bakugo• 3 years ago

If the writer of the article wanted people to actually read the article and judge it by its overall quality, they wouldn't have used such a blatantly clickbait-y title.

59nadir• 3 years ago

I agree; the title is doing a lot of work in terms of getting people to read the article and I'm sure the author is aware. This is a trade-off and when the article doesn't quite sell on the promise of the title you are bound to get a little backlash.

thepostman0• 3 years ago

Clickbait titles gets criticism because some people won't read beyond the title

starbugs9• 3 years ago

Clickbait title gets criticism, because it makes bold claims drawing the attention of people who when reading the article find that their time was unfairly wasted, because said claims are a pile of ...

Excuse my French.

rewmie• 3 years ago

Hacker News is going to nitpick at the bold claim of the title instead of focusing on the fact that (...)

...that the claims are somewhere between false and cherry-picked?

wodenokoto• 3 years ago

If we are keeping the nitpickings to this thread, I think adding a “when” to the beginning would keep it just as interesting a title and be good enough in terms of correctness

dig1• 3 years ago

The title is misleading, just like other commenters mentioned. Just check how much indirection "rb_iv_get()" has to make (at the end, it will call [1], which isn't "a light" call). Now, check generated JIT code (in a blog post) for the same action where JIT knows how to shave off unnecessary indirection.

We are comparing apples and oranges here.

[1] https://github.com/ruby/ruby/blob/b635a66e957e4dd3fed83ef1d7...

vidarh5• 3 years ago

Yes, it may well be misleading, yet you're making the point of the commenter you replied to while pointing out why the article is interesting. If you're a position where you're using Ruby and considering whether writing a C extension will speed things up, it has valuable advice on what other things to also try. That is a lot more interesting to me than the title.

xxs4• 3 years ago

Talking about technically well written:

When showing microbenchmarks - show the exact hardware used, make sure the CPU does run at fixed frequency (no boost, most laptops can't control the freq. esp Macs)

lelanthran• 3 years ago

Hacker News is going to nitpick at the bold claim of the title instead of focusing on the fact that it's a well written technical article t

I think that's an unfair characterisation of HN readers.

They aren't criticising the article after all, they're criticising a clickbait and misleading title.

asah• 3 years ago

"faster than C" - TBH, I kinda dread these clickbait posts because faster than C doesn't matter in 2023, where speed is dictated by multi-core and GPU.

Python is abysmally slow, but provides access to GPU and (soon) multi-core.

mgaunard2• 3 years ago

Without control, power is nothing.

C gives you control.

ahoka2• 3 years ago

It doesn’t though.

maccard2• 3 years ago

Did you see the post yesterday about the people who deployed a kubernetes cluster because it took 24 hours to parse a few GB of CSVs in JavaScript?

nottheengineer2• 3 years ago

I'd like to read that but can't seem to find it. Do you remember the title or have a link?

dailykoder• 3 years ago

uhm, isn't the CPU access in python written in C (mostly)?

astrobe_• 3 years ago

One should remember that not all problems can be solve faster with concurrency - not all problems are "embarrassingly parallel". Besides, even when they are, one gets diminishing returns by adding more cores. Raw performance on single core is still relevant.

That said, support for concurrency is relevant but more for "masking" I/O latency than for MIPS, a problem which is apparently more common. For a language created after 2010, it is nearly a fault not to have something less primitive than threads to handle concurrency. It is like still manually managing memory manually like C also does.

I agree that "faster than C" claims too often hides some kind of cheating. But we have HN "reviewers" for that ;-).

john_the_writer3• 3 years ago

I don't understand why we have to pretend Ruby could ever be as fast as C. Or faster. That's like saying a jockey is faster than their horse. Ruby is riding on C, so it cannot be faster.

Also why even bother. Ruby is fast enough for the job it does. It's higher level and makes things that are "harder" in C, easier. Dev time is less, and thats where we win. We take the win, and accept what it is.

michaelteter• 3 years ago

I think the point of the essay is that we shouldn’t immediately assume that going to C is the best way to speed up part of a Ruby system.

It might be conventional wisdom to break out to C for the speed gain, but a purpose made, hand written piece of Ruby, with JIT optimizations, and be a better/faster solution.

vidarh5• 3 years ago

Some Ruby implementations are "riding on C". Others are not, e.g. Truffle Ruby.

While beating C overall would be very hard because of the dynamism of Ruby, most Ruby code is not very dynamic after a short warmup (loading for most code; connecting a database etc. for code using a dynamic ORM), and at least on specific code it's at least theoretically possible for a Ruby JIT to beat C by optimising to the actual runtime configuration and evaluate out all the dynamism that costs Ruby performance.

TruffleRuby goes part-way there, in terms of guarded section that makes assumptions about types but where the guards shunts code that breaks those assumptions onto a slow path, but a Ruby JIT or a combination of a bytecode compiler and JIT, can at least in theory go much further. E.g. a whole lot of the Ruby code library contains type checks that a compiler could hoist and/or elide.

Yjit is a baby step, but a good start.

rolae• 3 years ago

People say the title of the article "Ruby Outperforms C: Breaking the Catch-22" is misleading, which is true, this is about Ruby code optimized by JIT outperforming a extension written in C.

But to give some context: the author Aaron Patterson is a Ruby and a Rails core team member. The article and headline is clearly targeting the ruby community, where this article has been very well received. I think it's a good title for the intended audience.

The post clarifies in the first section:

In this post I’d like to present one data point in favor of maintaining a pure Ruby codebase, and then discuss some challenges and downsides of writing native extensions. Finally we’ll look at YJIT optimizations and why they don’t work as well with native code in the mix.

edit: added original title of the hackernews post / article

wonnage• 3 years ago

This is specifically about breaking the myth that performing expensive self-contained operations (e.g, parsing GraphQL) in a native extension (C, Rust, etc.) is always faster than the interpreted language.

The JS ecosystem has the same problem, people think rewriting everything in Rust will be a magic fix. In practice, there's always the problem highlighted in the post (transitioning is expensive, causes optimization bailouts), as well as the cost of actually getting the results back into Node-land. This is why SWC abandoned the JS API for writing plugins - constantly bouncing back and forth while traversing AST nodes was even slower than Babel (e.g https://github.com/swc-project/swc/issues/1392#issuecomment-...)

mananaysiempre• 3 years ago

Interesting that both of the points you state completely contradict my experience with LuaJIT.

Parsing has always been one of the things its tracing JIT struggled with; it is still faster than the (already fairly fast) interpreter, but in this kind of branch- and allocation-heavy code it gets nowhere near the famed 1.25x to 1.5x of GCC (or so) that you can get by carefully tailoring inner-loopy code.

(But a tracing JIT like LuaJIT is a different from a BBV JIT like YJIT, even if I haven’t yet grokked the latter.)

LuaJIT’s FFI calls, on the other hand, are very very fast. They are still slower than not going through the boundary at all, naturally, but that’s about it. On the other hand, going through the Lua/C API inherited from the original, interpreted implementation—which sounds similar to what the Ruby blog post is comparing pure-Ruby code to—can be quite slow.

The SWC situation I can’t understand quickly, but apart from the WASM overhead it sounds to me like they have a syntax tree that the JS plugin side really wants to be GCed in the GC’s memory but the Rust-on-WASM host side really wants to be refcounted in WASM memory, and that is indeed not a good situation to be in. It took a decade or more for DOM manipulation in JS to not suck, and there the native-code side was operating with deep (and unsafe) hooks into the VM and GC infrastructure as opposed to the WASM straitjacket. Hopefully it’ll become easier when the WASM GC proposal finally materializes and people figure out how to make Rust target it.

In any case, it annoys me how hard it is in just about any low-level language to cheaply integrate with a GC. Getting a stack map out of a compiler in order to know where the references to GC-land are and when they are alive is like pulling teeth. I don’t think it should be that way.

danmur2• 3 years ago

I think 'magic fix' could be replaced with 'fun thing to do'

vidarh5• 3 years ago

I think a parser is perhaps the example of where resorting to a compiled extension can be beaten by something more JIT-favourable.

In a language like Ruby it tends to be heavily dominated by scanning text, and creation of objects, and 1) you can often speed it up drastically by reducing object creation. E.g. here is Aaron writing about speeding up the GraphQL parser partly by doing that[1], 2) creating Ruby objects and building up complex structures in the C extension is going to be almost exactly as slow in the C extension as in the Ruby, 3) the scanning of the text mostly hits the regexp engine which is already written in C.

(That said, I heavily favour not resorting to C-extensions unless you really have to; even without going as far as some of Aaron's more esoteric tricks for that parser you can often get a whole lot closer that you think, and the portion you need to rewrite if you still have to might well turn out to be much smaller than you'd expect)

[1] https://tenderlovemaking.com/2023/09/02/fast-tokenizers-with...

ksec2• 3 years ago

to present one data point in favor of maintaining a pure Ruby codebase

Chris Seaton has been stating this for over 5 years. It is unfortunate this mental model has never caught on in Rails.

chucke7• 3 years ago

Evan Phoenix is saying it for even longer, and created rubinius as a means to prove it :) (it has since been abandoned, but Chris's truffleruby ported rubinius core and stdlib implementations, so it's great that they've all been feeding each other for quite a while)

chucke7• 3 years ago

This article outlines well the paradox that JITs require to be truly more efficient: if more of the target language is available to optimize, it'll get waaay more optimized, compared to dropping down to the layer below and try to hand-stitch it.

Of course, there is massive overhead in doing so. Just look at go, which had to rewrite practically everything already available in go, and must always require a native implementation (protobuf for example shares the underlying interface across ruby, python, php... but then has a full separate implementation in go, and java I think). And they have the budget for it at least, Google won't let go die under the overhead it created for itself.

So definitely, write more ruby, enough of those "fast-C gem - rewritten as C extension", but still keep using low level libraries like libpq.

grncdr• 3 years ago

It’s funny you mention libpq, because my first thought upon reading the article was “I wonder if a pure Ruby implementation of the postgres wire protocol could possibly lead to performance improvements?

danmur2• 3 years ago

This pure Python library claims quite fabulous performance: https://github.com/MagicStack/asyncpg

I believe it because that team have done lots of great stuff but I haven't used it, I just remembered thinking it was interesting the performance was so good. Not sure how related it is to running on the asyncio loop (or which loop they used for benchmarks).

whizzter• 3 years ago

Iirc NodeJS started with a libpq binding but it was moved to pure-JS since as in the article, overhead of going in and out of JIT-land penalized it (as well as lacking any object optimizations).

chucke7• 3 years ago

Probably, but you'd have to reimplement a lot of things you get for free out of libpq (prepared statements, pooler support, other things I'm forgetting about). But fwiw, there's already this: https://github.com/mneumann/postgres-pr

vidarh5• 3 years ago

I'm just toying with a Ruby terminal talking direct to X (no X lib) running a pure Ruby TrueType renderer (and running a pure-Ruby editor in it...). Ruby is still not "fast" (but I haven't tested it with yjit yet). I put up with it because I know I can make it significantly faster later (and the quick and dirty first approximation is memoize what I can, like glyph shapes)

As it turns out for a whole lot of things Ruby is slow for people because one of the nice things about programming in Ruby a lot of the time is expressing the problem the more readable way first rather than paying attention to performance. When people are aware, it makes it easy to iterate fast and fix performance later. The problem is a lot of the time that leads to pathological cases of people not thinking this through at all.

You'll notice most of his performance increase came from writing better Ruby. Clearly the original Graphql parser was nowhere near optimal. It's worth keeping in mind that writing a C extension ought to be a last resort after making the Ruby as fast as you can first, because often that means you don't need to.

E.g. another of Aaron's recent articles (EDIT: [1]) is about speeding up a parser by cutting down on object creation, and one specific example he gave was to return just the token type from the lexer instead of [token_type, token_value]. The latter forced creation of an Array object to hold each token and a string object for the token value (though for fixed tokens you could avoid that by returning the same frozen string literal), and for a whole lot of tokens the parser had no interest in the token string (e.g. if you see :lparen, or :rparen, getting "(" and ")" is entirely uninteresting). When people run into slow Ruby code like that it's tempting to resort to C right away, rather than understand why their Ruby is slow.

I love that yjit makes it easier to get to a point where you don't need to reach for C, though.

EDIT: [1] https://tenderlovemaking.com/2023/09/02/fast-tokenizers-with... It's actually about the GraphQL parser mentioned in the original article.

chucke7• 3 years ago

Great point. Not just writing ruby, but writing more performant ruby. I found that we several "here's how to implemented this GoF pattern in ruby" to "thank" for a lot of the unoptimized code we see around.

coldtea5• 3 years ago

Of course, there is massive overhead in doing so. Just look at go, which had to rewrite practically everything already available in go, and must always require a native implementation (protobuf for example shares the underlying interface across ruby, python, php... but then has a full separate implementation in go, and java I think)

Both Java and Go could just as well use FFI for those things "already available" as C.

They don't because native (to the language) is better, and can be even faster (due to GC or lack of crossing the boundary).

Python, PHP and co just take the easy way out. Plus, given their speed, if stuff like JSON and co were native-to-the-language they would be much slower than C extensions.

chucke7• 3 years ago

That's true, but I think that it is due to a few factors: JNI is considered a big penalty in the java, and treated as a "last resort" (libvips is an example); go community has been advocating against C bindings since the runtime was itself rewritten, and treated as an antipattern; protobuf and grpc teams are compromised of mostly googlers, which have more incentives to optimize java and go (it's known that a big chunk of their services are java and c++, and most new stuff is go).

And the rest might as well just use the bindings, as it's probably easier to maintain for them, and low in ggwir priorities lidt. There's actually a pure ruby protobuf library (I believe square employees use and maintain it), but it probably won't ever be merged into google/protobuf for those reasons.

Mawr3• 3 years ago

Of course, there is massive overhead in doing so.

Some concrete numbers for Go:

• Calling C from Go: ~40-50ns [1] [2]

• Calling Go from C: ~100-200ns [3]

[1] "Cgo calls take about 40ns, about the same time encoding/json takes to parse a single digit integer. On my 20 core machine Cgo call performance scales with core count up to about 16 cores, after which some known contention issues slow things down." (https://shane.ai/posts/cgo-performance-in-go1.21)

[2] "In response to the "cgo is slow" questions, this shows that cgo calls are about 50ns on my four-year-old x86 MacBook Pro. Is that fast or slow? It depends on what the cgo call is doing. If it's executing a single add instruction, an extra 50ns is slow. If it's doing something more substantial, an extra 50ns may be nothing at all." – rsc (https://news.ycombinator.com/item?id=36006347)

[3] "Calls from C to Go on threads created in C require some setup to prepare for Go execution. On Unix platforms, this setup is now preserved across multiple calls from the same thread. This significantly reduces the overhead of subsequent C to Go calls from ~1-3 microseconds per call to ~100-200 nanoseconds per call." (https://go.dev/doc/go1.21)

iamgopal• 3 years ago

can numba and typed python will have similar effect ?

KeplerBoy• 3 years ago

sure, in cases where Numba works well, it certainly is faster than a badly written C extension.

It can however not be faster than a well written C extension. They're all just LLVM frontends.

eutropia• 3 years ago

The point here is simplifying the codebase by removing an extra language.

Rubyists like writing ruby, not C extensions. And using the JIT allows us to simplify and keep enough performance.

JIT-optimized Ruby can outperform a C extension

Related Stories