Ruby 3x3: Matz, Koichi, and Tenderlove on the Future of Ruby Performance

https://blog.heroku.com/ruby-3-by-3/

by potomak • 10 years ago

215 84 10 years ago

blog.heroku.com

DamnYuppie• 10 years ago

I am not sure it doesn't make sense for them to just move over to JRuby and have them all focus their efforts their.

ehsanu12• 10 years ago

Yeah, jruby+truffle could be pretty amazing for ruby performance.

However, having multiple ruby implementations is great too, and they probably would rather keep working on an independent implementation that they have full control over, and whose goals may not align perfectly with that of other implementations.

dakull7• 10 years ago

As they pointed out in the interview - it's not a free meal i.e. one could expect high memory usage and slow start-up times.

For me the slow start-up and the JIT requiring ~50K/req for warm-up just ruin the whole experience in coding the app in Ruby I'd rather just use Elixir.

Another interesting project is IBM's OMR [0] - compared to JRuby there are no slow start-ups and weirdly, being still a dev. preview it can run Rails and its deps. without issues, even better than Rubinius stable.

[0] https://github.com/rubyomr-preview/rubyomr-preview

sams993• 10 years ago

At Discourse our biggest constraint we are fighting now is memory, not performance (don't get me wrong, I wish MRI was faster). JRuby is usually more memory hungry than MRI and getting reasonable boot time is not easy on JRuby.

Additionally we don't even boot on JRuby today, I wish we did but it is hard to justify effort here.

dakull7• 10 years ago

Did this help in any way? https://tunemygc.com

Just by playing around with RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR you can reduce memory consumption quite a bit.

riffraff• 10 years ago

Additionally we don't even boot on JRuby today

what is the blocking issue there?

ksec3• 10 years ago

Can you elaborate a little, given how memory is very cheap these days.

MrBra5• 10 years ago

I don't think everyone likes depending on Java.

andy_ppp6• 10 years ago

I see no reason to not just move to Elixir at this point for 10-20x performance increase and the tools to build distributed systems at your fingertips.

sams993• 10 years ago

I dunno, maybe cause re-writing a 10 year old working app in another platform is not something that is easily achievable or even desirable.

http://www.joelonsoftware.com/articles/fog0000000069.html

andy_ppp6• 10 years ago

Yes I think that's obviously true; I'm definitely not advocating huge app rewrites if that wasn't clear!

pdimitar3• 10 years ago

I would still advocate migration even of 10-year old apps to new languages. You will learn the language and its frameworks better, you'll enrich your neural paths and many other things will suddenly become clearer, and you might even help your customer succeed even more. Worth the shot most of the time, I believe.

I am currently tasked with maintaining and upgrading a Spree 2.2 (ancient version) and Rails 4.0.8 (very old) based e-commerce shop and even slight modernizations are extremely painful and have proven impossible. The test suite I inherired is sorely lacking in several areas so I can't even rely on that. Most of the problems in the project are related to payments and real cart / order / payment flows and the payment processors don't provide sandboxing environment. Cute, right?

I tried once to upgrade it to the latest Rails 4.1.* and I drowned a weekend in fuzzy and frustrating tasks I can't even remember anymore, only to give up frustrated after what has practically been almost 3 work days fit in a weekend.

My point is, if you want your app not to die by itself (while adding some features and chasing 2-year old bugs that cost your customer money even today) and if you want to modernize it, you _will_ suffer. Not a matter of "if", it's a matter of "how and when".

One of my former customers used to say this about rewrites or maintenance: "Both approaches are gonna hurt a lot so we might as well suffer for the right cause". Granted that's not an universal rule but IMO it's quite on point for a good chunk of scenarios.

MrBra5• 10 years ago

Elixir is not a faster Ruby, it's a completely different beast which only shares some syntactical constructs and keywords.

sotojuan• 10 years ago

Please don't make us Elixir fans look bad.

pdimitar3• 10 years ago

How does he/she do that?

herbst2• 10 years ago

Because some of us might enjoy the ruby stack and are willing to pay the few extra bucks for better servers?

Why does everything have to be black and white anyway.

andy_ppp6• 10 years ago

I'm not sure why the hostility but you are entitled to your opinion; it's really not the performance that makes Elixir better for me (it's everything else - I come from a Python background and I'm just so so tired of the GIL) and I haven't suggested rewriting any working apps in it!

I am well aware that Elixir is different than Ruby, even though it borrows some great syntax ideas!

kondro• 10 years ago

Because if you put a framework as complete as Rails on Elixir it would probably only be marginally faster anyway.

People always seem to compare extremely simple microservices to Rails apps and exclaim how much faster it is in comparison.

andy_ppp6• 10 years ago

I've been wrong on here before but I've never just invented stuff without facts...

https://github.com/mroth/phoenix-showdown/blob/master/README...

rrradical3• 10 years ago

I could be reading the tone of the interview wrong, but it seems like there's a weird sort of anti-intellectualism bent to the Ruby community.

In the middle, there's this exchange:

  Jonan: So you would just use pointers to point to those immutable objects?
  Koichi: Yes. Also, I want to note that immutable doesn't mean frozen object.
  Frozen objects can contain mutable objects. So I mean those immutable objects which only contain children
  that point to immutable objects.
  Jonan: So if we had a nested hash, some large data structure, we would need to
  freeze every object in that in order to reference it this way. Is there a facility in Ruby right now to do that?
  I think I would have to iterate over that structure freezing objects manually today.
  Matz: Not yet.
  Jonan: So there might be?
  Matz: We need to provide something to freeze these objects.
  Aaron: A deep freeze.
  Matz: Yes, deep freeze.
  Jonan: Deep Freeze is the name of this feature maybe? I think that would be an excellent name for it.

So yes, they all agree that Ruby's current 'frozen' feature doesn't grant real immutability guarantees.

But then at the end...

  Jonan: OK. So then, a friend asked me when I described guilds, he writes a lot of Haskell,
  he asked me when we are we going to have "real immutable objects", and I don't quite know what he means.
  Is there some distinction between an immutable object in Ruby and an
  immutable object in a different language that’s important?
  Matz: For example in Haskell, everything is immutable, it’s that kind of language, everything is immutable from day one.
  Jonan: Yes.
  Matz: But in Ruby we have mutable objects, so under that kind of situation we need a whole new construct.
  Aaron: Frozen objects should really be immutable. It's really immutable.
  Jonan: OK.
  Aaron: I don't...
  Jonan: You don't know what this person who air-quoted me "real immutable" was saying?
  Aaron: Yeah I don't know why they would say "real immutable".
  Jonan: Should I unfriend him on Facebook? I think I'm going to after this.
  Matz: At least tell him if you want "real immutable" go ahead and use Haskell.
  Jonan: I think that's an excellent option, yeah.
  Aaron: You just to need to say to them quit "Haskelling" me.
  Jonan: I should, I’ll just tell them to quit "Haskelling" me about immutable objects.

I get that they are joking a bit, but do they really not understand that the Haskeller is referring to the lack of deeply immutable objects? The hostility seems a bit weird to me, given that languages like Haskell push forward our understanding of the usefulness of things like immutability, which then get absorbed into other languages (like Ruby, clearly).

mmahemoff2• 10 years ago

I do think this is the culture of Ruby, appealing to some and not to others.

Ruby is a practical language by design and it's common to see good-natured poking fun at academic concepts that sound good in theory, but fall apart on corner cases (and real projects are full of corner cases).

Case in point: https://twitter.com/dhh/status/216238003817938944

People looking for conceptual purity should look elsewhere.

tinco• 10 years ago

Please don't look elsewhere for conceptual purity. Ruby is absolutely ideal for exploring any concept that can be expressed in object oriented programming, and even many that are expressed in algebra.

Ruby has been designed fundamentally as an Alan Kay style object oriented programming language. In that light in my opinion it is more suitable for conceptual purity than most other scripting languages.

DHH's criticism of SRP has to do with legitimate drawbacks. I think all of us have experienced the over-engineering SRP can invoke when applied naievely. Also note that DHH's framework Rails has made significant steps in enabling SRP for modern Rails applications, so this is not a flat out rejection of the idea.

jdminhbg• 10 years ago

The SRP isn't really an academic concept, it's an industry concept: https://en.wikipedia.org/wiki/Single_responsibility_principl... I'd say it's more of a question of taste and organization.

abvdasker• 10 years ago

What about this seems anti-intellectual? Anti-intellectualism is not the same as having a decent sense of humor. They're obviously joking around and being a bit irreverent here.

rrradical3• 10 years ago

The Haskeller was probably talking about the same 'deep freezing' feature that they mentioned above, right? Except it already has a name...immutability. But even as they were joking, the Ruby folks assumed it was some obscure theoretical feature that they didn't have any use for.

dakull7• 10 years ago

I think that's mostly Aaron being Aaron, it's not necessarily some form of anti-intellectualism but more like just his personality at work.

I don't really have a problem with the tone of the interview but more with how vague the proposed objectives are.

Also, there seem to be some rather misleading facts in it - like the AOT and the 30% speed-up - in practice, after testing it - it helped to load Rails faster but other than that the benchmarks were virtually identical (don't get me wrong, booting Rails faster is great but they imply something completely different in the interview)

MrBra5• 10 years ago

In fact what they said is that AOT only helps in the startup phase, making it 30% faster.

preordained• 10 years ago

The wordsmanship and joking about "real" immutability are in no way unfamiliar stuff to people in other languages, Ruby aside. In Java, it's the same deal, where maybe you can't change an object's fields, but it's fields' fields might be mutable. One layer deep, shallow freeze, shallow copy are actually pretty well understood concepts by "non-Haskellers", as well as the drawbacks and the need oftentimes for 100% deep guaranteed immutability--and the advantages and safety that brings. I think that the joke is that the label "immutability" in languages that don't support it from the ground up--unlike like Haskell--isn't really worthy of the name. But since we already dubbed them as immutable objects (even though they weren't really) you need some extra emphasis for true immutability...super immutable, seriously immutabile, really-really-immutabile, etc.

wasd• 10 years ago

I suspect you don't know many of the Ruby core committers. They're an incredibly jovial bunch. Aaron loves puns and word play, just check out his twitter feed: https://twitter.com/@tenderlove. After rereading it, I can see why their sense of humor may have been lost in transcription.

rrradical3• 10 years ago

That's fair. I got that they were joking, but I guess I didn't know to what extent.

vidarh• 10 years ago

So yes, they all agree that Ruby's current 'frozen' feature doesn't grant real immutability guarantees.

It grants immutability for the object itself. What they are pointing out is that other than fundamental objects like String, Fixnum etc., those objects will usually contain references to other objects (EDIT: any member variables etc., as well as the class, which means the ojbects observable behaviour can also change even if instance variables don't), and those will still be mutable unless they too are frozen.

So we can do immutability in Ruby currently, it's just a pain to do and requires knowledge about the objects - you could implement a generic "deep_freeze" that iterates over instance variables etc., but it would have uninented consequences because it would know nothing about which object references are shared where it's ok to break that by "dup"'ing the object to freeze it (and without dup'ing it, you risk other code that isn't prepared to work with a frozen object breaking)

This is mostly a "problem" because writing Ruby to work on immutable objects would often result in code that's not idiomatic Ruby, and so you won't get many Ruby developers to agree it's a real concern, though the trend is towards freezing more objects.

nick_urban• 10 years ago

They're being ironic. This is part of the Ruby culture in general, and Jonan and Aaron in particular tend to double-down on it. I don't think there's any hostility to Haskell or anti-intellectualism here.

technion• 10 years ago

Half of the Ruby that I write is not web applications - it's scripts that configure servers, import and export data, optimise a manual process, etc. Ruby is perfectly performant in those scenarios.

It's become a horrible meme I can run a script that reformats a 30 line CSV file the way I want it in 0.05s and someone will almost certainly lean over and say "Ruby is too slow, you really need to upgrade to <x>". Or the intranet application I wrote in Sinatra last year for a total of four users, where all I got questions about how it could be usable without "asynchronous".

That said, I don't understand Ruby's performance position. IBM released a preview of their JIT implementation over a year ago, with much fanfare. By many accounts, it held major practical improvements. Since then, I haven't heard a thing about it, and the Ruby team seem to be spending years talking about implementing the same thing.

treehau5• 10 years ago

I really enjoyed Advi Grimm's "Ruby Tapas" videos because many of his examples did not involve web development at all.

omgdlight2• 10 years ago

ruby is really good for solving small problems quickly. Don't get me wrong, proficiency in your shell will go a long way. But when something starts to get complicated I can bang it out in ruby and it's just done.

Web apps do drive ruby's adoption, though. Now that I know ruby I use it for many things, but I learned it initially for rails. Many other languages (like python) can fulfill ruby's scripting usages (I would argue javascript can't - despite its many strengths it's just a little too awkward)

jwdunne• 10 years ago

Matches my use case completely. I haven't built a rails app but learned ruby with a view of doing that but never did (I believe it's important to know the language before the framework - in hindsight, a good decision with Rails specifically).

I like the standard library and I love the gems that fill the gaps the stdlib shouldn't really fill.

Many devs I know reach for the language they use day to day and their scripts and productivity suffer. Ruby is almost like an openly secret weapon in that sense.

misterbowfinger• 10 years ago

I wish Ruby wasn't run by Matz and his team. The number of improvements they can offer is just too little. And they don't do well to get and include feedback from others. It's true that they've been working on Ruby for a long time, and they deserve credit. But it's time for Ruby to grow past Matz and his team.

If Ruby can offer: - real threads - great performance

Then the conversation about Ruby would be about the right things - do you like the syntax, the feel, would it work for your team, etc. We wouldn't be bickering about "performance" all the time.

banachtarski• 10 years ago

The decision to avoid LLVM (for JIT) due to its lack of age/maturity strikes me as extremely odd. I can't think of a piece of technology that has as much universality, buy-in, functionality, and compatibility.

witty_username2• 10 years ago

IIRC LLVM's functionality for JIT is new and/or experimental (patchpoints).

Sure, LLVM can literally compile just in time, but JITs need much more than that.

Ono-Sendai• 10 years ago

LLVM has been able to JIT for years. Some fancy stuff might be WIP but the basic support has been there for ages.

readittwice• 10 years ago

There are some JITs that used (tried to use) LLVM but eventually stopped using it, see HHVM or Apples JavaScriptCore (despite that Apple is heavily involved in LLVM). LLVM is great as a AOT-compiler, but JITs seem to need faster compilation-times for some benchmarks. See this blog for some numbers from Apple: https://webkit.org/blog/5852/introducing-the-b3-jit-compiler...

But yeah, their reasoning for avoiding it was a bit strange. Although I understand them when they say that LLVM as a dependency may be to huge for Ruby. LLVM is huge and the bigger project compared to Ruby with major backers (Apple, Google, etc.). What is important for Ruby may not be what has high-priority for the LLVM project. IMHO Ruby is big enough to warrant its own JIT.

rurban• 10 years ago

LLVM is too big and not good enough.

The only viable options are dynasm, a simple selfmade jit as in potion/tinyrb or rpython. perl6 went with dynasm.

rujit looked good to me, but the tracing overhead was too much. They should really analyze why it was so big, even as tracing jit.

And the datastructures need to be slimmed down, as in potion/tinyrb or php7 or lua/lisps. Only tagging schemes lead to performance. This has nothing to do with the jit. size matters.

dakull7• 10 years ago

This is a really nice read by RuJIT's creator - Masahiro Ide (his Doctoral Thesis [0]).

[0] http://kamome.lib.ynu.ac.jp/dspace/bitstream/10131/9368/1/id...

P.S. this is available via google results, I'm assuming it's public domain

nikic• 10 years ago

We tried to implement an LLVM based JIT compiler for PHP. There's basically two problems: First, LLVM is excruciatingly slow. If I remember correctly, it took more than a minute to compile all the WP functions for a front-page access (this is something like 100k VM instructions translated to LLVM bitcode). Second, LLVM is not great at optimizing the kind of IR that a JIT compiler for a dynamic language emits. It is much more important to perform language-specific optimizations prior to lowering. HHVM basically came to the same conclusion [1]. They lowered vasm to LLVM IR and let LLVM perform additional optimizations and native codegen. The difference was basically noise.

On the other hand, there are projects that do successfully use LLVM for JIT. We now experiment with dynasm, which doesn't optimize, but is fast and much more pleasant to use. libjit looks like another popular option. [1]: http://hhvm.com/blog/10205/llvm-code-generation-in-hhvm

ksec3• 10 years ago

We keep seeing discussion like this and yet we haven't seen much improvement. They used to joke about only those company making money are Rails. So why haven't Github, Shopify, Twitch, Groupon or others pour any resources in ?

Ruby is slow, at least from a Web Development perspective, Rails is VERY slow. So both together combined makes things a little bit bad.

Look at the boring stack from StackExchnage

https://news.ycombinator.com/item?id=12790650

https://twitter.com/Nick_Craver/status/790527231600787456

Any Rails Shop would be happy if they could hit 3x those numbers even with caching.

kivikakk• 10 years ago

So why haven't Github, Shopify, Twitch, Groupon or others pour any resources in ?

GitHub does; Tenderlove works on Ruby there!

herbst2• 10 years ago

Shopify does too. Just look at their Github

dorianm• 10 years ago

For those who want ruby-like syntax and native performance, there is crystal: https://crystal-lang.org/

eropple• 10 years ago

Ruby isn't about the syntax, though. Ruby is about the metaprogramming and what the language allows you to do.

Crystal is neat, but it isn't interesting.

iagooar• 10 years ago

Too little, too late.

Disclaimer: I'm a Ruby developer.

Making Ruby 3 times faster is not going to save it. It's just way, way behind the raw performance of other programming languages. Also, with the arrival of new, modern programming languages, there isn't that many productivity reasons anymore to justify the poor performance and lack of sane concurrency.

I love Ruby, the programming language, but I kind of start seeing that others haven't been idle and there are some growing ecosystems that can soon leave Ruby behind. It kind of feels like Perl back in the day: it was still a good programming language when it started losing ground.

We, Rubyists, have seen a lot of talented people leave both the Ruby and Rails core teams. You see them go to other, younger communities, making them bloom. In the meantime, the Ruby ecosystem seems stale and lacking fresh ideas.

The other problem is that there is not a single company that will throw as much money as it takes to improve the language, as happens with JavaScript, Python or PHP.

I am not trying to say that Ruby or Rails are a thing of the past and shouldn't be used. They are still great tools and put in the right hands can give your company / project huge productivity and more-than-acceptable performance. But there is a time for everything, and I think that the golden years of Ruby are gone.

59nadir2• 10 years ago

The other problem is that there is not a single company that will throw as much money as it takes to improve the language, as happens with JavaScript, Python or PHP.

I don't know if this is really the reason I believe Ruby hasn't evolved as much as other languages. There are no big entities pushing big on Haskell, for example (Facebook uses it, but you don't see them driving development), but GHC evolves like crazy to support more language features, be better optimized, etc..

Isn't it fair to say that a community that loves to say "X isn't <good attribute> but it doesn't need to be!" is just showing disinterest in evolving? There will always be a point if you pat yourself on the back too much about your one big advantage, where you find yourself with increasingly less of an advantage. Your disinterest in improving caused you to stagnate and suddenly there is no advantage anymore.

At some point relative properties like "It has a very broad ecosystem" and "It's the most concise language" don't turn out to be very big advantages or no longer be true, in the case of the latter. There are other examples of this in the clique formerly known as "scripting languages". Most of them can be beaten by implementing new languages on platforms that started with less ideal languages.

indolering• 10 years ago

Isn't it fair to say that a community that loves to say "X isn't <good attribute> but it doesn't need to be!" is just showing disinterest in evolving? There will always be a point if you pat yourself on the back too much about your one big advantage, where you find yourself with increasingly less of an advantage. Your disinterest in improving caused you to stagnate and suddenly there is no advantage anymore.

The same criticism could be leveled at Python with regard to the GIL and JIT (or lack-thereof). CPython lacks these mechanisms because Guido prefers to keep the official implementation simple. When it comes down to it, a language is based on what maintainers want to do.

dakull7• 10 years ago

Making Ruby 3 times faster is not going to save it

Adding something akin to Guilds it could give it a nice advantage if implemented nicely.

the Ruby ecosystem seems stale and lacking fresh ideas

I think that's mostly because it's not the new shiny toy what it's missing is some sort of cohesion to push it forward.

Maybe, I'm just biased but even with languages like Elixir - I keep coming back to Ruby.

RVuRnvbM2e• 10 years ago

Is your experience mostly web dev? Because not all areas of software are so mercurial.

Making Ruby 3 times faster is not going to save it. It's just way, way behind the raw performance of other programming languages.

Like Python? Ruby is comparable in performance.

Also, with the arrival of new, modern programming languages, there isn't that many productivity reasons anymore to justify the poor performance and lack of sane concurrency.

There is no other language that makes it as easy to write a DSL. And there are plenty of places to look for concurrency in Ruby, built right into the language. ruby-concurrency provides even more abstractions.

The other problem is that there is not a single company that will throw as much money as it takes to improve the language, as happens with JavaScript, Python or PHP.

Isn't that company Salesforce/Heroku?

I am not trying to say that Ruby or Rails are a thing of the past and shouldn't be used. They are still great tools and put in the right hands can give your company / project huge productivity and more-than-acceptable performance. But there is a time for everything, and I think that the golden years of Ruby are gone.

I disagree. There are plenty of projects written in Ruby that are still going strong. The language is just maturing and web dev is not the only thing people do with it anymore.

duaneb• 10 years ago

Like Python? Ruby is comparable in performance.

Only on a cursory glance... python has pypy, cython, greenlets, twisted, numpy/pandas/scipy, and dozens of game engines of varying quality. You can write high quality, high performance python code that fits right in with the rest of the community.

Ruby performance is a joke. It doesn't even ship with a useful memory or performance profiler, hence New Relic's existence. You're correct, it's a great DSL language, and performance shouldn't matter at that level.

omgdlight2• 10 years ago

I'm curious to hear your opinion on which ecosystems are likely to replace ruby. I think nodejs and some of the newer jvm langs are well positioned to do well in the web app market, but I can't see what's going to replace it for one off scripts. But then, perl is still being used by some folks for that, so maybe that's where ruby is going.

kyledrake3• 10 years ago

TLDR: There isn't an actual plan to make Ruby 3x faster, it's just a bullshit number Matz made up to be cute. Instead there will be some marginal improvements here and there, as per usual.

Some vague references to the idea of building a JIT are discussed. Rubinius already does this, but they apparently long ago decided that Rubinius doesn't exist, so I guess that's off the table.

juandazapata• 10 years ago

You must be fun at parties.

pwelch• 10 years ago

Ha, I wish I could up vote you twice.

kyledrake3• 10 years ago

You better believe it.

I was at the Rubyconf in Denver sitting behind Matz, and watched him fiddle with his computer and tweet instead of paying attention to Brian Ford's well-informed talk on the deep structural problems within the Ruby development community, which included their indifference to concurrency issues https://www.youtube.com/watch?v=BagNfTbXn3w

I then went one of the talks at the same conference on improving Ruby concurrency for IO that was largely unattended because it was scheduled to run at the same time as an Aaron Patterson talk on literally making sausage.

It's clear to me through observation that the Ruby core devs don't take this topic seriously. And while current performance isn't presently a huge problem for my work, it's clear to me that a lot of people are walking away as a consequence of this.

I'm sorry if bringing this up makes me uncool at your parties or whatever, but if we're going to have a real conversation about this, let's have that instead of just singing kumbaya around a dumpster fire. Matz is out there selling people on the idea that there's going to be a 3x performance increase when there obviously isn't going to be one.

And FWIW, I had drinks with Brian, the other concurrency speaker, and I do recall getting tacos and singing some karaoke with Tony Arcieri. You'll have to ask them if I'm good for parties or not.

Ruby 3x3: Matz, Koichi, and Tenderlove on the Future of Ruby Performance

Related Stories