Rust programs versus Go

https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/rust-go.html

by chatmasta • 8 years ago

144 202 8 years ago

benchmarksgame-team.pages.debian.net

CodinM• 8 years ago

whoa

simias4• 8 years ago

I expect Rust to be faster than Go as a rule because of its somewhat lower level nature and cost-less "unmanaged" abstractions but I didn't expect it to be a full order of magnitude faster than Go for some benchmarks. Are the Go snippets not properly optimized or is there something else going on?

I think pitting Go against Rust is somewhat inflammatory but if you look at the Rust vs. C++ benchmark the results are much more similar: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... so clearly it's not Rust doing something right, it's the Go implementation doing something wrong. Actually even Java fares (mostly) better: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

oblio• 8 years ago

The JVM has a ton of work put into it, it wouldn't surprise me if Java is faster than Go for quite a few things.

jon-wood• 8 years ago

It’s less Go doing something wrong and more just that they do different things. Go has a garbage collector which will naturally slow things down somewhat, while Rust handles memory (de)allocation at compile time.

_ph_4• 8 years ago

The problem is, that some of this benchmarks rules prevent the Go solution from using the same efficient code than its competitors, so the numbers are just not valid.

hellofunk2• 8 years ago

Rust handles memory (de)allocation at compile time

You are saying that Rust is doing its runtime memory allocation at compile time? Are you sure you don’t mean compile time management of memory? Two very different different things.

atilaneves2• 8 years ago

I'm still puzzled by the continued belief that a tracing GC will "naturally slow things down somewhat". Why do you think that's the case?

slezyr2• 8 years ago

Rust vs. C++ benchmark the results are much more similar

Not really... Try compare sources of those programs. For example spectral-norm uses OpenMP and other things that make source look really different from the one you would write in real life. Rust program looks much simpler and closer to real life code.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

codetrotter3• 8 years ago

> the results are much more similar

Parent is talking about performance, not source code

v_lisivka2• 8 years ago

C/C++ are cheating by using Intel specific instructions, like _mm_cvtpd_ps. I.e. they are with hardware-specific acceleration. If standard library will be used instead, or if Rust code will be updated to use same instructions, we will see different results.

p0nce2• 8 years ago

If you use a LLVM compiler, _mm_cvtpd_ps just translate to LLVM IR and is not Intel specific anymore.

Const-me5• 8 years ago

That instruction is SSE2. SSE2 instruction set is guaranteed to be supported on all x86-64 CPUs, be it Intel, AMD or VIA.

Similarly, the intrinsics are supported by all modern C and C++ compilers for decades already. Some compilers even generate SSE code by default for code like double x = 42.0 * y, in place of older x87 floating point instructions. x87 code is slower, and has multiple numerical stability issues.

Tuna-Fish• 8 years ago

Explanations for the few largest offenders:

1. regex-redux

Rust has it's own regexp engine, that lacks a few of the bells and whistles of PCRE, but is much faster than it. (And has no sheer perfomance cliffs where a carefully crafted target string or search expression can DOS the app.) Also, I think the rust implementation can do less copying.

GO just uses PCRE. (Which is weird, isn't RE2 a google project?)

2. binary-trees

The benchmark needs to build a massive binary tree. The Rust implementation allocates it's tree in a TypedArena, which is basically a bump allocator for objects that are all dropped simultaneously. It requires very minimal metadata or work per allocation. (Not zero, because it does properly call all destructors.) The GO implementation has to use the platform allocator. The results shouldn't be very surprising...

3. Manderbrot

Not sure, but I think the Rust version is designed for autovectorization while the Go one is scalar.

After that the Go runtimes are within 2.5x of the Rust ones.

Jweb_Guru• 8 years ago

Not zero, because it does properly call all destructors.

I believe that there is an optimization that allows this to be completely avoided for types with only trivial or no destructors.

pron• 8 years ago

The GO implementation has to use the platform allocator.

So does the Java implementation, and still it fares much (3x) better than Go.

burntsushi6• 8 years ago

With respect to your comments about regex, could you please share with me where you got your information? I would love to know so that I can go bug whoever it is to correct it.

Rust has it's own regexp engine, that lacks a few of the bells and whistles of PCRE, but is much faster than it.

The only true part of this is that Rust's regex engine lacks many features found in PCRE. Mostly, these features revolve around things that are either impossible or not-known/difficult to add to a linear time implementation in an efficient way. The most popular features among these are backreferences and various types of lookaround.

However, it is completely unsubstantiated to say that Rust's regex engine is faster than PCRE. At best, you can say that they are competitive with each other, where one regex engine will do better than the other on various types of workloads. Many such cases are not at all related to the feature sets of regex engines, and probably more related to missed optimization opportunities. (Anyone who has implemented a regex engine knows that the size of the set of missed optimization opportunities is unbounded.)

There are of course some of those cases where it comes down to pathological differences. For example, since PCRE uses a backtracking engine even when it isn't necessary (they do have a so-called "DFA" matcher, but it must be invoked explicitly), that means it is susceptible to exponential behavior when Rust's regex engine isn't. But Rust's regex engine has pathological behavior too. The only difference is that Rust's pathological behavior takes the form of large constants, where as PCREs takes the form of exponential growth in the size of the text being searched.

(And has no sheer perfomance cliffs where a carefully crafted target string or search expression can DOS the app.) Also, I think the rust implementation can do less copying.

If you're talking about PCRE vs Rust's regex, then I see no reason to claim that one does more or less copying than the other. I suspect both do exactly as little as possible. Certainly, neither of them will copy the search text.

GO just uses PCRE. (Which is weird, isn't RE2 a google project?)

No. Go has its own regexp engine: https://golang.org/pkg/regexp/

The benchmark showcased in the OP for Go uses PCRE, and that's presumably because Go's PCRE variant is faster than Go's native regex engine. See https://benchmarksgame-team.pages.debian.net/benchmarksgame/... and compare `Go #2` (PCRE) with `Go` (Go implemented regex engine).

Go's regex engine in the standard library is written in pure Go, and was written by the same person that wrote RE2. (I wrote Rust's regex engine, but it was heavily inspired by RE2.) Go's regex engine does not have all the same optimizations as RE2, and in particular, it suffers from some very high constant factors in many more cases than what RE2 and Rust's regex engine. While Go's regex engine doesn't exhibit pathological behavior in the form of exponential runtime in the size of the search text, it also can be handily beaten by PCRE in a number of non-pathological cases.

flavio812• 8 years ago

   binary-trees

   source    secs
   Go        28.80
   Lisp      8.49

You know what I would choose if I needed a garbage collected language.

Btw,the lisp version uses no fancy tree allocator, just plain standard lisp lists.

igouy18• 8 years ago

> GO just uses PCRE

Currently, there are 4 Go regex-redux programs and 2 use PCRE.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

> The GO implementation has to use the platform allocator

… because there doesn't seem to be a Go third-party or stdlib arena allocator.

egnehots• 8 years ago

  simplicity

  source      brain load
  Rust        95%
  Go          23%

simias4• 8 years ago

It's true that performance is not always the main metric to consider but in this case I'm not really sure that Go makes that much of a difference. Take the "regex-redux" benchmark for instance, compare the two versions:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I don't find the code complexity obviously worse for the Rust version, yet it runs a full order of magnitude faster in this benchmark.

weavie2• 8 years ago

Go seems to perform really poorly in the regex-redux. Against both Node and Python Go is way faster (as one would expect) apart from with the regex-redux..

https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Does Go just suck with it's regex performance? Would this be easy to fix by just pulling in a C regex library?

chatmasta3• 8 years ago

What stuck out to me about the regex-redux benchmark is that PHP is faster than both of them! Is that because PHP is effectively a scripting language over C, especially for well-defined tasks like regex matching?

weavie2• 8 years ago

You aren't really going to experience Rusts complexity when your program is essentially just a single function, so it probably isn't really a fair comparison..

Dowwie5• 8 years ago

There's a "matrix moment" where a programmer familiar with Rust no longer sees cascading green characters but rather the program itself.

p0nce2• 8 years ago

How about using a language where you can start at this moment?

codedokode• 8 years ago

Also, in readability Go wins. Because you don't have to put single quotes and HTML tags after every variable.

ghr452• 8 years ago

I think syntax ends up being mostly superficial. The real readability gains come from being able to map().filter().sum() instead of needing 3 for-loops mutating outer variables.

Volt• 8 years ago

Right, you use aboriginal angle brackets instead.

larschdk• 8 years ago

For toy programs you are correct, but Rust guarantees way more than Go in terms of correctness. If you need to reason about, implement and debug such correctness, I don't think the difference is a factor 4. In fact, Rust's proposition is that it makes it easier to ensure (less easy to forget about) correctness.

flavio812• 8 years ago

  more benchmarks

  source    generics
  Rust      100%
  Go        0%

  source    if err!=nil return { nil, err };
  Rust      0%
  Go        100%

  source    undeserved hype
  Rust      100%
  Go        100%

jimmy1• 8 years ago

source: good generics implementations

Haskell: 100%

All Other Languages: 0%

source: flawed generics implementations

Haskell: 0%

All Other Languages: 100%

notheguyouthink5• 8 years ago

Sidenote, appending to that list, my NodeJS "brain load" was insane. When I switched from Node -> Go ages ago it was night and day.

Funnily, in some scenarios, I find Rust to be less of a brain load than Go. In others, Rust is way way worse. As I mentioned in another post, we use Go to get shit done. I hope Rust can take that mantel some day.

pjmlp3• 8 years ago

I try to get shit done, but end up spending my time writing code templates for //go:generate.

11252• 8 years ago

They're sorted by highest delta, which makes it look so significant in the first moment.

Also, though one of golangs target was to get close to c-like performance, they never intended to be the "performance-killer" since the focus is also strong simplicity and productiveness.

More info: https://golang.org/doc/faq#Why_does_Go_perform_badly_on_benc...

timClicks• 8 years ago

Another angle where Rust is doing well is in peak RAM usage. Its memory usage is generally (although certainly not universally!) much lower in these examples.

11252• 8 years ago

Yeah, no doubt! As already written somewhere above, the GC is not cheap.

cmyr• 8 years ago

Nothing in these results should really surprise anyone. Go is a good tool for writing networking services, but it is not (and does not intend to be) a systems language.

For another perspective, look at Go versus Python: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

logicallee2• 8 years ago

to be fair, it was explicitly designed to be a systems language, and its specification says "Go is a general-purpose language designed with systems programming in mind."

This should be removed, and I would say that it gave up on its design goal of being a systems language.

sagichmal2• 8 years ago

"Systems programming language" has multiple, equally-valid definitions. This no-true-Scotsman rules lawyering about language classification is the least interesting possible discussion about a programming language.

yahyaheee• 8 years ago

There’s that kubernetes thing of course ;)

pjmlp3• 8 years ago

The TPC/IP, WLAN, disk management, system update stacks in Fuchsia sound very like systems programming to me.

rqs3• 8 years ago

Go makes writing networking services like a dream for sure, and Rust also seems have it's own plan[0] on network service field. It could be awesome to see they compete with each other and come up something good.

[0] https://internals.rust-lang.org/t/announcing-the-network-ser...

PopsiclePete5• 8 years ago

Rust is still figuring out its "async" story. Go figured it out, and went full steam ahead before the language was even made public - day 0, basically.

I'm not holding my breath for Rust dethroning non-perf-critical Go projects anytime soon.

_ph_4• 8 years ago

I think Go is an excellent systems language, but these benchmarks are not a good measure, for the reasons mentioned by others in this discussion.

lmm• 8 years ago

Go is a compiled managed language; unsurprisingly performance is similar to Java or C# (or, say, Haskell or OCaml). There was a lot of frankly dishonest promotion of it as a C-like systems language for performance, but reality wins out in the end.

pjmlp3• 8 years ago

Given the way it is being used in Fuchsia for writing critical system components like TPC/IP, WLAN, disk management, update infrastructure, it wasn't really dishonest.

C code in the 80's was about 80% inline Assembly on 16 bit machines, so surely there is plenty of room to improve Go's optimizer if they so wish.

Of course Go will never fit into those PIC micro-controllers with 16 KB of RAM.

littlestymaar• 8 years ago

Being “compiled” (which in reality means “statically-compiled”) doesn't give you magical gains you know. It can even be the opposite, since you can't benefit from JIT optimizations.

What makes C, C++ or Rust efficient is not just that they are statically compiled, it's that they have massively-tuned optimizing compiler spending a lot of time in compile-time optimizations. Go's compiler is way less aggressive with optimization because they wants to keep the compile-time low (which is probably a good idea in their niche) and because it has been much less of a focus than for LLVM or Gcc.

Also, the lack of generics and the pervasive use of dynamic dispatch (with interfaces) also prevent many optimization but it keeps compilation fast, which is what Go developers want. (Please note that this is not an argument in favor of Go developers not adding generics to the language, they could totally add generics but keep the dynamic dispatch to keep the compile-time as it is right now).

Dowwie5• 8 years ago

And that's not all! We really need to see memory consumption.

rqs3• 8 years ago

You knew what does the word "mem" on that page means do you :D

Dowwie5• 8 years ago

What mem?

brohee• 8 years ago

Memory usage is hidden on mobile in portrait mode, but displayed in landscape. Od design choice...

Aissen4• 8 years ago

Anyone has tried modifying the go regex benchmark to use https://github.com/BurntSushi/rure-go ? It seems to use a library that isn't even on github anymore.

ysleepy2• 8 years ago

"Go bindings to Rust's regex engine." - I't would not be in the spirit of the comparison. - Even though most scripting languages also have native regex implementations.

burntsushi6• 8 years ago

The fastest Go benchmark for regex-redux is using PCRE, which is not a native Go regex engine. It's written in C.

burntsushi6• 8 years ago

I had fun with this:

    [andrew@Cheetah benchgame] time bench-native < /tmp/input5000000.txt
    ...
    real    0m20.709s
    user    1m19.839s
    sys     0m0.183s
    [andrew@Cheetah benchgame] time bench-pcre < /tmp/input5000000.txt
    ...
    real    0m10.510s
    user    0m35.362s
    sys     0m0.211s
    [andrew@Cheetah benchgame] time bench-rust < /tmp/input5000000.txt
    ...
    real    0m2.235s
    user    0m3.410s
    sys     0m0.157s

For comparison, this is the timing for the Rust program, unchanged from the benchmark site:

    $ time ./target/release/bench-rust-regex < /tmp/input5000000.txt
    ...
    real    0m1.452s
    user    0m2.643s
    sys     0m0.158s

See https://gist.github.com/BurntSushi/9d35258444fda83de208d31fd... for source code and full results.

And no, I won't submit this myself, although I would find it absolutely hilarious if someone did and managed to get it in. :-)

The PCRE version is definitely sub-optimal. I don't think it's using the JIT, for example.

Enabling SIMD optimizations in Rust's regex crate shaves off another 13% (for Rust program) and 9% (for Go program). The SIMD optimizations will be enabled by default in Rust 1.27 for CPUs that support them.

Aissen4• 8 years ago

Awesome work, thanks !

boyter• 8 years ago

Not really that surprising. I spent a good chunk of time trying to make a Go program as fast as a well written Rust one (tokei) which I wrote about here https://boyter.org/posts/sloc-cloc-code/ and in the end the GC was what held me back.

I suspect its possible to get Go programs for the most part close to the performance of Rust but in the end the lower level and fewer abstractions mean a well written Rust or C/C++ program will almost always be faster.

I wonder how different these benchmarks would be if the Go runtime was set to have GC disabled though. A fairly simple thing to try out and see.

ashleyn• 8 years ago

GC at the systems level is an idea that has had its chance. 40 something years out, and we all mostly use C, with Rust being the only contender to knock it off its perch. It's probably time to give up the dream; GC is only good if you're willing to assume the always-moderate cost of its implementation, relegating it to scripting languages.

PopsiclePete5• 8 years ago

Go doesn't really play in the "systems level" field all that much, tho. They even removed the "systems" bit from the Go home page.

Go doesn't compete with C, unless C was the wrong choice to begin with. Nobody is writing an OS in Go. It competes with Python and Node.

I've reduced my Python/Node code by roughly 92.5% after Go reached 1.0 and I couldn't be happier.

ysleepy2• 8 years ago

I would be interested in results of go-llvm - measuring some proxy of the intrinsic overhead due to the design choices of go vs. rusts focus on zero-cost abstractions.

The clean-room compiler backend of go has some usability advantages, but sacrifices the sort of comprehensive optimization that llvm does.

jerf3• 8 years ago

I haven't heard anybody claim that go-llvm is much faster systematically. I'm assuming that I would have by now. Similarly, the gogcc implementation doesn't seem to have any speed advantages, because again, I assume I would have heard of them by now.

The Rust team has some really interesting blog posts about how you can't just wave LLVM at a program and expect the optimizer to work magic, but that you need to process the code for LLVM first, such as this one: https://blog.rust-lang.org/2016/04/19/MIR.html Bear in mind that not only is the little paragraph about LLVM optimization relevant to my point here, the entire blog post is ultimately about the things the Rust compiler can do that the LLVM optimizer could not; if LLVM just optimized the code magically perfectly, MIR either never would have happened, or would have happened later because the pressures to create it would be less if LLVM was already as optimal as possible.

dullgiulio2• 8 years ago

Oh no, not The Benchmarks Game again :(

This whole thing is meaningless and deeply flawed.

The sources of the various solutions vary a lot in quality: different libraries are used, different settings (in Go vs Java the use of threads is almost never the same).

I suspect the only true end of this benchmarking exercise is to make people comment vigorously on HN and similar forms.

Can we stop posting them?

vonseel• 8 years ago

suspect the only true end of this benchmarking exercise is to make people comment vigorously on HN and similar forms.

I suspect you are right and some of the posters suffer from “watching the likes roll-in” addiction

bananarepdev• 8 years ago

Probably going to get downvoted for that, but could you recommend me some similar forums? :)

dx034• 8 years ago

Reddit can be decent IMO (e.g. reddit.com/r/golang)

chatmasta3• 8 years ago

I’m the OP. I figured this post would either get tons of upvotes or flagged off the first page. The title really is a firebomb on HN lol.

But honestly I find it genuinely interesting. I’m planning to take some time to learn either rust or go this month, so comparisons like this are helpful for me. And I also appreciate the contrarian comments and discussion of the merits (or lack thereof) of the benchmarks.

giancarlostoro2• 8 years ago

I used to like Go a bit, but never did a full blown application in it, just did enough to get a feel of it. Having tried Rust recently (after not liking the idea of not having classes like in Go) I have to say I prefer Rust for several reasons.

For starters the tooling for Rust is well done. If I want to install Rust on any machine I'm on I just go to https://rustup.rs/ and go from there, then I can use rustup to update my tooling or add features to the compiled.

The other thing is the wealth of packages[0] that expand upon Cargo (Rusts build system and package manager) namely the cargo-deb package which lets me build my Rust projects directly into a Debian package.

Continuing on about tooling I have to mention that for IDE's / Text Editors: CLion w/ Rust Plugin is fantastic, but if you want something fully free the VS Code plugin is also amazing for Rust. There's a great effort towards building the Rust Language Server, which thanks to Microsoft is a generic spec that once implemented for one language can be used by any editor / IDE that supports langserver[1].

I will say depending on what you want to do with Rust there can be a bit of a learning curve. I'm still just bleeding my way through some parts because I don't know what everything means in Rust just enough to try different things.

I will say Rust is nowhere near where it could be, but it's at a stage where I think it's definitely usable. They definitely took their time to make sure they got things right, and they got plenty right.

[0] https://crates.io/ [1]: http://langserver.org/

notheguyouthink5• 8 years ago

I really, really like Rust. Yet, I use Go.

This is mainly do to firstly, Go just fits way way better for my shop. The transition from my fellow employees from Python to Go is pretty straight forward. Secondly, Go is just nicely setup for ease. I had so much hell in Rust cross compiling a simple https server.. it was mind numbing. It might be better now, but I was shocked at how difficult it was.

Finally, Go is just more nice to write in when you're.. well, writing it out. I guess I tend to write as I think about data structures, abstractions, general design. I don't draw it ahead of time or w/e, I just write code. Go is very minimal, and is pretty forgiving to quick rewrites/etc. In Rust I found myself writing myself into a corner constantly when all I was trying to do was get an idea out of my head.

With that said, I vastly preferred Rust's feeling of knowing exactly what my program is doing. No question on if something is nil or not, you know it is or isn't (or know it's unknown haha).

I think for me, in another 2 years I'll push again for Rust > Go in my shop as I imagine the Rust tooling will have improved to the point where it's easier and more quick to write things. Rust was experiencing a lot of cargo package churn, I spent hours debugging which packages worked together, it was a headache. Even understanding errors was a bit of a headache, despite the compiler trying really hard. Rust is making great strides though, and even though I use Go to get shit done, Rust is probably my favorite language.. that I don't use heh.

edit: Oh yea, and green threads, god do I wish Rust had green threads. I found Rust to be similar to Go in multithreading ergonomics, but I couldn't use them in the same manner as with Go - ie, fire off as many as I want. There are several libraries in development that should take care of this though, I believe.

skywhopper2• 8 years ago

I disagree that the comparison as presented on this page is useful for you. If there were any context or discussion of the results and what they mean in terms of differences between the languages, then maybe it could be. But raw benchmarks collected with no indication of why or how is not. In the regex test, for example, all the patterns are very similar to each other and don’t really cover the breadth of what a regex engine can do (specific patterns may influence the results greatly—there are some very simple patterns that send the default Java regex engine into O(n^n) fits). And regex in particular is a very well understood problem space. So something very interesting is going on there for the golang result to be so far off. What is happening? Well, we’ll never know from this page. Oh well?

PopsiclePete5• 8 years ago

No reason to learn Rust or Go - learn Rust and Go. I use Go for most "network service" type stuff at my job. I would probably not choose Rust for those, even if I was as comfortable with it as I was with Go (which I'm not, since Go is much, much simpler).

There's many variables when considering what language to pick for a problem and if speed was the only thing we cared about, we'd just do raw assembly always.

some_account• 8 years ago

Benchmarks are important actually. Maybe they are not 100% accurate but they do tell you that rust is faster than go. Since rust is also more complex than go, you have to decide if it's worth complexity and learning curve for the speed.

Without benchmarks, we all would use python for everything right? :)

_ph_4• 8 years ago

Well, no. Some of the presented benchmarks prevent the Go solution to be as efficient as the Rust one by their rules, e.g. prohibiting object pools in Go, but not in other languages. Same with hash tables. They benchmark the default Go hash tables vs. custom implemented ones for the benchmark in C.

vanderZwan6• 8 years ago

Yes, benchmarks are important, which is why we should use proper ones. The Benchmarks Game is not one of those.

igouy18• 8 years ago

> in Go vs Java the use of threads is almost never the same

Seems like you might be suggesting the Go programs don't use multicore but the Java programs they are compared against do use multicore, or vice versa, or …?

Currently just 1 comparison (reverse-complement) shows a sequential Go program and a threaded Java program.

tw1010• 8 years ago

Sometimes the most significant aspects of something is unmeasurable.

_Codemonkeyism• 8 years ago

Why is Java faster than Go?

cryptos• 8 years ago

Why should Go be faster? Both languages use a garbage collector and Java is highly optimized. Plus Java can be optimized even further at runtime.

glangdale4• 8 years ago

Because both are very similar languages and Java is way more mature? Go is essentially "Java, from Bell Labs".

diakritikal• 8 years ago

it's not: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

mrweasel• 8 years ago

Java, or more specifically the jvm have had some of the worlds smartest developers tweak every corner of the code for 20+ years. Java is faster (in some cases) because it's older and have been tested in pretty much any scenario... And it remains wildly popular, so development never stopped.

Similarly Go and Rust are faster than Java in many cases, because the developers have been able to learn for past language like Java, and avoid the same performance pit falls. Or simply because they didn't have to deal with legacy support.

Thaxll4• 8 years ago

Java is not faster than Go.

Edit: Why I'm beeing downvoted, this very much benchmark linked show that Go is faster than Java in most scenarios: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

benmmurphy• 8 years ago

if you look at how the benchmarks run i suspect they would disadvantage java. java is known for having slow startup times and also an optimising JIT that takes a while to kick in. go is definitely going to be faster for some command line application that takes 20s to run but if you have a long running process i think it will be much closer.

cdoxsey3• 8 years ago

These benchmarks have many flaws.

Consider this thread: https://www.reddit.com/r/golang/comments/51mhzv/on_the_binar...

The binary-trees benchmark would fare significantly better in Go if it were allowed to use an arena allocator like other implementations. But its not.

This despite the fact that the Rust version literally uses one: https://benchmarksgame-team.pages.debian.net/benchmarksgame/....

I guess because its not in the stdlib in Go?

Course its not in C either, so does C use malloc? Nope. It uses a memory pool too: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Its sad that these benchmarks get used for anything. They tell you next to nothing significant.

dvfjsdhgfv3• 8 years ago

I guess because its not in the stdlib in Go?

It's a game, and the sources are available. Nobody is stopping Golang enthusiasts from posting the results with a slab allocator or some other third-party packages.

cdoxsey3• 8 years ago

Yes the maintainer is. People have tried.

masklinn• 8 years ago

Nobody is stopping Golang enthusiasts from posting the results with a slab allocator or some other third-party packages.

You've never witnessed the "game"'s maintainer interacting with people have you?

glangdale4• 8 years ago

There are many predictable things in life: the tendency of up-and-comer languages to blitz these benchmarks due to careful optimization, and the tendency of people whose pet language has lost on a benchmark or two to get on here and write off the idea that anyone should ever benchmark cross-language.

Some of the benchmarks are pretty ridiculous and flawed. Others look pretty straightforwardly comparable. I have trouble looking at n-body (just fr'instance) and seeing any major difference or cheat on the Rust side (or for that matter, the much more mature C/C++ sides). Leaving aside the obviously ridiculous (the regex one is a absolute festival of 'who brought a fast regex library to the gunfight'), the trend is pretty clear: Go's code generator is pretty ordinary (it's not actually aimed at achieving ultimate code quality at any compile-time cost, so it's working as designed) and GC isn't free.

cdoxsey3• 8 years ago

I find microbenchmarks mostly useless. There are just too many variables in play.

Also they tend to focus on the developer who optimizes code, not the journeyman developer in your company who actually writes code. C or Fortran looks fantastic on benchmarks, but if you forced all your developers to use C I doubt you'd see a similar outcome.

They seem to push developers in the direction of coming up with simple rules of thumb like "Go is 2x slower than C" or "Python is 10x slower than C" which is complete nonsense. It really matters what the code is doing.

In particular for server programs developers come to these conclusions without considering the multi-core environment they're working in. ie, redis is a really fast key-value store that will use 1 core on a 32 core machine.

And as it turns out all the tips and tricks to improving performance in these cases works in all the other languages too. So just what are we to conclude from them?

I suppose a nice simple math benchmark can demonstrate that LLVM or GCC is better at optimizing code than Go?

igouy18• 8 years ago

> the regex one is a absolute festival of 'who brought a fast regex library to the gunfight'

Yes performant libraries are important.

kevin_b_er• 8 years ago

So, we're basically looking at the difference between malloc and an arena allocator?

riku_iki• 8 years ago

I guess because its not in the stdlib in Go?

I think the premise of these specific benchmark is to use language idiomatic approaches instead of hacky optimizations. Is arena allocator idiomatic for go? What library people usually use? I tried to do google search, and found this one, but it is last updated 3 years ago.. https://github.com/couchbase/go-slab

steveklabnik12• 8 years ago

A while back, there was some stuff about Rust and HashMaps and things. It ended up with this comment: https://www.reddit.com/r/rust/comments/5rwwrv/chashmap_effic...

I don't know if that policy has changed in the last year, but last I heard, that's the rules regarding this.

My understanding is that the intention is to use what's usual in each language.

igouy18• 8 years ago

> Go if it were allowed to use an arena allocator

Go is allowed to use an arena allocator.

> I guess because its not in the stdlib in Go?

A popular third-party library implementation would be fine.

For example, couchbase/go-slab except that it doesn't seem to be a live project and there's no indication that it is actually used by the Go community.

> so does C use malloc?

C using malloc -- https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

skywhopper2• 8 years ago

This would be interesting if there were any comparison of what exactly is slower or faster between the languages or the particular implementations. This data provides a great opportunity, but it's just presented as a dumb contest.

igouy18• 8 years ago

> This data provides a great opportunity, but it's just presented as a dumb contest.

Are you going to do-the-work to take that great opportunity?

dvfjsdhgfv3• 8 years ago

The results don't surprise me much, Go isn't that fast. Much more interesting are the results of the Rust vs C++ benchmarks: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... - it looks like in some cases Rust is actually faster than C++, and that's a big win for me.

Also, it would be great to see some Rust vs D benchmarks.

igouy18• 8 years ago

> Also, it would be great to see some Rust vs D benchmarks.

Make those measurements and publish them!

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

antirez• 8 years ago

Apples to Oranges, and in the benchmarks involving Apples to Apples, that is C vs Rust, the binary tree test is 2x slower in Rust which is tragic.

igouy18• 8 years ago

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

gekkonier• 8 years ago

Ah nice, a benchmark.unwrap()!

peteretep• 8 years ago

In before a specious and infinitely malleable definition of "systems programming language"

Tade0• 8 years ago

Even though Rust apparently "won" here(does anybody really win in such comparisons?) I don't care that much about it, because I don't consider this the main appeal of the language.

It's the compiler error messages, memory safety and the lack of a Garbage Collector that speak to me more than raw performance.

PSeitz2• 8 years ago

Garbage collector is actually something you'd want, if it caused no performance hit.

atilaneves2• 8 years ago

Depending on the GC and what the particular definition of "performance" is for your use case (Memory usage? Throughput? Latency? ...), they don't cause a hit. It depends.

For instance, if you never hit the GC memory limit in D, it's faster than manually managing memory since you never free it.

simias4• 8 years ago

Meh. I might have a huge Stockholm syndrome with manual resource management since I've basically started coding with C and used GC-free languages for most of my life since them but I never really understood why people liked GCs so much outside of scripting languages where convenience trumps correctness and you just litter your resources around instead of tracking them properly.

Are people using GC languages really often thinking "uh, I don't know when I won't be needing this resource anymore, thankfully the GC will figure that out for me"? Because I certainly never do. Either the resource is scoped and I expect it to be destroyed at the end of a certain block RAII-style, or it's some longer-lived resource (connection handle, object in a videogame, Window object in a GUI) that I'm going to store in some container and destroy when I no longer need it. If I have multiple references and a complex dependency graph I might use a reference counted container which is effectively a very simplistic GC but it's the exception and not the rule and it's still completely transparent and easy to reason about.

I never, ever, feel like I'm missing a GC when I'm coding in Rust or C++. In C I miss destructors and RAII but that's it. I just don't understand why they're so popular, at best they save you a few lines of cleanup code in your destructors, at worse they make your code behave sub-optimally and non-deterministically because you don't know when your objects are going to be destroyed and your destructors are going to run. Then if you need to write critical code you have to carefully step around the GC to be sure that your perf will be deterministic.

I genuinely don't get GCs.

kjeetgill3• 8 years ago

There's more to it than that. Having a garbage collector can be faster even, but would you take a faster program if it cost you memory footprint or predictability? It just depends on what you're doing.

notheguyouthink5• 8 years ago

Even though Rust apparently "won" here(does anybody really win in such comparisons?) I don't care that much about it, because I don't consider this the main appeal of the language.

Completely agree. For me, my primary consideration is developer sanity. A subtle compromise between speed, ease, and clarity. I don't program in Node or Python anymore because while it has ease, clarity is completely missing. I used Rust for 6 months or so and while it has speed and clarity, the ease was just not there for me. Go hits a nice sweet spot for me.

I think if Rust managed to improve ergonomics more and more it would really nail the sweet spot. But, there's still work to do.

stunt• 8 years ago

Don't forget GO is not low level langauge. it is fast because it is compiled.

Not sure why there is argument against it. I thought it is obvious for everyone.

Rust is lower level, gives less abstractions, and designed for performance.

Rust performance is comparable with C and C++. That was the whole idea.

PyroLagus• 8 years ago

I wouldn't say that it gives less abstractions. The Rust devs are obviously careful about adding abstractions that add a performance cost, but Rust has quite a few abstractions that Go doesn't have; generics being the most notable one.

acaloiar• 8 years ago

Before getting heated about the meaning of these results, remember that benchmarks are good for one thing: measuring performance.

Performance is important, but there is laundry list of other factors that programming languages optimize for. Because factors such as development pleasure, mean time to programmer error, and mean time to grok code are not easily quantified and compared, it's difficult to put two languages head-to-head on these often more important factors.

These results clearly show how Rust and Go compare on a small set of performance problems. And if you're trying to solve any of these problems using either of these languages, and performance is what you're optimizing for — Rust is a clear winner. I wouldn't try to extrapolate these results to mean anything more than that.

Thaxll4• 8 years ago

This week there was a popular benchmark about implementing a treap in various languages, Go was as fast as both Rust version and c++ with shared pointer. It was even faster than those languages on some other platforms.

So yeah benchmark ...

https://github.com/frol/completely-unscientific-benchmarks

kibwen• 8 years ago

What's the point of posting this? Rust and Go are only "competitors" in the sense that every language competes with every other. You might as well post the page comparing Go to Fortran or Rust to Clojure. Rust and Go have different objectives, philosophies, tradeoffs, and intended domains. Posts like this seem like they serve no purpose other than to inspire flamewars. And I say this as a heavy Rust user, so don't think that I'm embarrassed by these microbenchmarks. :P

igouy18• 8 years ago

> You might as well post the page comparing…

They already have! :-)

thefounder• 8 years ago

It would be interesting to benchmark the time required to write these programs as well.

borlum• 8 years ago

Link to live dashboard of Rust vs Go activity on GitHub:

https://demo.humio.com/shared/dashboards?token=g79uTaUrFY9ky...

Rust programs versus Go

Related Stories