Comparing Go and Java, Part 2 – Performance

http://boundary.com/blog/2012/09/17/comparing-go-and-java-part-2-performance/

by spahl • 14 years ago

70 82 14 years ago

spiffworks• 14 years ago

Is this right? Java really has better latency than Go right now? And with 2 processors, Go's latency is 25ms for 50 concurrent requests. Anybody with better experience care to comment on this?

georgemcbay• 14 years ago

I have no idea if it is "right" since I'm not going to duplicate this exact benchmark, but it seems likely. Raw performance hasn't been a huge focus for Go, though this has been changing. The tip version produces much faster code than the last official release that seems to have been used here. Also you often get more efficient Go from the gccgo frontend than from the Google compiler since it can take advantage of a lot of front-end-neutral optimizations that gcc already has (though this gap is closing as the Google compiler gets improved). For the runtime, work continues on the goroutine scheduling, gc, etc.

Go is getting a lot faster and pretty quickly.

eternalban2• 14 years ago

1 - There has been a lot of man years put into the JVM (threading and garbage collection), and Java memory model. I'm somewhat surprised by the magnitude of the difference. Java and JVM are very mature technologies.

2 - Go's CPM paradigm is not necessarily more performant than the preemptive threading model of Java, and I don't believe that has ever been claimed for CPM. It is claimed that it is "easier" to write concurrent code using the (CPM/Go) message passing paradigm, but in my experience, you either get concurrency (and can do both variants ok) or you don't, in which case any real world concurrent system will hardly be "easy". That said, for the concurrency novice, Go is far less intimidating experience given the language level support for goroutines (fibers), channels, selectors, etc. (think Java NIO ...)

cgh• 14 years ago

There's still this lingering belief that Java is slow, maybe related to the terrible start-up times of the JVM. But once it's going, Java is fast - really fast. And since its niche is basically enterprise webapps, a lot of attention has been paid to scalability and concurrency.

pjmlp4• 14 years ago

A little known secret is that most mainstream languages have nice libraries to handle multi-core programming that go beyond basic thread handling, similar to what Go offers.

You just have to know where to look for, plus you don't need to throw away mature languages and tooling.

erichocean• 14 years ago

For the JVM, Akka is pretty sweet. It comes with both Java and Scala bindings.

http://akka.io

sitharus2• 14 years ago

That seems reasonable to me - Java's had decades of optimisation, but Go's a fairly new language and runtime.

I've just started playing around with Go, I'm quite liking it. It's a different approach to things, which is always fun.

leothekim• 14 years ago

Agree. I've been playing with Go a bit, and though I still lean towards python and scala, Go definitely shows promise. The interfaces model and channel synchronization primitives are compelling enough to keep me interested in its progress.

koenigdavidmj2• 14 years ago

It seems like it might be a fun project to compile Go to Java bytecode. GCC might actually have all the relevant bits---it includes gcj, a Java compiler including bytecode generator, and a Go frontend.

Knowing this, someone's already done it---anyone aware of such a thing?

durin42• 14 years ago

http://code.google.com/p/jgo/ but I have no idea how complete it is, I only know it exists.

strlen4• 14 years ago

The big pull of go for me is AOT compilation to native code (which translates to fast start time for command line utilities), close integration with the OS,support for value types, and stack allocation.

These are not currently available on the JVM (although this may change as of JDK 8).

seunosewa• 14 years ago

This could be cool, but it'll be tough to simulate Go features like lightweight threads and cache-efficient object layout in RAM.

ww520• 14 years ago

Java came out at in 96. GO has been out for couple years ago. The difference makes it far from decades.

sitharus2• 14 years ago

Decade and a bit then?

RobAtticus• 14 years ago

So Java has had 1.6 decades of optimization. He didn't compare the difference, just what Java has.

mritun• 14 years ago

I'd make a guess in absence of code.

Since none of the implementations were IO bound (unsaturated link), I'd bet on memory management as the deciding factor (it's hard to assume that an authentication service will be CPU bound).

Then it's not difficult to see why Java won. When it comes to garbage collectors, JVM has the best (production) implementation of GC bar none.

Go runtime has a lot of catching up to do.

stcredzero• 14 years ago

I would like to see Ulterior Reference Counting, or age-based hybrid GC for both.

http://www.powershow.com/view/14655a-MGM5M/Ulterior_Referenc...

bdcravens• 14 years ago

First paragraph: "In Part 1, we looked at the code of two web services that implement an authentication web service. One written in Java, and one written in Go. It’s time to beat up on them a little bit."

http://boundary.com/blog/2012/09/13/comparing-go-and-java/

masklinn6• 14 years ago

And the code repo: https://github.com/collinvandyck/go-and-java

luriel5• 14 years ago

Your guess is probably quite accurate, is also probably mostly benchmarking the quality of the PostgreSQL drivers, the Go ones are good, but I'm sure there has been much more work in optimizing the Java ones.

dchest• 14 years ago

Some notes from looking at the Go code. I don't know how important are those for performance, more like style nits.

Java version doesn't do type conversion: https://github.com/collinvandyck/go-and-java/blob/master/jav...

   rs.getString("id")
   rs.getBoolean("admin")

Go version does conversion at runtime, as row.Scan() accepts interface values.

Also, the usage of sql interface is not optimal (at least with regards to code length) -- since the query is for one row, why not use QueryRow instead of Query?

Authorization header decoding is strange -- it creates new base64 decoder, then string reader, then decodes. Why not just use base64.StdEncoding.DecodeString() and get rid of a few lines of code and a few allocations?

https://github.com/collinvandyck/go-and-java/blob/master/go/...

Similarly, JSON marshals into a newly allocated slice instead of creating a decoder and then marshaling directly into ResponseWriter.

https://github.com/collinvandyck/go-and-java/blob/master/go/...

Constants for HTTP error codes, that are declared in net/http, for some reason are redeclared

https://github.com/collinvandyck/go-and-java/blob/master/go/...

- - -

I sometimes wonder why various benchmarks include colorful charts, but fail to include a few lines from a simple run of profiler. It's so easy to do, and yet nobody bothers to learn where the cycles are actually spent!

reddit_clone3• 14 years ago

Does anyone else find 4 lines of error checking (log and panic) boiler plate code for every line of functional code a bit tedious?

eta_carinae• 14 years ago

Annoying as hell, that's what you get when you don't support exceptions.

eternalban2• 14 years ago

http://play.golang.org/p/WUxODxK_4r

masklinn6• 14 years ago

> that's what you get when you don't support exceptions.

That's what you get when you refuse to use them anyway.

Go has exceptions, there's just a dogma about never using them.

Evbn• 14 years ago

You can ignore errors if you want to crash, just like Java.

wonderzombie3• 14 years ago

Yes, exactly. If you find the boilerplate tedious, feel free to ignore errors completely. :) For test code it doesn't matter. It is the equivalent of providing no exception handler, or catching Exception or Throwable and doing nothing.

shanemhansen4• 14 years ago

It's annoying, the only thing it has going for it is it's better (in my subjective opinion) than the alternative.

In go you can happily ignore an exception by assigning it to _. That's probably a bad idea for a larger piece of code, but for little scripts, go for it.

I may be already permanently damaged by go. Every time a function can return an exception I have to stop and ask myself: "self, what should you do if this happens?". I'm starting to think it results in better code. Granted, not every shell script/utility benefits from this level of introspection, but that's what python's for I guess.

p.s. Anyone who's had to deal with checked exceptions in java puts up with a crazy about of boiler plate too.

p.p.s. Disclaimer I have been professionally employed writing all languages mentioned above, so hopefully I'm relatively unbiased.

masklinn6• 14 years ago

> It's annoying, the only thing it has going for it is it's better (in my subjective opinion) than the alternative.

Why? There's nothing necessarily wrong about letting it crash, and letting a layer above report the error cleanly. Hell, in Erlang the usage is even to let an other process entirely handle the error.

In fact, my opinion would be the complete opposite of yours: checking every single return value (if only to return it to the caller unaltered) is fine for little script, but it's a bad pattern to need for large pieces of code, it's verbose, redundant and unhelpful.

stickfigure2• 14 years ago

I have invented a remarkable new programming tool that wires up to your chair, keyboard, and 110v AC. It gives you an electric shock every time you complete a line of code, reminding you to stop and think about it. Think of the productivity!

Checked exceptions are indeed obnoxious and a major language design failure, which is why pretty much every modern language since just has plain old (non-checked) exceptions. And even in Java, you can work around the brain damage by wrapping checked exceptions with runtime equivalents in API facades. Exceptions are still incredibly useful, and lack thereof is my biggest complaint about Go.

I'm also disappointed by the convention of capitalizing/lowercasing names to export them or not. Realize that you want to export an existing private method? What's that, your IDE doesn't support refactoring? Get typing, you have a lot of method calls to update.

KevinEldon3• 14 years ago

The article mentioned DropWizard (covered more in part 1). DropWizard is a very nice (the best?) way to write RESTful web services in Java. Coda Hale, that author, has glued together some of the best Java libraries with very sensible configurations, almost to the point of saying "look stupid, this is how you're supposed to do it (in Java)". The application configuration patterns (ugh... I said "patterns"), clear separation of resources, built-in metrics, and health checks is worth studying (at least for journeymen like me).

Check it out: http://dropwizard.codahale.com/

rudiger• 14 years ago

How does this compare to using Spring application development framework (with Maven, of course)? My understanding is that the Spring framework is how you're "supposed to do it" in Java, and it certainly is very popular.

http://www.springsource.org/

eckyptang3• 14 years ago

I've done a fair bit with Spring in Java. Unfortunately the promises seem to fall short of delivering as your application scales up in complexity. I've experienced a number of problems with annoying little bugs and some odd brick walls with transaction management (between JMS/Hibernate).

I'd rather build something with Java EE6 if I had the choice now, or ASP.Net MVC+WCF+NHibernate if I had the choice of platform.

stickfigure2• 14 years ago

Spring and Dropwizard are really not comparable. Spring is a large, invasive framework which covers many different facets of enterprise application design. Dropwizard is a set of commonly-used libraries (jetty, jersey, jackson, slf4j, a couple others) with a little bit of glue to stitch them together.

KevinEldon3• 14 years ago

As I understand it Spring offers a complete enterprise application development framework, while DropWizard focuses on delivering production quality RESTful web services with minimal developer work. I've used Spring a little, it's nice... very full featured; in fact I still have production code using Spring's JDBC and LDAP libraries and Spring Security... they've worked out very well for me. I think Spring is the Rails to DropWizard's Sinatra. They're both very good, but if you need less and DropWizard can cover what you need it's a lot easier to use and probably uses the libraries you'd want to use anyway (unless you know a lot of Spring or JEE already).

brown9-2• 14 years ago

There is never just one answer on what libraries/frameworks to choose, and anyone who tells you that there is just one way you are "supposed to do it" probably isn't worth listening to :)

With that said, I am a big fan of Spring and I don't consider it "intrusive" as some other commenters do. But if you aren't interested in using it as a DI container then you may find it easier to just get closer to the other libraries used (i.e. Jersey etc) and avoid any helper code.

strlen4• 14 years ago

> http://www.springsource.org/

http://static.springsource.org/spring/docs/2.5.x/api/org/spr...

Does that answer your question?

shanemhansen4• 14 years ago

No surprises here. Google did a benchmark on go, java, scala, and c++ that's worth reading. http://www.readwriteweb.com/hack/2011/06/cpp-go-java-scala-p...

Here's my personal experience. Simple go code is actually comparable to c, even for non-io bound tasks. I was very surprised when OpenSSL's AES implementation and go's AES implementations performed similarly in my microbenchmarks. The jvm actually performs very well (go figure that over a decade of optmization in running enterprise workloads would result in a fast runtime). I've inspected the generated assembly in a go code and compared it with that from the equivalent c code and there's no doubt go isn't the most efficient language ever created.

Use go if you want something that's (pretty darn) fast, productive, and has a standard library written by some of the most respected names in the field. Don't use go if you need the most mature and performant runtime and libraries. I have no doubt that they will get there eventually.

luriel5• 14 years ago

> No surprises here. Google did a benchmark on go, java, scala, and c++ that's worth reading.

No, is not worth reading, is misleading at best and has been throughly debunked:

http://blog.golang.org/2011/06/profiling-go-programs.html

Not to mention it used an ancient version of Go, even Go 1 is dramatically faster than that, and since Go 1 there have been even more dramatic performance improvements, but the main issue is that the guy that wrote the benchmarks really had no idea what he was doing (there were similar criticisms from outside the Go communities about the quality of the benchmark).

klrr• 14 years ago

Wow, thanks for sharing.

shanemhansen4• 14 years ago

Honestly I'm happy to hear that. I'm one of those rare individuals who gets to write go at work. I'm really happy with go performance and memory usage. Like really really happy. The different between the cpu usage of a go program using protocol buffers and a python program using protocol buffers is pretty dramatic (go is the clear winner).

Most folks would assume a paper coming out of google involving go has some go experts involved. Clearly this is not the case.

michaelcampbell3• 14 years ago

Every time I read something from Russ Cox he impresses me a bit more. That guy is amazing.

davidw3• 14 years ago

It'd be fun to throw Node.js and Erlang into the mix.

dgv• 14 years ago

You can see vs Erlang in http://shootout.alioth.debian.org/u64q/benchmark.php?test=al... and have a idea, Go is based on Newsqueak (lauched later of Erlang, other perspective to implement CSP).

davidw3• 14 years ago

Sure, but this benchmark was more about concurrency, whereas those benchmarks are more generic. Erlang is not the fastest language out there, but it's supposed to be good at this concurrency stuff...

igouy3• 14 years ago

You chose not to compare against the faster Erlang HiPE measurements because...? ;-)

jamwt• 14 years ago

Please, if you're writing req/resp benchmarks, please include 98/99/99.9% latencies. These are essentially the only numbers that actually matter.

At the mean, at concurrency 50, we have things like 37.5ms vs. 12ms. To a user, that is "instant" vs. "instant". Sure, if we stack a bunch up on a page, maybe we'll start to care, but..

Much more directly, in real-world scaling scenarios, it matters far more if the 98% number is 2s or 248ms than the mean is 37ms. A 2s latency means 2 out of every 100 requests (and likely > 2% of page views), the user experience will suck. And, all it takes is 6 requests to ensure that 25% of users experience this (your homepage alone probably requires 6 requests).

I'm not just raising this to be pedantic--I've seen plenty of systems that have been tuned to do very well on things like req/s or even mean latency but do very poorly for 1 in 100 or 1 in 1000 (vs. being a very fair scheduler at the cost of overall throughput). We reward these systems by measuring and praising the wrong thing. (e.g. mongodb vs. riak)

edit: (btw, not intended as a specific defense of either go or java wrt the article, just a general statement about benchmarking these systems)

kristianp• 14 years ago

Here's an insightful comment from the blog page, about cpu and memory usage:

I downloaded patricks fork and got 4300r/s with maxprocs 1 with the go version, out the box.

Recompiling and running with gomaxprocs=100, i got 4400r/s.

mvn deploy and running auth-1.0.jar on JDK 1.7 on my box similarly peaked out at 3100r/s.

It's worth noting though, that I was using patricks httperf bench.sh modification, and it appears that httperf was cpu bound in both cases, with the kernel and postres taking about half a core between them.

Using wrk, by contrast, spun main (the go program) up to 2.5 cores, and 6000r/s. Java under wrk lit all cores for a time, then hit `Exception in thread "async-log-appender-0" java.lang.OutOfMemoryError: Java heap space` `Caused by: ! org.postgresql.util.PSQLException: FATAL: sorry, too many clients already`

A bit more investigation and tuning later, using 10 clients `wrk -c 10 -r 100000 -t 4 -H 'Authorization: basic apikey_value' http://localhost:8080/authenticate` Java was spinning out 10kr/s. Go was limited to 6kr/s. It's worth noting some more details however: Go only ever spun up two postgres forks, whereas java spun up 10. There's scope for optimization there. Go used 8mb of ram, whereas java was sitting on 130mb after a few runs. The Java version, cranking out at 10kr/s was maxing the kernel out on one core, so that's probably approaching the practical limit for single machine tests.

I suspect putting a load balancer in front of a couple of instances of the go program would allow you to totally smash the java performance, given that it lit 2 cores at 6kr/s, and java lit 8 at 10kr/s. The memory usage tradeoff is significant - JVM sitting at 130mb and Go sitting at 8mb. Clearly everyone needs to draw their own conclusions on this, for their own purposes. The JVM solution is carrying a stats server, a ton of other tooling and so on. The Go system has some - pprof was included, but it's limited by comparison. Arguably gdb and so on can actually be of real use in the Go case, but that's also an exercise for the reader.

Interesting any which way you look at it. There are a lot of other interesting side effects (that need working out) in both programs as evidenced by this simple testing. Postgres also needs some tuning if you really want to slam this with anything remotely resembling a real world scale test.

My tests were done on OSX 10.8 12B19 on a 2.3GHz i7 wiht 8GB of DDR3 at 1333MHz, an intel 510 SSD, totally untuned postgres. The machine is a Macbook from whatever year that is.

Here's wrk, for anyone looking for it: https://github.com/wg/wrk

Comparing Go and Java, Part 2 – Performance

Related Stories