To boldly go where Node man has gone before

http://blog.jgc.org/2012/05/to-boldly-go-where-node-man-has-gone.html

by jgrahamc • 14 years ago

131 114 14 years ago

blog.jgc.org

kbd3• 14 years ago

So does Go's net/http module really provide basically exactly the same programming model as Node? If so, why use Node?

One thing the author didn't mention is that Go will use multiple cores in this example, whereas Node is single-threaded. Right?

jgrahamc• 14 years ago

It's true that Go can use multiple cores, but I was doing these tests in a Ubuntu VM that was restricted to using a single core so that Go's inherent 'multicore' advantage was dialed down. I'll add a note to the post.

kbd3• 14 years ago

That's a(nother) pretty big real-world advantage for Go then, no? I know the solution for Node is to just run one instance per-core, but it's automatic with Go.

I'd love to see a second set of benchmarks of the same code on multiple cores.

chc4• 14 years ago

> So does Go's net/http module really provide basically exactly the same programming model as Node? If so, why use Node?

Well, for starters, Go is much more awkward for working with JSON than JS is (which is to be expected since JSON is literally a subset of JavaScript). More importantly, Go lacks the ridiculously vibrant ecosystem for Web development that Node has — no NPM, no Connect, no Jade, no Stylus, etc.

paufernandez6• 14 years ago

'npm' is covered by the Go tools themselves. You can import packages using their github/bitbucket/code.google.com URL.

_ak2• 14 years ago

So what exactly do you think is awkward about Go's way of working with JSON? Here's a code example from one of my current projects:

    type UploadProgress struct {
        Progress int `json:"progress"`
    }

    //...
    // variable progress is of type Progress
    if json_data, err := json.Marshal(progress); err == nil {
        w.Header().Set("Content-Type", "application/json")
        w.Write(json_data)
    }

json.Unmarshal works the same way. IMHO, not exactly more awkward than using JSON.parse and JSON.stringify.

samdk• 14 years ago

I don't program in Node very much or in Go at all, but I think there are reasons to use Node that aren't "Node is fast". I think the "Node is fast" mantra was originally really meant as a comparison to Ruby/Python/PHP, not as a comparison to lower-level languages like Go. (I personally wish Node would just abandon speed as a selling point altogether--I think it'd lead to many fewer misunderstandings.)

In any case:

Node lets you reuse the same code on both the client and server. This can be very valuable if you're writing a JS-heavy webapp. In my experience, the most useful bits of code are the in-code data representation (models in an MVC world). Being able to have identical functionality client- and server-side can save a lot of time.

Node also has Socket.IO, which is very useful if you want to use websockets. (Go has a library for it, but it hasn't been updated in a long time.) The vast majority of the Node ecosystem revolves mostly around building interaction- and communication-heavy webapps. I don't know Go or its community at all, but I don't believe they have the same kind of single-minded focus that Node has. If you're trying to do the things Node does well, you will likely benefit from the community support.

There are, of course, many downsides to Node as well, but much as I dislike Javascript, I still think it's one of the best choices for writing highly interactive webapps right now.

jay_kyburz• 14 years ago

Yeah, I agree, I'm using node right now to build a big web game and there are huge benefits to being able to run the same code on the sever and client.

I wouldn't use node if it wasn't for this because I don't particularly like javascript.

All my other games have used Python on the back end and you can really feel how clunky and painful javascript is when you have to switch between the two five times a day.

kitsune_• 14 years ago

I doubt it does. net/http looks like a basic http listener? As such, the comparison looks biased to me. Mind you, I don't know much about node.js or Go.

chc4• 14 years ago

Isn't Node's http.Server also a basic HTTP listener? What's the distinction you're trying to draw?

jff• 14 years ago

Writing http servers with Go really is pretty easy; here's one I wrote while at SC11: https://bitbucket.org/floren/goblog

It's for an older version of Go, and has only been tested on Plan 9, but it shows a static content + blog server, with JSON config files and templated HTML for the blog pages.

zemo3• 14 years ago

No, the model is completely different. In Node, you pass in a callback to establish what should be done following the return of a longrunning I/O operation. In Go, you use goroutines to manage the concurrency.

Basically if you say this:

    fn()

The Go runtime will execute the function in its entirety and wait for a return value.

If you say this:

    go fn()

The Go runtime will execute that function in a new goroutine (the return value is discarded) and then immediately proceed to the next line in the current goroutine.

So if you have some callDB() function that performs some long-running i/o, in node, you might do something like this to perform that operation without blocking the surrounding code:

    ...
    callDB(someparam, otherparam, function (result) {
        // do something with result here
    })
    ...

while in Go, you would do something more like this:

    ...
    go func() {
        result := callDB(someparam, otherparam)
        // do something with result here
    }()
    ...

which winds up being more like this

    func superGreat() {
        result := callDB(someparam, otherparam)
        // do something with result here
    }

    go superGreat()

With the benefit being that if you want superGreat to run concurrently, you use the go keyword, and if you want it to be run in a synchronous style, you just call it normally.

The problem with Ruby/Python/etc is that it is awkward to take something that is blocking and make it nonblocking. The problem with node.js is that node.js says "blocking is bad; therefore, never block". Go takes a different approach, in that it makes it obvious to know how to write something so that it does or does not block, so that the developer is free to use whatever I/O paradigm fits the problem at hand.

The net/http package allows you to register a handler, which is a function that takes and http request and writes a response. There is a main request loop that, when receiving a request, will create a new goroutine for each request and execute its handler in its own context. So within your handlers, you mostly just write blocking code, because you're already in your own isolated goroutine, and the only thing you'd be blocking is the processing of the current request. If you want to do something in the background, you run it in a new goroutine. goroutines are very cheap, so you can be quite cavalier about their usage.

If you want to use a callback-passing style, you can do that in Go (because it has function literals and closures and first-class functions and all that), but that's not idiomatic by a longshot.

The absolutist position that node.js takes by saying that "all i/o must be nonblocking" is no better than our previous options of "all i/o is blocking". Sometimes the simplest and most readable solution is simply to block. There is a yin to the yang of blocking.

densh2• 14 years ago

Important point there seem to be missing from your goroutine description is it's scheduling with respect to system calls -- all interactions with underlying OS is asynchronous in Go even though it looks like a blocking API. E.g. networking internally uses polling specific to given OS. So if one goroutine is waiting for a system call to complete it will not block the rest of goroutines.

shazow• 14 years ago

My favourite thing about posts like these is the little comparison snippets between a framework I already know (Node) and a language I want to learn (Go). This is very educational, and the benchmarks are the cherry on top. Thank you.

veemjeem2• 14 years ago

I'm sure the reason why users pick node is probably similar reasons why one picks ruby/python -- it's not for the speed.

The language of the web is javascript. Until web browsers allow Go as the embeddable language, node will remain useful. I seriously doubt people would pick Go over node because of a micro-benchmark.

kbd3• 14 years ago

> The language of the web is javascript. Until web browsers allow Go as the embeddable language, node will remain useful.

Do people really get that much code re-use between client and server that using Javascript on the server provides a large benefit?

Raynos2• 14 years ago

In a medium sized web application, 50-70% of server-side code can be re-used verbatim between server and client.

tomjen32• 14 years ago

For small apps, no.

For complex ones yes. The actual code shared may not be that much, but consider the advantages in sharing validation code between the client and server -- you never accidentally forget one requirement on the server side, leaving you vulnerable to an attack.

KaoruAoiShiho4• 14 years ago

Check out stuff like meteor or firebase. They barely even differentiate btwn client and server code.

Rolling your own framework based on the same ideas is not hard.

phoboslab• 14 years ago

I'm currently working on the next major version of my JavaScript game engine[1]. The main new feature will be real time multiplayer. Being able to run the same game code server and client side is invaluable.

[1] http://impactjs.com/

vibrunazo• 14 years ago

You can compile Go to js with gwt. You can reuse code that way. I personally even reuse Java code between Android, front end Web and server. Because of gwt.

I don't think there are many people who choose js because they like it. It's usually because they feel forced. Javascript is a terrible language, I always prefer one of the compilers.

Dart can't come fast enough.

petercooper• 14 years ago

You can compile Go to js with gwt. You can reuse code that way.

This makes me wonder.. and this is totally off the cuff, non-researched pondering.. could there be merit to a JavaScript dialect that would compile to Go?

densh2• 14 years ago

Go is not related to java in any way. It has it's own runtime. Applications are compiled to native binaries on each platform.

Is there a separate branch/fork of GWT with Go support? That would be absolutely awesome in fact but it doesn't seem to be true.

veemjeem2• 14 years ago

Where is this GWT compiler for Go? From what I understand, GWT is for Java translation only, and it only works well on code that doesn't reference any third party libraries. It doesn't work that well on code that makes heavy use of annotations either.

xxbondsxx• 14 years ago

Choosing Node.JS isn't just a performance / scaling decision for me -- one of the main draws to Node is the huge community and wealth of awesome modules. Socket.io, Now.js, cradle, redis, and dozens of other modules abstract away the tedious work and let me build functionality _really_ fast.

Go is really promising but it won't win over the majority of casual developers until there's a community around it. I might as well write a webserver in C if I'm only concerned with performance

paufernandez6• 14 years ago

As we are commenting here, the community is assembling by the minute... Go allows you to import "github.com/user/package" and will download and compile the code, so creating a new project is even easier than with node, no need to "npm install ..." first.

For a taste of what's out there, see: http://go.pkgdoc.org/index.

jlarocco• 14 years ago

The GitHub thing was neat the first time I saw it, but then I thought about it some more, and I'm not sure it's such a good idea.

I couldn't find any way to specify a specific tag or revision, so it's basically always getting the bleeding edge developer build of the library code. That might be nice in some situations, but for getting work done I almost always want the most recent stable release.

Am I missing something?

KaoruAoiShiho4• 14 years ago

npm is awesome, this direct import doesn't seem like a feature, more like a lack of a feature.

cageface4• 14 years ago

I also consider it a huge plus that Go produces a single, statically linked native binary. No fiddling around with directories of libraries and search paths required.

erjiang• 14 years ago

Go has a growing number of packages for things that you might want to interface with: http://go-lang.cat-v.org/pure-go-libs and http://golang.org/pkg/ There are already packages for most databases, data interchange formats, crypto, etc. that most people want to use.

Another draw of Go that you didn't mention is the language and toolchain design itself. I know Javascript better than Go, but would rather write servers in Go than JS.

peterwwillis• 14 years ago

You might like Perl too.

Edit: haha I love how people downvote this like i'm being sarcastic. Down with Perl, that horrible language with a community and modules 100x larger than Node!

m0shen2• 14 years ago

I don't understand the attitudes either. While I enjoy writing Javascript for Node, when other Node users tell me they hate Perl, I can do nothing but give them a blank stare.

So you want an environment that is easy to program in a functional style with closures and first class functions. Has a large ecosystem, including hundreds (thousands?) of packages to do asynchronous IO. Is easily extensible and can fall back on code written in C. Finally, has a large and active community ( Madison NAPC is sold out )?

And Perl isn't even on your radar?

hack_edu• 14 years ago

> one of the main draws to Node is the huge community and wealth of awesome modules. Socket.io, Now.js, cradle, redis

Really? In my experience, all but the most mature modules are hardly beta-quality and in constant flux. Find a good module for your job? Too bad it only runs on 0.4. Find another to get around that; oops, too bad your version of gcc needs to be patched and re-built. The language and its entire package ecosystem is so immature, I'm blown away people trust it in production at all.

"So fix it and submit a Pull Request," you say? That doesn't help me when I'm trying to deliver on a deadline. I love contributing, but I often have more pressing things on my plate.

zzaman• 14 years ago

In my 6ish months of programming in Nodejs full-time, I have yet to run into a library that only runs on 0.4 only or requires my version of gcc to be patched and re-built. Which Node libraries were you trying to use?

blantonl3• 14 years ago

I'm using cluster, crypto, express, memcached, mysql, and mysql-pool in production and they have worked flawlessly for a node deployment that serves a pretty busy JSON API (200-300 reqs a sec on average).

mkramlich• 14 years ago

> one of the main draws to Node is the huge community and wealth of awesome modules.

If you think Node has a huge community and wealth of awesome modules wait til you to discover the older and more mature languages/platforms, like Python, C, Java, etc.

"What I like about Justin Bieber is the massive history of his career, spanning decades, and the broad range of his musical style across several genres." :)

daliusd• 14 years ago

It is funny because I have chosen Node for one my project because I have not found easy way to do that in Python. I must say I really love Python - it is my favorite programming language. I could do the same project in Python but node.js looked slightly better and faster in this case.

Programming language is just a tool - one from many tools you can choose to make your projects and node.js have done some basic things better than Python (e.g. packaging). I have learned to love JavaScript and it is really good language now. Still there are projects I wouldn't choose node.js as my tool.

christkv• 14 years ago

you know what both platforms are great, I'll use them both thank you.

bdr• 14 years ago

The point of Node is that it uses JavaScript, and browsers only speak JavaScript. You have three choices: (1) use Node, (2) duplicate code client- and server-side, or (3) use server-side code that compiles to JS. Right now 3 is the least popular (and 2 is the most) but I think it will end up being the accepted solution in a couple years.

josephcooney• 14 years ago

Or use a plug-in in the browser that speaks your language of choice.

adestefan• 14 years ago

Are we now going to have a run of posts that tweak a stupid micro-benchmark?

david_k• 14 years ago

I can personally vouch for the maturity of Node especially Socket.io. We're building a web-based email client that relies heavily on real-time communication with a Node-based email importer. See http://philterit.com. I've found Node to be faster and more lightweight than Rails 3, which we initially used for our prototype. With JavaScript being the lingua franca on the client-side, specializing in it as a team has permitted us to master it and offer a far better user experience.

tinyjoe• 14 years ago

Nobody is comparing Node with Assembly. sigh

moocow01• 14 years ago

If there is one thing I've learned its to ignore benchmark tests on relatively established technologies.

You'll drive yourself into the looney bin if you hop from one thing to the next based on these comparisons that inevitably popup every month or so.

If you enjoy programming in JS with Node use it - if you enjoy programming with Go use it. Your enjoyment will far outweigh performance differences that will be minor in 99% of real world cases.

IsaacSchlueter• 14 years ago

Yes, exactly.

taligent• 14 years ago

Also it is often external influences that will drive performance improvements e.g. Nginx, Redis, Memcached, Javascript caching etc.

guscost• 14 years ago

I use Express, because I can make and host a fast web page in 3 minutes, and if I ever come across those fabled unsolvable "scaling problems" I can abandon whatever needs to be rewritten without a second thought. Admittedly this is usually for promotional media and assorted small projects, but it still works great.

arturadib• 14 years ago

Node is popular not only because it's very fast (yes, it's still pretty darn fast even if there might exist faster alternatives), but because it's JavaScript.

That's an enormous advantage for startups as (a) the JavaScript hiring pool is much larger than most other languages (particularly in comparison to nascent ones, like Go), and (b) the ability to write client- and server-side code in the same language means small teams can be far more integrated and collaborative than otherwise (and if the startup has a single coder, the point is even stronger).

Assuming these benchmarks are accurate and translate into similar results for more complex applications, I'd gladly pay the 10-20% performance penalty to get the above benefits. Node is fast enough for most startups.

euroclydon2• 14 years ago

I think any person who can write an application in Node could also do it in Go, plus if your having trouble attracting good developers, [edit] Go, not node [/edit] should make the position more attractive.

virmundi• 14 years ago

I've heard the JavaScript developers are easy to find argument before. However, good JavaScript developers are really hard to find. Especially when you need them to take those skills and run them on the server.

There is a big difference between a jQuery hack and a good JavaScript developer.

antihero• 14 years ago

Exactly, I just see the PHP problem repeating itself. Yes, you can get a PHP developer for £7/hour, but are they going to write good code? Unlikely.

Killswitch• 14 years ago

This. What attracted me was the fact that Javascript on the server. Javascript for the client, and Javascript for the database (MongoDB) seemed much more appealing than 3 separate languages. (Normally a PHP/MySQL dev)

blantonl3• 14 years ago

I consider myself a pretty conservative developer when it comes to the tools that I use, and mainly stick to the core LAMP stack for my deployments - MySQL, Apache, PHP etc. I've ventured down the path with MongoDB and currently use it for one special use case where the data served is static.

But for Node - I've really come to the conclusion that it is a great tool for back end API work, such as serving very short requests, Ajax calls, etc. Couple the ability to develop in Javascript while serving JSON makes Node a really nice place to develop in.

Plus, performance is off the charts for my API serving applications. On a dev instance I can serve close to 5500 reqs/sec for one of our APIs where PHP was at about 1000 req/sec. And the difference in memory usage is fantastic.

BTW, our production Node deployment is Node, with the memcache, mysql, mysql-pool, cluster, and express modules. I've been thrilled so far.

sciurus2• 14 years ago

Out of curiosity, how were you running PHP?

euroclydon2• 14 years ago

Good God man, your iPhone theme is awful! I tried to pan the screen to view all the results and it navs to the next blog post?!?

exogen2• 14 years ago

"Easy to build, fast, scalable."

So which part of the test demonstrated that it was hard to build, slow, and/or not scalable?

If you're not impressed that JavaScript reached 80% parity with a statically typed, compiled language – well, sorry dude.

Anyway, nobody is begging you to be impressed. 99.999% of successful projects are built with technology that is completely unimpressive.

riobard• 14 years ago

Just want to point out one obvious thing that I found very interesting: that the V8 JavaScript engine which makes Node exist in the first place comes from the same company which makes Go.

cageface4• 14 years ago

Google's Robert Griesemer, who worked on the V8 VM, is one of the main contributors to Go.

ilaksh• 14 years ago

I think that actually the Node example was significantly simpler.. there were no pointers, and no types needed to be specified. Also I can use CoffeeScript with Node. The only way Go is easier to write is if you know Go but don't know Node. I say its easier to learn and use Node/Javascript.

I am surprised that Node performed so well.. I would like to see that same Go benchmark with 4 CPUs vs. Node and also vs. Node with a clustered web server and the code to see how much more code that involves for clustering with Node.

Also I would like to see some Go code and benchmark that reads a file and/or database in the request and works efficiently.

Actually the code for a Node cluster that would use all of the available CPUs for http in CoffeeScript would look like this:

    cluster = require 'cluster'
    http = require 'http'

    numCPUs = require('os').cpus().length
    if cluster.isMaster
      cluster.fork() for i in [1..numCPUs]
    else
      app = http.createServer()
      app.on 'request', (req, res) ->
        res.end "hello world\n"
      app.listen 8000

Maybe someone could take V8, change it so it compiles (immediately) to static code and uses CoffeeScript, and add a way to optionally specify types (in an unambiguous and readable way) and pointers only when you have to.

robertpateii2• 14 years ago

This test shows Node handling 100 requests simultaneously just as easily as handling them separately.

However Go's memory usage increases by about 5X.

So what happens when you reach 1000 simultaneous requests? Which will perform better then?

I think this is called "selection bias."

reissbaker2• 14 years ago

Not only that, but Node actually got faster as simultaneous connections increased while keeping a completely stable memory footprint. The benchmark showed the opposite of what the author claimed -- Node scaled effortlessly. Its performance was completely stable and predictable under load, whereas Go's wasn't.

Not to knock Go -- it's a beautiful language, and absolutely faster than Javascript for most purposes (and will no doubt increase in stability and speed as time goes on). But this post had some pretty flawed analysis of the benchmark presented.

supersillyus2• 14 years ago

FYI, the original article has been updated with graphs showing reqs/sec, virtual, and real memory for both using 100, 500, and 1000 simultaneous requests. Aside from virtual memory (do we care about that?) it seems to show Go as continuing to perform better.

exogen2• 14 years ago

If you consider something to be fast and scalable (e.g. Go), then something with ~80% of the throughput (e.g. Node in this test) is probably also pretty fucking fast and scalable. Scalability and performance don't walk fine lines like that; there's a reason we talk about orders of magnitude.

cageface4• 14 years ago

Static typing + no callbacks make Go vs Node a no-brainer, IMO. I'm doing mainly iOS work these days but I've been brainstorming ideas that require a server component just so I can use Go for something.

lachenmayer• 14 years ago

What do you mean by "no callbacks"?

cageface4• 14 years ago

You can do non-blocking I/O without the nested callbacks you have to use for everything in Node. You just fire off a block to a goroutine and then code sequentially there.

_urga• 14 years ago

Any language faster than Javascript would be fine if it worked in the browser.

tluyben2• 14 years ago

I pick Node because it's easy to move code from backend to frontend and back. I find this very comfortable while creating and testing software.

To boldly go where Node man has gone before

Related Stories