The emperor's new clothes were built with Node.js

http://notes.ericjiang.com/posts/751

by erjiang • 12 years ago

91 58 12 years ago

notes.ericjiang.com

mslider• 12 years ago

Isn't it just the case that nodejs is the least worst way of writing a concurrent web server? we know it's not perfect, but what is?

pjc50• 12 years ago

A whole class of programmers have learnt to write continuation-passing code in Javascript for asynchronous operations, as that's the way AJAX works. Moreover, it's the only way you can do things in the browser.

Node is web-browser style javascript programming, detached from the browser GUI and given the ability to accept inbound connections. Along with this has come a whole slew of fun wheel-reinvention around frameworks and templating engines.

bphogan• 12 years ago

I'd like to think that if we weren't all watching Node we'd have more people to pitch into other things. I hear a lot of bluster about "learn new things" but I often just see people sticking with what they know. When the bright minds of web development are figuring out new ways to make JavaScript less awful, those minds aren't focusing on other solutions.

FWIW I think Elixir holds a lot of promise for the future. Projects like https://github.com/dynamo/dynamo look really cool. Of course, it's new, and alpha. But maybe we could all give it a try?

wcummings• 12 years ago

Erlang pretty much blows Node.js out of the water, in terms of sophistication of concurrency primitives (to mention one thing). Erlang is really just syntax on top of BEAM, which is pretty tailored to the language (you'll notice Elixir and Erlang share identical types etc).

Elixir adds some nice metaprogramming features, and I love that it exists. Erlang is used for a lot of Important Things in production, and its value to many developers is stability and fault tolerance. Erlang moves slow. It's nice to see a sister language that can more rapidly develop new language features (maybe some of the goods ones will get added to Erlang (:

Joe Arms post about Elixir is pretty interesting [1]

[1] http://joearms.github.io/2013/05/31/a-week-with-elixir.html

meritt2• 12 years ago

I'd argue the simple approach of having a web server (nginx, apache) send requests to a pool of workers (perl, php, python, ruby) is considerably superior.

aikah2• 12 years ago

the problem is can workers do something else while waiting for the db to respond? AND are all libs built for that use case? tbe only advantage of nodejs is you have no choice but to write code a certain way.

voidlogic• 12 years ago

I've spend a lot of time porting node.js code to Go. While the Go applications typically get an unfair benefit of being the result of the refactor, they have without fail (in my XP) had similar to drastically better runtime characteristics, while I would argue (and my teammates agree), being much simpler to read and maintain.

I have have written a few web apps in Go as well spending most of my time writing things like high performance proxies, CDNs and databases in Go, and IMHO Go is a "more least worst" way to write highly concurrent code compared to node.

akilism• 12 years ago

Computer Programming: A million wrong ways to do something and a million right ways to do something.

Cthulhu_• 12 years ago

And where the only correct answer is "It Depends"

erjiang7• 12 years ago

Those "million wrong ways" are not distributed equally amongst languages/environments :)

logn2• 12 years ago

I prefer http://silkjs.net

It handles each request in a forked subprocess so it's a lot more straight forward.

hardwaresofton8• 12 years ago

without sounding like a web 3.0 idiot -- how do you manage to scale that? subprocesses are not cheap

angersock• 12 years ago

subprocesses are not cheap

This statement depends heavily on the runtime environment. Consider either Erlang or fibers as implemented in C/C++, for example.

logn2• 12 years ago

I'm sure someone else reading can answer your question based on more experience and expertise than me... anyhow the subprocesses are re-used so there's no overhead in their creation. And if your server is doing I/O bound work (or is just waiting a lot) then it should perform ok. If everything is CPU bound, then you'd want to not create more subprocesses than cores. SilkJS lets you define the number of subprocesses you want.

More info:

http://silkjs.net/sync-vs-async/

general_failure3• 12 years ago

I don't know about silk but forking subprocess model usually involves caching the subprocess. So it's going to be as light weight as any other cgi thing.

general_failure3• 12 years ago

Node is popular because it is fairly simple to learn and use. That really is the mains reason why I have seen most people pick it up.

aikah2• 12 years ago

it's simple to use when one has an understanding of networking principles.I wouldnt say it's simple to use has "PHP",where bascially as long as you can download WAMP,write echo and use FTP you're good to go.

It's definetly not simple to write webapps in nodejs,where a simple exception can take down your entire node server.

I'd even say go concurrency model is simpler and easier to use than nodejs.

erjiang7• 12 years ago

I assume you read what I wrote and I wasn't clear enough. Node is actually quite DIFFICULT to learn and use. It's very very easy to write code that's broken in some way if you don't have a deep understanding of Node.js.

Fun unrelated anecdote: someone was showing off their concurrent Node.js web scraper and as requests piled up, it was apparent that it could handle no more than 5 concurrent requests. Why? The HTTP library's internal concurrency limit defaulted to 5.

dap4• 12 years ago

How many concurrent requests can the six-line HTTP server on Node's front page serve? (I don't know the answer. I've taken a quick stab a few times on a few platforms, but I keep running into local TCP port exhaustion, leading to multi-second latency bubbles, long before I run into any Node.js or physical limit.)

GrinningFool• 12 years ago

Node is popular because it feels light and easy to use.

    http.createServer(...a simple function...)

Holy crap! I just created an http server and now I can respond to events with just a few lines of code!

Once you get deep enough into to it to realize that it creates a tendancy towards complex deeply nested code and that making a good, maintainable application is a non-trivial task... it's far too late. Because of the 'fast' feel of it, you're convinced that these are just things you have to figure out and learn best practices for - it couldn't be that the 'fast' feel was deceptive to begin with.

rdtsc• 12 years ago

Node.js along with other callback and promise based frameworks are insidious. They are so because they look good in small examples. "Oh look, here is how to serve a file in Node.js". You just have one callback for the request, then another for the error and so on.

In larger cases it turns into a mess, but by that time there is already a buy-in. Run a short little demo, looks fast, read about how "cool it is" on the web some place. Ok let's rewrite everything in it.

Quite often once people started heading the wrong path, it becomes hard to look back and say "Yeah this is the wrong way, let's turn back". That is just basic psychology.

At the same time, I think Node.js is great. It is great as a differentiator when you ask people why they picked it and see what they say. If they come back with the list in the article (basically list "evented","async","webscale" jargon), then that tells me they don't quite understand what is going on underneath. Which is not bad, people have the right to be confused, but it just lets me know on what level to converse with them on.

compsci_mofo• 12 years ago

We learnt that Node wasn't cancer, but it also wasn't a good technology either. Still, it was useful in keeping the clueless muppets away, lest they fuck up other tech ecosystems.

hardwaresofton8• 12 years ago

Sarcasm is pretty heavy in this article, but I think it's a little overly negative...

He makes a lot of comparisons with Go, but Go was created after Node. Many of the languages he notes as possible replacements for the non-blocking rant are obtuse (I love Haskell, but there's no way it's not obtuse)

[EDIT] - turns out they were made pretty much at the same time 2009... Learned something today, I guess. Felt like Go came after node to me but I guess it didn't

Also, I must be the only one that doesn't really mind callbacks. Maybe it's stockholm syndrome.

I also think node should get kudos for at least seeming different enough to prompt the move from enterprise customers. Any move away from burdened "enterprisey" software is worth praising I think. Not because java is bad/lame by default, but instead because it might prompt other companies to lust after something new that might be better (whether it's python, node, go, or haskell is irrelevant)

Touche• 12 years ago

Also, if you're going to choose a language to compare to, don't choose one where you can't even implement map, filter, etc. I'll take callbacks over having to use loops for everything.

smrtinsert• 12 years ago

Node hasn't impacted enterprise customers. It has impacted a few high profile silicon valley companies, also Java is not synonymous with enterprise - look at Twitter for example.

I imagine Twitter every day looks online at these node articles and just laughs. That's a team that realizes Java != JEE and that JVM programming is a great thing to be doing.

henk53• 12 years ago

Java != Java EE

Not every usage of Java involves Java EE, but that doesn't mean Java EE isn't a really valuable tool for many of us. It has had a bad name in the past, but last version have totally changed that and it's now a very sane tech.

Twitter of course is a highly specialized thing that operates on a Scala that almost by definition not every web site out there will be at. It makes sense to build a lot of their own tools and only use lower level infrastructure as a base.

So the JVM and Java is low enough for them to build their stuff on, but many parts of Java EE may be too high level for their very specific use case.

lmm• 12 years ago

I also think node should get kudos for at least seeming different enough to prompt the move from enterprise customers. Any move away from burdened "enterprisey" software is worth praising I think. Not because java is bad/lame by default, but instead because it might prompt other companies to lust after something new that might be better (whether it's python, node, go, or haskell is irrelevant)

The infuriating thing is that it's not the technology that makes the difference, it's the clean slate and getting away from stupid policy. I bet the Paypal people could have written a better version in Java than the Node version, if they weren't required to use the existing internal library and follow existing coding standards and deploy using approved infrastructure and...

erjiang7• 12 years ago

I picked Go not because it's my dream tool (not having 'map' is kind of sad) but because it's proof that you can have your cake and eat it too. Otherwise, people think that you have to choose between programming sanely and high-performance, but not both.

hardwaresofton8• 12 years ago

Yes, I agree -- but Go's got it's own warts. I think they picked up that (err,result) common function signature from node (or they just thought of it themselves, either way).

However, I think once you start trying to manage go routines and all their channels, and all the messages that are flying around, go will start looking less utopian. Of course, a lot of that is subject to code architecture (well architected code will be easy to follow), but I feel like no one's found enough of the dark corners of go yet.

Oh one thing I wanted to mention - Why didn't you include any samples of go routines in your code?

Unless I am misunderstanding go immensely, a statement like "(task3(task2(task1()))" wouldn't actually be asynchronous, you would need to spawn dependent go routines right, and wait on completion on the channels?

mercurial2• 12 years ago

He makes a lot of comparisons with Go, but Go was created after Node.

It's not like Node invented evented concurrency. Something like Twisted predates Node by many years, for instance.

hardwaresofton8• 12 years ago

Yes, very true -- Tornado also.

However, if twisted were it's own language, it may end up looking much like node. Twisted is just a library (a damn good one, yes), but if you were to try and transform it into the base framework for an entire ecosystem people might also rail against it from time to time, due to the async-by-default nature.

Also, to me, twisted seems like a much more high-level tool (replicating protocols over simple parallelization/concurrency), than something like gevent

mlitchard• 12 years ago

"(I love Haskell, but there's no way it's not obtuse)" I invite you to drink the kool-aid, then you won't think Haskell is obtuse anymore.

Love, Haskell fan-boy

hardwaresofton8• 12 years ago

Oh I have partaken of said kool-aid. Maybe I haven't had enough of it.

I've done toy examples, and watched lectures (Erik's C9 lectures are amazing), and I just haven't found anything to write in Haskell.

At this point Haskell is starting to seem like the sketchy guy in the corner at a party. "Oh you think I'm obtuse? just drink a little bit more of this kool-aid here"

ilaksh• 12 years ago

I use ToffeeScript. Its a nice alternative to nested callbacks that looks like normal synchronous code.

dap4• 12 years ago

Just because lots of people say "non-blocking, good concurrency, event-based model" without understanding it doesn't mean it's not correct. With a single-threaded, event-based model, you can get an awful lot of concurrency at reasonably consistent latency without ever thinking about threading issues or managing thread pool size to balance memory and concurrency -- and often with less memory, too.

Yes, there are downsides: it's not a natural model for many people; you do still have to consider many complex concurrency-related issues; and control flow libraries are usually needed to make complex code much more readable. But these are either easy to deal with or intrinsic to the complex problem at hand, and the above still stands: once you understand how to program with this model, the "default" implementation usually scales very well.

Yes, Node was not the first to do this, but that's not relevant to anything.

Besides all that: I'd put Node's postmortem debugging and dynamic tracing capabilities up against any other dynamic environment.

As a side note: it's enormously condescending to tell people that a very broad technical choice is not only wrong for their technical problem -- without even knowing what it is! -- but that they're also part of some mass delusion for even thinking that there's anything to it. I hate arguing about this stuff. You keep trashing Node, and we'll keep using it to build large distributed systems.

erjiang7• 12 years ago

Hi, I attempted to address your argument in the section roughly titled "Node.js has super concurrency!". You may have to scroll down "below the fold", so to speak.

I presented two different ways of writing a single-threaded, event-based function that can handle "an awful lot of concurrency without ever thinking about threading issues or managing thread pool size to balance memory and concurrency". Essentially, it is possible to get good concurrency in a single thread without the use of many nested callbacks or abandoning the language's natural control flow primitives. It's not necessarily the case that you can't have both event-based concurrency and sane code.

dap4• 12 years ago

Thanks. Far from not having scrolled far enough, I was actually responding directly to several points in that section, starting with the suggestion that people who say "non-blocking", "super concurrency", and "evented" may not understand what they're saying. I was also responding to your point that Node wasn't the first runtime to do so.

Yes, other environments have compelling concurrency models. As for control flow: it doesn't matter much to me whether the control flow abstraction is built into the language or merely a ubiquitous construct in the code I use. (In fact, that's a tradeoff, too: I like that I can go inspect, modify, and extend these non-primitives easily, and I've done so to add better observability.) The Go example is interesting, but less explicit -- that, too, is a tradeoff. It's all tradeoffs; to pretend like one thing is strictly worse (or better) than all others is not founded.

mambodog2• 12 years ago

Why is it that most of the articles which pop up complaining about node's concurrency options don't even mention streams? For most data-manipulation tasks, FRP-style composition of higher order operations on streams turns out to be the best choice; not callbacks, promises, etc. Not even bringing them up in a discussion of concurrency in node makes it hard to take the article seriously.

hardwaresofton8• 12 years ago

But what about when you need the whole object in memory before you want to work with it? Also, many of the libraries written for nodejs are not necessarily stream-ready -- but I would argue most are callback-ready.

xhrpost• 12 years ago

Not sure I've ran across what you're describing. Are you talking about using streams as an option to get around callback hell? Have a link describing this?

mambodog2• 12 years ago

http://caolanmcmahon.com/posts/how_i_want_to_write_node_stre...

http://highlandjs.org/

https://github.com/dominictarr/event-stream

http://reactive-extensions.github.io/RxJS/

wyuenho• 12 years ago

"everything they call is callback based, returns a Q promise, returns a native promise, is a generator, is a pipe, or some other weird thing because it’s Node.js. (Just tell them to check the type signatures.)"

I think this addressed your question indirectly.

erjiang7• 12 years ago

Yes, streams/pipes/channels/whatever can be great for certain classes of problems. I omitted them because I very very rarely see them used in Node.js. They are probably under-utilized and I simply didn't have enough examples to think about, since the overwhelming majority of the libraries and code I've encountered use callbacks. It could be a great topic to write about if you have the time.

aespinoza• 12 years ago

I agree with a lot of the arguments in the article, but I do feel a very negative opinion about Node.js. I don't believe Node.js is that bad.

The 2 points I always hear from Node.js are:

1) Node.js is fast. It is really not that fast. It all depends on what you are comparing it to.

2) It is easy to learn. It is not easy to learn. The difference between browser javascript and V8 javascript are considerable. Not everybody will pick it up as easy as you imagine, specially if you are choosing Node.js to allow frontend devs work on backend code. It is a very different mindset.

I like the idea that is Node.js, but I don't think it is mature enough to take over the backend. I think there are other more mature languages right now that do the job without making programs look like spaghetti code.

I think people should take this article with a grain of salt, and keep getting their hands dirty in Node.js. I think it has a future in the cloud.

woah• 12 years ago

These articles always seem to boil down to only one thing- "I'm bad at callbacks".

I mean, really, is there any other criticism here? There is a valid point about the true nature of Walmart and Paypal's rewrites, but that's pretty tangential.

There's kind of a thing about some completely rookie mistakes in error handling, but that's really not relevant to anything, and again, tie into the "I'm bad at callbacks" thing.

I'd really be interested in some criticisms of node that aren't variations on stories of the author's past difficulties with callbacks, as most people who use node have no problems with them.

EDIT: Downvotes? Please, point out to me the part of the article that is not about callback troubles.

pessimizer• 12 years ago

This is the article I'm going to refer people to to explain the problems that I have with nodejs. All of these (absolutely true) observations aside, I really like how the the nodejs trend has brought concurrent async-style thinking into the mainstream of web programming.

By "all of these observations aside," I mean that I hate working with it personally, and I'll be using Erlang/Webmachine thank you, but I learn new ways to think about modern problems every day from many of the great people that are part of the nodejs juggernaut.

kybernetikos• 12 years ago

If you read the interviews with Ryan Dahl, one big reason he chose javascript for his scripting language after considering a number of other options was that javascript did not already have a huge body of blocking libraries.

If you're trying to script a nonblocking system where everything is nonblocking all the way up and down the stack, you're going to end up creating a parallel ecosystem to the normal one, and confuse people by not allowing them to do the normal things they would do in language X.

Javascript didn't already have a huge blocking ecosystem.

And speaking of ecosystem, that's another really good reason to like node. The stream programming that is coming out of the community is brilliant, and the focus on constructing large projects by writing thin modules that use other thin modules ('onion oriented programming') is fantastic.

I've been doing a lot of Scala recently, and it's extremely noticeable that despite the languages power to do things in many different ways, the Scala community frequently seem to choose a strangely convoluted path filled with hidden magic going on behind the scenes where every new library you depend on feels like learning a new language from scratch.

It might well be that the Go community is also as good as the node community for some of this stuff, but the community and library ecosystem is a definite plus point.

Personally I've never had a problem with Javascript coding. It's perhaps easier to write stupid code in it than a lot of other languages, but I know it well enough to know where to be disciplined, and it almost never surprises me now.

aashishkoirala• 12 years ago

I love your ending :) "Does your local Node.js Meetup group need a presenter? I am available for paid speaking engagements. Email me for more info."

The emperor's new clothes were built with Node.js

Related Stories