A HTTP Proxy Server in 20 Lines of node.js Code

http://catonmat.net/http-proxy-in-nodejs

by pkrumins • 16 years ago

116 25 16 years ago

f00li5h• 16 years ago

=^_^=

simonw4• 16 years ago

If anyone's looking for a project, a reusable HTTP proxying library for Node.js with easy hooks for adding pre- and post- checks and transforms would be incredibly useful. Being able to implement API rate limiting and authentication as a Node.js reverse proxy that wraps around an existing web application would be even easier with a well-tested, fully featured proxy library.

weixiyen• 16 years ago

Does anyone have any idea how it would compare performance-wise to what's out there right now? vs Nginx for example.

sh1mmer• 16 years ago

Ry has some Nginx / Node comparison in his JSConf slides http://nodejs.org/jsconf2010.pdf

Summary: Node isn't quite as fast as NginX but it's close, and it's programable, and he's still tuning it.

andrewvc• 16 years ago

EDIT: After looking at this http://nodejs.org/jsconf2010.pdf from sh1mmer's post, it looks like I need to be using a Buffer object to serve data over 64kb due to v8's garbage collector making a performance mess when using strings.

EDIT2: After looking at the code again, file reads come in as strings, so I don't actually see that Buffers would help, at least until the node file API starts passing around buffers instead of strings.

I can't speak to performance wrt to proxying, but I benchmarked its performance serving files recently[1].

Now, I didn't benchmark nginx, but as you can see, its pretty slow at that task, averaging ~10 MiB/s on fairly decent hardware, across a single gigabit switch.

One problem node has, is that it sucks dealing with strings containing non UTF-8 data:

From the node docs:

Node supports 3 string encodings. UTF-8 ('utf8'), ASCII ('ascii'), and Binary ('binary'). 'ascii' and 'binary' only look at the first 8 bits of the 16bit JavaScript string characters.

1. http://blog.andrewvc.com/nodejs-can-really-scale

silvestrov• 16 years ago

This is not a server in 20 lines of code, but server configuration in 20 lines (there is no parsing of the http protocols, but only calls into a lib that implements an http proxy).

The examples could be implemented in fewer lines of Apache configuration. I would have been nice with some examples that shows what node.js can do, which can't be done (easily) with Apache.

zaph0d• 16 years ago

Agreed, the code is trivial, but HTTP is a first-class protocol on node.js. Can't blame Peter for not reinventing the wheel.

simonw4• 16 years ago

I think this does show something that can't be done easily with Apache: scripting decisions about what to proxy in JavaScript, in this case to implement auto-reloading of configuration files for IP filtering and regex-based host blacklisting.

jimbokun• 16 years ago

At the top of the page it says "good coders code, great reuse."

TimothyFitz• 16 years ago

Not quite accurate. The code he wrote does the "proxy" logic while libraries underneath it are doing the work of parsing HTTP. He's definitely not doing "calls into a lib that implements an http proxy."

More to the point, what he has is a proxy in 20 lines of code which will let him make, say, a custom load balancing http proxy in 40 lines of code.

euroclydon2• 16 years ago

Since this is near the top of the HN page, I'd like to ask a question about node.js:

What is it suited for? I read the about section of the node.js homepage, and it says that "Node's goal is to provide an easy way to build scalable network programs." Okay, that sounds cool, and I like Javascript, so I want to see if I can fit it into any projects I'm working on, but then I go on to read: "Almost no function in Node directly performs I/O." Okay again, single thread, no I/O, I can see why it's fast. But what if I need I/O, like from a DB? Does that mean I should not use node.js? What is a suitable method to store and transport data when using node.js? MemCache?

alunny• 16 years ago

The core of Node has very few functions that perform I/O, but there's no reason you can't use I/O in your own programs (in the original link, the network stream is read and written out, and also the blacklist file is read, synchronously). The readFileSync call is an anomaly in Node - typically you use evented I/O - open a stream and set listeners/callbacks for particular events.

There are modules developed for most of the major open source databases (I'm not sure what the status of the async MySQL driver is, but there's definitely something you can use).

simonw4• 16 years ago

Database adapters for Node are beginning to show up - there's an async PostgreSQL one and I think there's one using libdrizzle (which should work for MySQL as well). Since Node speaks HTTP fluently it's great for talking to things like CouchDB, and there's also DBSlayer http://code.nytimes.com/projects/dbslayer which provides an HTTP API for running MySQL SQL queries.

Node + Redis is an awesome combination.

Don't forget APIs - if your application simply hits a bunch of HTTP APIs from other providers (mashup style) Node is a great fit.

BrianHammond• 16 years ago

Thanks for the shout-out in the slides about the Redis client for Node.js.

http://github.com/fictorial/redis-node-client

gcampbell• 16 years ago

The important word in that sentence is "directly". There are plenty of ways to do I/O in node, but they generally involve event-based callbacks so that they don't block the thread.

euroclydon2• 16 years ago

Are there any special considerations for keeping the TCP connection open while the thread is waiting for the callback? I'm assuming no, since node.js doesn't seem to exist in order to facilitate long running requests.

radicalmatt• 16 years ago

I suppose I should chime in and say that I am not exactly sure what Node.js's about page is talking about when it says 'I/O'. Could someone clue me in as to what exactly this means?

fizx3• 16 years ago

I benchmarked this a few weeks ago at around ~300req/s on my recent macbook, which was a little disappointing.

For kicks, here's my CoffeeScript/Node.js code I used. I did the benchmark with ab entirely on localhost. Normally that's not ideal, but it was enough to tell that this isn't as fast as I'd like.

http://gist.github.com/382372

felixge• 16 years ago

You need to create a new client for each incoming request. In your code, you are actually queuing up all responses.

So your result is probably an order of magnitude off.

fizx3• 16 years ago

Good to know, thanks!

Edit: I feel like I tried that, and a couple other variations, and this was the fastest. But it's worth revisiting, and I'll definitely try it again.

A HTTP Proxy Server in 20 Lines of node.js Code

Related Stories