How to handle 1000s of concurrent users on a 360MB VPS

http://markmaunder.com/2009/how-to-handle-1000s-of-concurrent-users-on-a-360mb-vps/

by joelg87 • 15 years ago

255 78 15 years ago

markmaunder.com

stock_toaster4• 15 years ago

At the end of the article, why is apache even being used anymore?

Just use nginx and php-fpm (or php-cgi/fcgi).

ramidarigaz• 15 years ago

That's what I use. I thought about the method in the article, but it seemed too complicated, and nginx + fcgi is plenty fast for me.

Are there any performance benefits to having Apache as well?

stock_toaster4• 15 years ago

Performance benefits? No, I wouldn't think so. I can think of too many performance downsides to it.

The only benefits I can think of offhand would be if you had a requirement for something that Apache specifically provided, or custom modules you needed run. Something like suphp, modsecurity, or shared hosting configuration, etc. All policy type stuff, not performance.

mfjordvald2• 15 years ago

None at all. It's purely because most people aren't willing to invest the time it would take to not need .htaccess.

In fact by not using PHP-FPM you lose out on awesome stuff like the slow log!

geuis• 15 years ago

I use nginx exclusively as my server and I run node instances and wordpress behind it. Why even use apache?

etherealG• 15 years ago

mind if I ask what you use to tie wordpress in to nginx? cgi?

ognyankulev• 15 years ago

As written in another comment, nginx + php5-fpm is a great combo, no need for Apache at all. I run both Drupal and WordPress (the latter against my will :-P ) in such a setup.

I think the only reason people use nginx -> Apache mod_php is because there's a lot of documentation and knowledge about Apache. nginx requires a little bit different mental model how HTTP requests are served but it's worth it.

EDIT: You can think of PHP5-FPM as Apache+mod_php but only serving PHP requests. So nginx handles static files and URL magic and when there's request for PHP, it passes it to PHP5-FPM, just like it would pass it to Apache+mod_php. This is oversimplifed but it's kind of like that.

muppetman• 15 years ago

You can do the same on a really high traffic Drupal site using the boost module. It's basically a module that will, for non-logged in users, generate a static html page and serve that instead. PHP isn't even called, the .htaccess just serves up the cached content.

And that's using Apache!

tobylane2• 15 years ago

I've heard a lot of bad things about htaccess, is its slowness comparable to php over static, or apache over nginx?

true_religion2• 15 years ago

htaccess is just a configuration file used by Apache to determine what should be served to whom.

georgebashi• 15 years ago

That'll definitely give you a speed boost, as the interpreter doesn't have to get involved (if configured right). However, this article is talking about IO clogging apache workers. The issue is that the worker must do two jobs: generate the page (using mod_php) then send it to the client. Using nginx in front of apache means that nginx handles the IO to the client (which it is extremely good at), leaving the apache workers to handle nothing but generating pages, hence getting overall higher requests/sec from the same machine.

mise• 15 years ago

I've been on Apache servers for years, so I come from that frame of mind. My managed VPSs are running Apache, and one of them is running nginx for static resources. I think my managing host (Servint) doesn't even support nginx. How, for argument's sake, would I move my PHP/MySQL sites to run on a managed nginx server?

ComputerGuru• 15 years ago

At the risk of stating the obvious, you would move to a better VPS that charges as much or less for a better package :)

chopsueyar4• 15 years ago

He means Linode.

gsharma• 15 years ago

Repost: http://news.ycombinator.com/item?id=970682

Interesting comments on the original thread.

stock_toaster4• 15 years ago

oh wow. I didn't even notice the posted date on the original article until I followed the repost link you provided, and saw how long ago that had been posted.

nwmcsween2• 15 years ago

nginx is actually pretty bad at all benchmarks I've thrown at it, you can see some here: http://www.webhostingtalk.com/showthread.php?t=918355 this was on an 8 core intel Xeon 5520 server system it maxed 8 cores (18ghz) @ 7500 requests per second for static content.

mfjordvald2• 15 years ago

I haven't read through the forum thread as there are ten pages and much of it seems off-topic. But the requests per second doesn't even matter here, the fact that you were capable of maxing the CPU at all shows me you did something wrong. I easily get 12k requests on an i7 box and the CPU wasn't even stressed then. Eventually it was IO that limited it.

chopsueyar4• 15 years ago

Even though it was eight cores, he only had 1024KB of system RAM.

stock_toaster4• 15 years ago

I found that forum thread quite hard to follow, and there are blobs of ab runs sprinkled all over. I clicked a few pages through, and didn't see any configs posted, sysctl values, etc. Was it more of an ad-hoc test you were running just to get some vague ideas about performance, as apposed to a comprehensive performance comparison?

My personal (anecdotal) experience shows that nginx is amazingly fast. Some nginx configs can impact performance if you aren't careful (I am thinking of try_files vs if-tests[1]), and there are various nginx tunables (worker process count, cpu affinity, etc) as well.

I have personally seen issues with apachebench when there are lots of connections (not to mention linux system tunables such as max file descriptors!), iptables state tracking, etc...

[1] http://wiki.nginx.org/IfIsEvil

nwmcsween2• 15 years ago

This was stock RHEL 5, I tested nginx compiled with gcc -O2, -O3, -O2 -ffast-math epoll was enabled, I even specified the cache line size at build time and tested hard enabling AIO the nginx config consisted of simply serving a simple static html page. I tried every option under the sun at the time tcp_nopush, etc. I spent days testing apache vs nginx vs litespeed, instead of drinking the kool aid.

piotrSikora• 15 years ago

You're doing them wrong. Just check results posted by Leif Hedstrom (Apache Traffic Server developer)... He's getting 100k req/s on far worse hardware: http://www.ogre.com/node/393

lamnk• 15 years ago

Ugh, then there is something seriously wrong with your configuration.

This is another interesting benchmark between nginx, lighttpd and varnish: http://nbonvin.wordpress.com/2011/03/24/serving-small-static... Even on a Core i3 laptop nginx can handle 70k req/s

SaltwaterC4• 15 years ago

First of all, there's no such thing as 18GHz. The GHz don't sum up. It's like saying that if a woman makes a child in 9 months, two women can make a child in 4.5 months. It doesn't work that way.

Moving forward to the actual issue. Something is really broken into your configuration. The first thing is the file AIO. But this is Linux's fault. The file AIO is broken by design. It bypasses the kernel file cache. Which is a pretty large performance hit per se. The second thing is the fact that I can easily get 55k req/s on a Q9400 with nginx, without even touching the sysctl parameters of a Ubuntu 10.04 x86_64. The only actual difference is nginx 1.0.x instead of the ancient 0.7.x you would actually find into the standard repos aka the production version I publish into my company's private repo.

jschuur• 15 years ago

Worth noting the the story is from Dec '09.

gaius• 15 years ago

People were doing this with Squid years before that, in the 90s.

iphoneedbot2• 15 years ago

Yeah.. squid is actually great. I do wish squid is a packaged that just needs to be turned on for most VPS

mmaunder2• 15 years ago

This is surreal. I just wrote a draft blog entry about the changes in startup economics due to event-servers like node and nginx (publishing tomorrow), and I hit HN just before heading to bed and this blog entry is #1.

Thanks for all the love today guys. Have an awesome memorial day weekend if you're in the States.

PawelDecowski• 15 years ago

Or bank holiday weekend if you're in the UK ;)

I've been using nginx for about 2 years and it performs so much better than Apache.

I'm not using Apache as a backend though, I use FastCGI. It performs even better.

jrockway• 15 years ago

Why use nginx instead of Varnish as the frontend proxy? If you are using nginx as your http <-> fastcgi gateway, that's one thing, but if you're just using it as a reverse proxy, it seems like a lot of extra stuff to maintain.

I like living on the bleeding edge, so I do varnish -> mongrel2 <-> apps instead... but varnish -> nginx <-> apps or varnish -> apache <-> apps seem equally reasonable. The key is caching and being able to handle slow users without tying up your app. All three of these setups do that. nginx to apache only saves your app server; it doesn't do much caching.

mmaunder2• 15 years ago

I must admit I've never used varnish, but from what I gather, it's function is caching, not handling large numbers of concurrent connections. Nginx's strength is handling tens of thousands of concurrent connections with very little incremental CPU load and a tiny memory footprint. It then reverse proxies those connections to a threaded-model server occupying each thread for very little time and quickly freeing them up for other work.

From what I gather, varnish uses a threaded model with 1 thread per connection which is what nginx solves. So it might make sense to put nginx in front of varnish so you get high concurrency with great caching.

Nginx does offer some caching but I've played with it in high load environments and wasn't impressed with the performance.

StavrosK8• 15 years ago

Varnish is arguably more performant than varnish. I've had amazing success with both, and I don't think both are necessary together, but they're both screaming fast. I noticed one day that varnish had served 100,000 visitors per day from the cache due to a Google mistake only because I saw it in my analytics. The server load was at 0.05 all day.

I love varnish. Hard.

stephth• 15 years ago

How do you test the strength of your setup? Some solid way to test my setup would probably reduce my fear of experimenting with bleeding edge.

jvdongen• 15 years ago

You could use something like grinder (http://grinder.sourceforge.net/) to create a loadtest scenario which you can use to establish a baseline before a change and run after a change to assess impact.

True loadtesting requires quite a bit of thought to go into your test-setup, but for those kind of differential tests it is usually enough to make sure that your test-client and network connectivity to the application are constant factors and good enough to generate a decent load.

You won't catch the subtleties with those tests, but you will normally catch the 'oops, my perfomance dropped by 50%' type of errors. The subtleties you can then work out during normal operations.

swah• 15 years ago

Clojure with mongrel2?

chopsueyar4• 15 years ago

Are the loopback requests limited by the TCP stack?

Is it possible to set this up where nginx and apache communicate via a socket?

arianb• 15 years ago

Yes, and that's the recommended way of setting it up. :)

ericb• 15 years ago

Dropping keepalives actually slightly hurts your best-case performance. As a config change, if you are using the prefork mpm (I wouldn't if scalability is your goal) it will generally help you though. With prefork the scaling math on worker memory * connections vs your box's memory is often the first place things will fall apart if you have keepalives on and real traffic.

I suggest the worker mpm if you don't want to switch to nginx.

jacques_chester2• 15 years ago

Missing: caching. Caching. Caching. And more caching.

I speak of Wordpress and database-backed PHP applications of its ilk. Wordpress loves to spray MySQL in a thick stew of queries for every single page load.

Without sensible -- preferably aggressive -- caching, your Wordpress site will die in the arse.

My setup generates cached .gz files, which nginx can serve directly. It flies. It didn't used to.

aonic• 15 years ago

Or if you dont want to deal with Wordpress to implement the caching, use Varnish. (also mentioned few times in this thread)

I've used Varnish for some high-traffic Wordpress sites by sitting it in front of Apache running Wordpress. With well planned TTLs, and logged-in cookie detection, most of the inconveniences of caching can be eliminated.

Also, using Disqus or similar services for comments will be helpful. You wont have to worry about comments not showing up until the cache clears, leading to confused users. That is, if you use the javascript implementation of these comment services.

Alternatively, plugins like WP Super Cache have automatic post cache flushing if there are updates, and you can also tell it to bypass cache for logged in users. But it is IO-bound.

euroclydon• 15 years ago

WP Super Cache will serve up cached files via PHP with the flip of a switch. While this isn't as fast as letting the server serve up cached files, it's an easy win.

nicksergeant• 15 years ago

It's not missing, he specifically addressed caching at the end of the article (indicating he didn't cover it).

getsat• 15 years ago

This is the best caching plugin I've found for WP: http://wordpress.org/extend/plugins/w3-total-cache/

It supports disk-based caches and memcached.

otterley3• 15 years ago

This technique naïvely assumes the only thing that is slow is the client. If a backend data provider that the web server communicates with (e.g., database) is also slow, this arrangement merely adds complexity without much corresponding benefit. The origin server will still block waiting on the data provider, which can cause process starvation in the pathological case.

Moreover, issues with slow clients often can be solved by raising the TCP send buffer size. As long as the response size is less than the send buffer size, it really doesn't matter how slow the client is: write() will return immediately, leaving the webserver free to serve the next request. Getting the data to the client then becomes the kernel's responsibility.

SaltwaterC4• 15 years ago

That would work, unless you're using a container-based VPS. OpenVZ is notorious for standing in the way of system tuning. Using nginx makes a lot of sense in this case, especially with the ey-balancer patch.

otterley3• 15 years ago

The TCP send buffer size can be controlled at the application level (see SO_SNDBUF in tcp(7)). There's no need to adjust kernel sysctl settings.

iphoneedbot2• 15 years ago

This approach really works well... I would add that off-loading MySQL to a different VPS within the same network works well too; latency within the same network is very negligible. Doing simple optimization such as moving/disabling mail and stuff you probably dont need works too.

jacques_chester2• 15 years ago

Indeed, pushing MySQL onto a different VPS in the same network basically halved page generation times for my sites.

This is partly because when a table contains a TEXT field, MySQL performs all joins against that table on disk, regardless of what fields you're selecting or joining on.

Which is a problem for, oh, I dunno, every PHP blog engine / bulletin board / CMS ever.

james4k• 15 years ago

I just find it odd that Apache hadn't employed the likes of epoll already. IRC servers have been using async sockets like this for yeeeears.

peterwwillis2• 15 years ago

Well, it does since apache 2.1 ('event' mpm module, replaces worker)

SaltwaterC4• 15 years ago

Event MPM is still a large pile of fail. Unless you're unwilling to switch from Apache, the benefits over Worker are mostly theoretical from the performance point of view. Hopefully this will change with 2.4, but only for those who didn't kick Apache out of their production machines.

mp3geek• 15 years ago

I use straight nginx on Fanboy Adblock (which gets hundreds of SSL connections per minute), no need for Apache.

peterwwillis2• 15 years ago

Hint: you can just run Apache twice; once with a mpm_prefork config for your apps, once with a mpm_event config to handle client connections.

rkalla• 15 years ago

As a good number of folks have pointed out, this article (helpful for sure) is basically nginx + PHP-FPM.

nginx servers up static content (e.g. if you are using WP-SuperCache and it's generating those static cache files for you, JS, CSS, images, etc.) then you configure a pool of warmed up PHP VM instances via PHP-FPM (Check /etc/php5/php-fpm or some equiv dir on your server).

Then you setup a rule that directed PHP requests to the FPM service, that would probably look something like this in nginx:

  location ~ \.php$ {
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME  /var/www/mysite.com/$fastcgi_script_name;
    include fastcgi_params;

}

and instead of adding the layer of Apache servicing the PHP requests, you are having nginx pass through the PHP requests directly to PHP processes to run.

Going from an Apache2 + FCGID configuration (somewhat similar to this, but with Apache) to nginx, I saw a 75% drop in server load.

I'm almost certain this still isn't a totally tweaked out setup that someone more familiar with this process could do better, but for my needs it took my server crashing under load no matter what I did, to typically being idle most of its life (~900k/month pageviews)

So I'm a big fan of nginx. I'm not saying you couldn't configure Apache to do the same... just 5 years of attempts to do so never got me anywhere with it.

How to handle 1000s of concurrent users on a 360MB VPS

Related Stories