Nginx just became the most used web server among the top 1000 websites

http://w3techs.com/blog/entry/nginx_just_became_the_most_used_web_server_among_the_top_1000_websites

by MarionG • 13 years ago

306 137 13 years ago

future_grad3• 13 years ago

Student here.

Why is Nginx more popular than G-WAN?

I saw the benchmarks for G-WAN earlier this week (http://gwan.com/benchmark) and am curious why most people would choose nginx over the performance that G-WAN provides?

dschiptsov6• 13 years ago

1. Look at the code. Especially src/core. 2. Nginx is a software pipeline, it has same ideas behind its design as modern CPUs. 3. Look at the code.

future_grad3• 13 years ago

Great advice.

http://tomoconnor.eu/blogish/gwan-snakeoil-beware/ was a good read.

FooBarWidget6• 13 years ago

I know it's cool to say good things about Nginx, and Nginx is good software, but your point (2) doesn't mean anything. I'd like to see some serious research and explanation into Nginx's code.

Full disclosure: I'm one of the authors behind the Phusion Passenger application server (https://www.phusionpassenger.com), built on Nginx. What I'm saying here is based on my experience with Nginx module development and research into Nginx code.

As far as I see, the main reason why Nginx is fast is because the programmer is brilliant. No, seriously. Look at the code. There are almost no comments, there are no unit tests, and yet Nginx has no well-known stability bugs, or even performance regressions. There are also tons of micro-optimizations, some of which are of questionable nature on modern CPUs (e.g. the huge switch statements in the HTTP parser). The Nginx code goes into great pain to ensure that there are no unnecessary copies or memory allocation. Everything assumes that you use static string structs that point to already-allocated memory instead of dynamically allocated strings. In return, writing an Nginx module requires extreme programmer discipline. Almost everything is hard.

Compared to Apache, Nginx's main reason for being faster is because (1) it's evented and (2) it has a lot less features than Apache. No .htaccess - implementing .htaccess is a huge performance penalty because suddenly on every request you have to perform additional filesystem operations. No dynamic spawning of processes (unless you use Phusion Passenger). No dynamic loadable modules, so if you want to add anything you have to recompile Nginx entirely.

axaxs• 13 years ago

G-WAN is not open source, and has no peer source review that I know of. I was not even able to run the binary to test it due to my distro being too old. The self-made benchmarks cannot be taken at face value. If you dig deeper, you'll find outside benchmarks showing poor performance in other metrics. I don't mean to insult the author or product, just providing my 2 cents into its lack of popularity.

FooBarWidget6• 13 years ago

Other than claims by the author of G-WAN, I've yet to see anybody claiming that G-WAN outperforms Nginx. I also cannot find any evidence of anybody using G-WAN in serious production. I could be wrong of course, but every time I've inquired about it, nobody responds. I also cannot find any kind of community around G-WAN.

Also, it seems that a lot of the "performance" in G-WAN is thanks to its microcaching feature. When it's under high concurrent load, it caches requests for about 1 second. I suppose it's useful on public cacheable responses and useful in benchmarks, but does this really count as "faster" in the sense that G-WAN is a faster system overall? I don't know.

That being said, G-WAN has many interesting aspects indeed.

future_grad3• 13 years ago

"I also cannot find any evidence of anybody using G-WAN in serious production."

This. This.

I find more articles about why NOT to use G-WAN more than anything.

fredsted• 13 years ago

I've yet to see anybody claiming that G-WAN outperforms Nginx.

What about this?

http://nbonvin.wordpress.com/2011/03/24/serving-small-static...

http://www.rootusers.com/web-server-performance-benchmark/

jlgreco2• 13 years ago

Other than claims by the author of G-WAN

It doesn't really help that the author gives off some mild "losethos" style vibes (the rantings about open source software, the weird "buy an encrypted archive of the source" thing, etc.)

I'm sure it is very impressive in some very limited circumstances, but it really isn't something that I want to get close to. I can't trust people so.. untrusting.

xionon2• 13 years ago

From their website: "But G-WAN was written to have an application server, so it also supports 7 of the most popular scripted languages: Java, C#, C/C++, D and Objective-C/C++."

As far as I can tell, G-WAN does not support the features or languages that many people want. As a Ruby programmer, G-WAN is (probably) impossible for me to use, and if I could fiddle with it enough to get it working, it would not provide enough of a benefit to justify replacing a working nginx setup.

There's also the prevalence of tutorials, books, and StackOverflow answers related to nginx; G-WAN is pretty sparse in this regard.

Basically: the eternal balancing act of "features" vs "speed." nginx hits a sweet spot for features & speed for more people than G-WAN.

chc• 13 years ago

Nginx doesn't really support Ruby either. You need an app server, either compiled in like Passenger or one you proxy to like Unicorn. Is the second option not available here?

simonz05• 13 years ago

Never heard about G-wan before, but from a quick read-through it is not quiet standard. It seems like somewhat unusual design, but interesting. From the g-wan website:

G-WAN is an all-in-one solution because communicating with other servers (FastCGI, SCGI, etc.) takes time (enlarging latency), and wastes CPU and RAM resources when the network can be avoided. Also, supporting many programming languages lets G-WAN serve most needs instead of having to use slower backend servers and fontend caches. Remember that our goal here is to use the ultimate low-latency and resource-saving solution. This is why G-WAN is a:

-Web server -App. server -Cache server -Key-Value store server -Reverse-proxy and elastic load-balancer server

rajivm• 13 years ago

I'm not too familiar with G-WAN, but there's many more factors than some benchmarks published by the author of G-WAN:

1) G-WAN is closed source. This means you're tied to their paid support for fixes and improvements. Web servers are pretty critical pieces of infrastructure that are not replaced easily (if you're doing large deployments, custom config, etc.) and so having this risk is huge. 2) Documentation, configurability, support for well-tested integration with various application servers. For example, Nginx supports uWSGI out-of-box. The G-WAN site seems to have limited documentation. 3) Nginx is well proven in production. Sometimes just being incumbent is a good reason. This often takes some large project or company taking a risk on the platform, proving it in the process; this is unlikely due to #1. 4) Other benchmarks indicate otherwise in regards to G-WANs performance; not to say its bad, but the selection of benchmarks on its site is likely biased towards it.

There may be many other reasons...

simonw• 13 years ago

I'd never heard of G-WAN. It looks like it isn't open source, which is a major turn-off these days for core infrastructure components like web servers.

njharman2• 13 years ago

Performance != Everything that is important.

WatchDog• 13 years ago

While we are on the subject of G-WAN, does anyone have any insight into the naming of the server.

The only reference to "GWAN" I could find refers to a mysterious NSA system [1].

1. http://en.wikipedia.org/wiki/File:Intel_GreenDoor.jpg

dschiptsov6• 13 years ago

This is just a PR and propaganda. We all know that JVM is the best "platform" ever developed by humanity and Jetty is the most efficient, fast and especially reliable under a high load web server.)

IIS must be on the second place, btw, considering how much man-hours and millions of USD were spent on it. We all know that MS technology is superior and very scientific.)

pestaa• 13 years ago

I can't decide if you're serious or not, but if this is irony, it is very much misplaced.

Nginx is rising because it deserves to, all others are more or less declining because they fail to catch the attention of the average sysadmin.

amackera• 13 years ago

I think it was meant tongue-in-cheek.

dschiptsov6• 13 years ago

Nope. nginx is rising because it is a very masterpiece of programming. Take a lock at the code - it is clean, readable, every syscall counted and thought of. No unnecessary copying of data, not idiotic casting from one kind of a data-structure to another. It is a piece of art of system programming. Look at the code. Quality wins here.

threeseed2• 13 years ago

I know you were joking but actually in those web benchmarks that have been posted some of the JVM frameworks actually beat the Nginx based one (OpenResty).

dschiptsov6• 13 years ago

Unfortunately, as far as I been taught, there must be some inconvenience. General purpose bytecode interpreter, even with JIT, cannot beat optimized thin layer of C code on top of an OS's specialized syscalls. Just according to some laws of nature.)

jstanley• 13 years ago

It would be interesting to see what they consider the top 1000 websites to be. Does anyone know if their list is public?

EDIT: Found it. This post http://w3techs.com/blog/entry/w3techs_surveys_are_now_based_... implies they are using Alexa's rankings.

nkozyra2• 13 years ago

I wonder how much of this is actually a reverse proxy to apache, though.

zaphoyd• 13 years ago

I wonder this as well, especially with "top" websites. How much high profile traffic these days is actually served by a single piece of software?

All of our high profile sites have layers of caching and load balancing between them and the Apache backends that actually generate the content. In our case Nginx is in fact on the public facing end performing reverse proxy and TLS termination only. That is not to say Nginx isn't a fantastic product, but it may not be entirely fair to Apache, Varnish, and friends to measure HTTP server marketshare in this way.

ownagefool• 13 years ago

It's likely a significant number still do this for legacy reasons but for new deploys utilising the likes of PHP-FPM, PSGI, uWSGI, & Rack should allow you to avoid using apache at all.

machbio• 13 years ago

A major chunk of PHP based applications would be doing these, else it would be standalone Nginx serving static files for the most part of the servers..

adPothier• 13 years ago

Nginx works very well with PHP-FPM through FastCGI, and there is about zero configuration needed as everything is up and running after an apt-get (minus some commented configuration regarding php in Nginx).

I think more and more php-based app are powered by this kind of stack.

Uchikoma• 13 years ago

Several sites I have insight into, use nginx in front of Tomcat, Jetty, Netty, so I agree this might be the main purpose of nginx, and it excels there.

drawkbox• 13 years ago

True but that is one way to take over the world, get in front of the biggest webserver out there, then slowly take over.

runn1ng• 13 years ago

I am a person that doesn't really understands webserver software. What is approximately the difference between Apache and Nginx?

Also, should I stop using LAMP stack for my small and experimental PHP websites that very few people visit, or is it good enough?

metachris• 13 years ago

For me the main reasons to switch from Apache to Nginx were that it's smaller, faster, very flexible and easier to configure. Plus the additional features (eg. reverse proxy, ...).

sciurus4• 13 years ago

Huh? Apache has been able to act as a reverse proxy since the 1990s. See https://httpd.apache.org/docs/current/mod/mod_proxy.html

There's a list of all the official Apache modules at https://httpd.apache.org/docs/current/mod/. Also, there are many third party modules.

stephenr2• 13 years ago

"Reverse proxy" is not an "additional feature" of nginx.

Apache can serve as a reverse proxy to a web server, or even a fully functional forward proxy for lan users to the internet.

Nginx was never meant to have more features than Apache, it's intended to be lightweight and fast.

If you can imagine it, Apache is likely to have a module for it. With nginx, not so much.

ameswarb• 13 years ago

Apache is a process- and thread-driven application, but Nginx is event-driven. The practical effect of this design difference is that a small number of Nginx "worker" processes can plow through enormous stacks of requests without waiting on each other and without synchronizing; they just "close their eyes" and eat the proverbial elephant as fast as they can, one bite at a time.

Apache, by contrast, approaches large numbers of requests by spinning off more processes to handle them, typically consuming a lot of RAM as it does so. Apache looks at the elephant and thinks about how big it is as it tucks into its meal, and sometimes Apache gets a little anxious about the size of its repast. Nginx, on the other hand, just starts chomping.

The difference is summed up succinctly in a quote by Chris Lea on the Why Use Nginx? page: "Apache is like Microsoft Word, it has a million options but you only need six. Nginx does those six things, and it does five of them 50 times faster than Apache."

Source: http://arstechnica.com/business/2011/11/a-faster-web-server-...

antihero• 13 years ago

Isn't Apache's default MPM event driven since 2.4 anyway?

yuvadam• 13 years ago

For small scale stuff, there's practically no difference between apache and nginx.

For larger scale deployments, nginx is generally considered to perform better than apache.

threeseed2• 13 years ago

Even for small scale stuff there are some benefits to using Nginx.

There is the ability to use Lua with MySQL, PostgreSQL, MemCache, Redis, Upstream etc to perform functions of an app server e.g. authenticate users, serve JSON from a database. It's fast and keeps your app layer focused on real business logic.

njharman2• 13 years ago

For small sites, I would word that as:

If you already have Apache, don't bother switching. If you're starting new, use Nginx.

singlow2• 13 years ago

Even for small scale stuff, the "default" nginx way of using PHP is much better - with php running in its own process via php-fpm or fastcgi. If you are running Apache with PHP-FPM you can get similar performance, but you find fewer how-tos and documentation on that since most Apache users leave modphp in place.

With modphp in limited memory environments you run into memory swap issues super fast unless you are careful about your Apache configs, since it will, in most default environments, spawn many more Apache processes than you have memory for. PHP makes the Apache process use 20-200M of memory for many standard apps like drupal or wordpress. Put it on a 1Gig linode without turning down the MaxClients and add 10 concurrent users and you will see the site grind to a halt. Now install nginx and php-fpm and follow anyone's basic how-to, it will only spawn as many php processes as your memory and cpu cores can handle, and nginx will keep spinning out the static files while php is churning in the background, living within reasonable memory bounds.

You can achieve the same thing in Apache, but I never really knew how to do that until Nginx taught me. I have never seen Apache set up right for this in any of my clients' existing servers and they can never scale at all.

Modphp is a bit faster if you have enough memory for your PHP application * the number of simultaneous connections, but that is hardly ever the case, since Apache will be reusing the same high memory processes for every jpg and css file that it does for all of the php requests and 2-3 simultaneous users can make 20-30 concurrent requests. You better make sure that your maxclients is set to your physical memory divided by your php app's memory use. In a 1G VPS this might be 1024M/64M = 16. Save some memory for MySQL if it is on the same server and you probably shouldn't have MaxClients over 10. Now all of the css, jpg and png files are waiting in-line for the 500ms php scripts to finish and the site loads slow. Many PHP apps eat up over 128M per process, even though a typical request is smaller, that one crazy request eventually bloats all of the processes and you are swapping. If PHP starts hitting swap, your request times are rising up to 3-5 seconds real quick.

pavs• 13 years ago

"Apache is like Microsoft Word, it has a million options but you only need six. Nginx does those six things, and it does five of them 50 times faster than Apache."

Chris Lea

TylerE3• 13 years ago

I'm not so sure I agree with that. Nginx is actually pretty featureful these days.

xionon2• 13 years ago

nginx is a good tool, and you should probably twiddle with it sometime so you know what it's about.

As far as I know, nginx can do anything Apache can do. nginx is generally faster than Apache. Apache can include modules without a recompile, nginx often requires you recompile to add features. nginx can be used as a reverse-proxy, a load balancer, or a thin SSL / SPDY layer, and succeed; Apache isn't as well suited to non-traditional web serving.

Personally, I find nginx's configuration syntax nicer to read and write than Apache's.

LAMP is fine for small experimental websites. Do what you're comfortable with! The scale at which the differences between nginx and Apache actually matters is pretty large.

stephenr2• 13 years ago

there is plenty apache does that nginx doesn't do. Also, as stated in another comment, apache can do reverse or forward proxying.

Anyone trying to argue for nginx on the basis of better feature support (over apache) is blind to the reality of things.

chrismealy• 13 years ago

Fifteen years of apache configs was enough for me. I'd use nginx if it was half as fast.

DirtyMonkey• 13 years ago

Completely agree with this. After a lot of tweaking I could get Apache (w/ the worker MPM) pretty close to the speed of NGINX, but the difference in configuration files was enough to convert me.

And with NGINX I understand _every_ line in the configuration files, with Apache it was almost like configuring by coincidence.

lmm4• 13 years ago

Also, should I stop using LAMP stack for my small and experimental PHP websites that very few people visit, or is it good enough?

It's plenty good enough; it's worked for decades already. If it ain't broke don't fix it. If traffic on a site is ramping up to the point where you're thinking "I might need to get new hardware for this soon", that's the time to start looking at a higher-performance webserver.

And honestly I'd look at moving away from PHP more urgently than moving away from Apache, if only because of their respective security records.

Osiris4• 13 years ago

Really? It's a heck of a lot easier to migrate your web server than to re-write your entire codebase in another language.

mortehu2• 13 years ago

I'd look at moving away from PHP more urgently than moving away from Apache, if only because of their respective security records.

In my opinion, this is like moving away from C due to its security record.

nathan_long• 13 years ago

Apache executes dynamic code using modules that it loads at boot time. Nginx makes you compile in the ones you want to use. That helps make it faster.

sciurus4• 13 years ago

Check out the C10K problem.

http://www.kegel.com/c10k.html

cenhyperion• 13 years ago

You shouldn't stop using your LAMP stack. The differences between how the two behave really only come into play when you're talking about thousands upon thousands of concurrent users.

tracker12• 13 years ago

Unless you are doing it all on the same server... having less load from one part of the server means more available for other parts, which can lead to a better overall user experience. It's not quite a drop in replacement, but could be well worth considering... trying to get under 300ms page loads shouldn't be just for the big boys.

jayflux• 13 years ago

It seems to me like Apache was designed more for hosting multiple websites for different users (the .htacces for example) With Nginx It's not so easy for a jailed user to change his settings unless he's given access to the config.

Just my opinion.

SideburnsOfDoom• 13 years ago

It's likely that our website would be counted as nginx since the response headers include "Server: nginx...". And there is an nginx load-balancing layer at the front.

But it's a pretty thin layer with the real "web servers" behind it.

unreal37• 13 years ago

Same here. I'm working with a big corporate client who is installing nginx box in front of an IIS one. The IIS box does the work, the nginx handles security, user throttling and reporting.

And no one will know IIS is behind it because it's a proxy. These stats don't tell the whole story.

babuskov• 13 years ago

There are two trends here.

Many people recognize that it's much cheaper to place nginx in front of other web servers than to buy more servers. We could say that nginx is mainstream now, so any serious website would use it. If you take "top million websites" instead of "top 1000", you might see different percentages. Automated tests just see nginx in the header, although something else is the actual workhorse for the site.

Some of the stuff does not need Apache anymore. As I moved almost all my projects from PHP to Node.js, there's little use for Apache now. And nginx serves static content with less CPU and RAM.

kilovoltaire• 13 years ago

Just in case you didn't notice, the graph also includes "top 1,000,000" websites, and the percentages are indeed quite different.

TallboyOne• 13 years ago

Probably because of the 10 trillion $1/month cpanel accounts

singlow2• 13 years ago

No reason to use Apache for PHP - I run all of my PHP sites on nginx with php-fpm or fastcgi. It has been much easier for me to manage performance, especially on low-memory VPSs with php running in a separate process. I suppose it would work just as well with Apache behind nginx, but I don't see any good reason to have it in the way unless you rely on htaccess files.

ciclista• 13 years ago

Same here, much lower memory print compared to Apache, and it's fast and easy to configure/set up.

jtchang• 13 years ago

I just want to say how incredibly lucky I feel to have two very mature and high performing OSS web servers.

Both nginx and Apache are absolutely solid pieces of software. I wish we had as many alternatives to Microsoft Excel as we do IIS.

kogir• 13 years ago

I've always found it interesting that of the major Web servers IIS is the only one with a programmatic configuration API.

eblume• 13 years ago

I have never configured an IIS server. Do you find that the programmatic configuration API is helpful? I rarely find httpd.conf or nginx.conf to be difficult to manage, but it's possible that we're missing out on a good thing in the OSS world. It's been known to happen.

jl6• 13 years ago

Does a shell/python script that writes a httpd.conf file count as a configuration API?

TallGuyShort• 13 years ago

Have you looked at Auegas? I've used it quite a lot from within puppet to make well-structured changes to Apache config files.

beachstartup• 13 years ago

generating and modifying unix config files programmatically has been happening for nearly half a century

the only reason a web daemon has an API in windows is because the windows registry is opaque, error-prone and in general, a giant piece of shit.

bdcravens4• 13 years ago

Simple text parsing + service restart obviates the need for an API. The shell is your API.

icebraining2• 13 years ago

I use jinja templates to generate nginx config files. I don't need yet another proprietary API with its own quirks and interoperability problems.

kalininalex• 13 years ago

I think it reflects the general Microsoft philosophy of having APIs for everything. After all, they are an API company. It shows in the PowerShell design with it's object API approach.

Also, with the API you should be able to configure IIS without restarting it, while changing web.config recycles web apps (in ASP.NET scenario). With load-balancing, however, this is less of an issue.

h2s• 13 years ago

It sounds wrong to me to refer to Apache and nginx as "alternatives" to IIS. They're the web servers, and IIS is the "alternative", kind of like how Excel is the spreadsheet application and LibreOffice Calc is the alternative. I'm aware that I cocoon myself in something of an anti-Microsoft echo chamber, though. Does reality match up to my prejudices?

_b8r0• 13 years ago

At the risk of second guessing the parent, they're considering the classical 'enterprisey' stuff as mainstream. If you look at what people use in most companies of a reasonable size, IIS is surprisingly common and Apache tends to only turn up when Java or PHP is involved, with appropriate nods to Oracle and IBM for their relative technologies.

Outside on the open Internet I absolutely agree that Apache and Nginx are the default HTTP servers. I switched from Apache to Nginx years ago (with a few exceptions) as it just seemed as though I could do more with less with it and it made sense for me to standardise.

RyanMcGreal• 13 years ago

Does reality match up to my prejudices?

According to the linked article, IIS has only 16% market share overall, and less than 11% of the top 1000 websites, so I'd say yes.

nwilliams2• 13 years ago

Does reality match up to my prejudices?

Similar to how Apple takes ~75% of the handset market profits, IIS takes almost all the profit in the web server market and is increasing revenues year after year.

It does quite well for itself given that the competition is fierce and free.

voltagex_• 13 years ago

I see nginx as more a general application server, and IIS as a specialised server you run when you need NTLM/Windows integrated authentication and ASP.NET et al.

nhebb• 13 years ago

My rule of thumb for OSS software is that if a type of software is integral to developers' jobs, then there are some very good tools available. Operating systems, web servers, text editors, database systems, ... are all good examples of this. If the type of software is not integral to developers' jobs, then there are fewer good alternatives. There are some pretty good ones (OpenOffice, GIMP), but they're often not as polished and lack power features.

philliphaydon2• 13 years ago

I love nginx! I've almost finished a blog series of running Mono on nginx on Ubuntu/Azure.

Awesome stuff!

adambard• 13 years ago

If you're on Azure anyhow, why not use Windows and skip Mono? I don't have personal experience with either, but I hear that Mono is still quite a bit slower (although still fast enough for most purposes, I suppose).

GeneralMayhem• 13 years ago

Also note the inverted shape of the trends: Apache is much more popular among less visited sites, but gets less so as you go up. nginx is unpopular with the masses, but more so with the most popular sites.

I can see at least two ways to interpret this:

1. nginx is just strictly better, and the sites at the top need that performance/capability and employ the top/best-informed architects to maintain their systems, so they're the early adopters. Everyone else is still on Apache by virtue of inertia, but will slowly move over as time goes on.

2. nginx is faster for high concurrency, and mostly gets used as a front-lines layer in front of something else, maybe even Apache itself. In this case, the charts will stay about like they are now, because only the most stressed sites need that much engineering.

I don't know that there's an easy way to differentiate between the two without either waiting and watching for change or knowing more about the internals of the top few thousand sites.

kalleboo2• 13 years ago

I imagine a majority of small sites are still on cheap shared hosting providers, and nginx doesn't support anything .htaccess which allows individual site owners in a chroot jail to configure the web server.

dakimov• 13 years ago

The open source continues to kill the industry burying the progress under loads of free shit like that.

t0mislav• 13 years ago

At work, we have Ngnix as reverse proxy for some static files. Oh boy is this thing fast with very very small amount of resources (cpu, ram) and big number of requests.

mike-cardwell• 13 years ago

And yet, according to Netcraft yesterday:

  "Apache and nginx, both open source web servers, have 
  lost market share this month whilst Microsoft gained 
  significantly, up by 2.43 percentage points, to just shy 
  of 20% of worldwide sites. For the second consecutive 
  month, nginx is powering fewer sites than in the previous 
  month's Web Server Survey, which is due, in part, to 
  almost 2M sites moving from nginx and to Apache. Within 
  the million busiest sites, a similar picture emerges: 
  nginx lost over 4,000 busy sites, many of which have 
  moved to Apache."

http://news.netcraft.com/archives/2013/07/02/july-2013-web-s...

mstrem• 13 years ago

Just for clarity, the numbers reported by Netcraft in the monthly survey blog post are referencing all sites, active sites, and top 1M sites.

Not to saying that the picture may not be different in the top 1k.

eliben• 13 years ago

Interesting - how can the trend seen in those charts be explained? The clearly falling popularity of Apache the larger the website and clearly rising popularity of Nginx complement each other nicely.

flaktrak2• 13 years ago

And so it should! Apache is last millennium's technology.

level09• 13 years ago

A better title would be: "Nginx is the most used web server in front of other servers"

Nginx just became the most used web server among the top 1000 websites

Related Stories