Security vulnerability found in Nginx

http://mailman.nginx.org/pipermail/nginx-announce/2012/000076.html

by dirtyaura • 14 years ago

170 56 14 years ago

mailman.nginx.org

matthavener2• 14 years ago

I may be misinterpreting the patch, but it seems like this is a NULL byte injection vulnerability, all stemming from '\0' termination in C-strings. I wonder if length-buffer pairs are more practical when security is a consideration?

gizzlon• 14 years ago

Not a "C guy", but I simply read it as:

Strings could be missing a "\0" and therefore some function could read too far later on..

matthavener2• 14 years ago

Not exactly. Some function expecting a c-string could actually read too short. What happens is some attacker sends a message using a channel/format that specifies a length and a buffer. But, the buffer contains a "\0". For example, (10, "wtf\0extra") is passed as a 10 char string. But, some c function might only see it as (3,"wtf"). The (usually) simply solution is to fail validation when a string of N bytes contains a NULL char before the N+1 byte.

wladimir2• 14 years ago

IMO, zero-terminated strings are, and always have been, a bad idea. Discriminating against a certain character instead of just regarding it as opaque blob of data will bite you in unexpected ways, and strlen() taking O(N) is another drawback.

The string functions in the C standard library suck. bstring is a better alternative: http://bstring.cvs.sourceforge.net/viewvc/bstring/tree/bstrl... . Glib also has a good string library...

kbolino2• 14 years ago

Something to bear in mind regarding non-standard string libraries: syscalls (like open[1] on POSIX or CreateFile[2] on Windows) use null-terminated strings, which means that you have to be careful about embedded nulls, no matter what library you use, when a string gets passed to a syscall somewhere in the chain.

[1] http://pubs.opengroup.org/onlinepubs/000095399/functions/ope... [2] http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...

jaryd2• 14 years ago

Hey guys,

Looking for a consensus on the most stable way to update nginx installations from source.

Thanks!

krobertson2• 14 years ago

I prefer to custom build deb packages (on Ubuntu myself) using a project called brew2deb:

https://github.com/tmm1/brew2deb

It provides a DSL for describing how to build and package something, including patching, and already has a nginx formula:

https://github.com/tmm1/brew2deb/blob/master/packages/nginx/...

Will likely be adding in this patch today...

pja• 14 years ago

It's a relatively simple task if you're on a Debian derivative to take the source .deb & add a new patch to it.

ef4• 14 years ago

I second the idea of building your own deb.

But if you're going to learn a DSL for building debs, why not just learn how to build debs? Your distro already provides the necessary build formula, along with well-integrated startup scripts, etc.

To start out, just build as given:

    apt-get build-dep nginx
    apt-get source nginx
    cd nginx-$VERSION
    dpkg-buildpackage

Then to customize, go into the source tree and update whatever you want to update, and rebuild. You'll see there's already a debian/patches directory where you can drop patches to apply automatically.

I tend to just keep the "debian" directory in source control, so I can take a fresh upstream tarball, check out my debian rules into it, and kick off the build.

radagaisus• 14 years ago

If you don't care about uptime just replace the old binary with the new binary and then restart the server.

This is from a book called "Nginx HTTP server":

1. Replace the old Nginx binary (by default, /usr/local/nginx/sbin/nginx) with the new one.

2. Find the pid of the Nginx master process, for example, with ps x | grep nginx | grep master or by looking at the value found in the pid file.

3. Send a USR2 (12) signal to the master process—kill –USR2 , replacing with the pid found in step 2. This will initiate the upgrade by renaming the old .pid file and running the new binary.

4. Send a WINCH (28) signal to the old master process—kill –WINCH , replacing with the pid found in step 2. This will engage a graceful shutdown of the old worker processes.

5. Make sure that all the old worker processes are terminated, and then send a QUIT signal to the old master process—kill –QUIT , replacing with the pid found in step 2.

some323• 14 years ago

Actually, this is from nginx documentation: http://nginx.org/en/docs/control.html

eggnet• 14 years ago

The above steps aren't optimal.

The makefile in the nginx source shows you how to do a hitless upgrade. This is what I do from the shell after correctly installing, essentially translated from the makefile.

kill -USR2 `cat /var/run/nginx.pid` && sleep 1 && test -f /var/run/nginx.pid.oldbin && kill -QUIT `cat /var/run/nginx.pid.oldbin` && echo restart successful

Obviously, modify paths to the pid files as appropriate.

karolist2• 14 years ago

I've just done that on Gentoo source installed nginx by following these simple steps:

* Get old version config flags with -V, e.g /opt/nginx/sbin/nginx -V

Mine had:

nginx: configure arguments: --prefix=/opt/nginx --with-http_ssl_module --with-cc-opt=-Wno-error --add-module=/usr/lib/ruby/gems/1.8/gems/passenger-3.0.9/ext/nginx

Download tarball:

* wget http://nginx.org/download/nginx-1.0.14.tar.gz

Extract and configure passing the same flags:

* tar zxvf nginx-1.0.14.tar.gz && cd nginx-1.0.14

* ./configure --prefix=/opt/nginx --with-http_ssl_module --with-cc-opt=-Wno-error --add-module=/usr/lib/ruby/gems/1.8/gems/passenger-3.0.9/ext/nginx

Compile and install

* make && make install clean

The make install action will check for old directories and files so no need to worry about stuff being overriden.

Verify your nginx version with:

/opt/nginx/sbin/nginx -v

nginx version: nginx/1.0.14

dhx• 14 years ago

This is poor advice that could be potentially damaging. Instead of manually building and installing from upstream source code you should bump the package version locally to ensure your system isn't littered with orphaned files later on. You'll get the benefit of portage's sandbox and other package management features too. If Gentoo's ebuild had Gentoo-specific patches, you'll get these compiled into the latest version as well. Sometimes these patches are very important.

A better approach to follow is (note this is only a rough guide from memory):

  cp -R /usr/portage/www-client/nginx /usr/local/portage/www-client
  cd /usr/local/portage/www-client/nginx
  mv nginx-1.0.13.ebuild nginx-1.0.14.ebuild
  ebuild nginx-1.0.14.ebuild digest
  emerge -1q nginx

GreySyntax• 14 years ago

My I suggest upgrading passenger some time :)

tptacek6• 14 years ago

This is a very bad bug, and you should fix it ASAP. Don't wait.

some323• 14 years ago

It is already fixed.

spindritf3• 14 years ago

What are the possible implications?

tptacek6• 14 years ago

You can safely trust me on this. Also, if you're concerned, the patch is very straightforward, minimal, safe, and fairly unintrusive. It's not going to break anything.

Just patch it.

rdl2• 14 years ago

In Soviet Russia, your SSL private key becomes public key. (the private key is held in memory by the nginx process. If you can dump memory, you can dump key)

(you may also want to generate a new key and get a new certificate if you use nginx, concurrent with patching this...)

mfjordvald• 14 years ago

Before people panic too much, while people should definitely upgrade ASAP. This exploit does require a hacked or otherwise broken backend. People cannot use this exploit to hack nginx remotely without already being able to manipulate an upstream.

tptacek6• 14 years ago

Applications are broadly vulnerable to this problem. It's true that your app server and your web server will share the blame, but that's going to be cold comfort.

(This comment sounds more disagreeable than I mean it to; sorry, it's tricky for me to comment about this stuff).

sbierwagen• 14 years ago

Nah, I don't feel like it. I'd have to download the nginx source, apply the patch, and compile it, and all that seems like a lot of work for not much benefit.

digitalsushi• 14 years ago

The ides of march bug.

uggedal2• 14 years ago

Arch Linux already has the 1.0.14 release available in the community repo[1].

There are still no new patch level version of 0.7.67 available for Debian Squeeze[2] or Ubutu 10.4 LTS' 0.7.65 version[3]. EPEL for RHEL and derivatives also lack a new upstream version[4].

[1] http://projects.archlinux.org/svntogit/community.git/commit/...

[2] http://packages.debian.org/changelogs/pool/main/n/nginx/?C=M...

[3] http://changelogs.ubuntu.com/changelogs/pool/universe/n/ngin...

[4] http://dl.fedoraproject.org/pub/epel/6/x86_64/repoview/nginx...

IgorPartola2• 14 years ago

Re [Debian/Ubuntu], wait, they incorporated the fix on 17 June 2011, but the patch is being exposed now on the nginx mailing list?

Edit: changed [2] to [Debian/Ubuntu]

uggedal2• 14 years ago

The vulnerability from last year sounded similar and I therefore initialy mixed them. Updated my comment.

Wilya• 14 years ago

1.0.14[0] and 1.1.17[1] have been pushed to the FreeBSD ports tree too.

[0] http://www.freebsd.org/cgi/getmsg.cgi?fetch=753841+0+current...

[1] http://www.freebsd.org/cgi/getmsg.cgi?fetch=756257+0+current...

spindritf3• 14 years ago

> here are still no new patch level version of 0.7.67 available for Debian Squeeze[2] or Ubutu 10.4 LTS' 0.7.65 version[3].

As a side note, there's an official Debian/Ubuntu repository maintained by nginx.org http://wiki.nginx.org/Install#Official_Debian.2FUbuntu_packa...

wkral• 14 years ago

1.0.14 has also been released in the Ubuntu PPA for those who use Ubuntu: https://launchpad.net/~nginx/+archive/stable

1.1.17 is also there in the development channel: https://launchpad.net/~nginx/+archive/development

rdl2• 14 years ago

This is particularly interesting because in a lot of deployments, nginx sits out in front of a lot of other stuff as a load balancer, where it is nicely exposed.

You REALLY should be using multiple boxes if you're running load balancers (especially sw load balancers) with some kind of heartbeat failover. That way you can upgrade single boxes easily, and are ok in case one of them dies. With a bug of this severity, you won't have time to test the patch, so it's probably best to upgrade one at a time in production.

Remember, even if you're running Apache or something else for your actual web server, you can easily have something like nginx sitting in front as a proxy/load balancer. Often in front of your security monitoring devices... and you may have forgotten about it.

aaronblohowiak2• 14 years ago

This only matters if your backend is going to set a header that contains a null byte. Since some people echo back user data in headers (ugh) this could cause an issue. Rails is more than happy to let you put NULLs in response headers, btw. Of course, all of the ASCII CTL characters (0..27) are forbidden by the spec: http://www.ietf.org/rfc/rfc2068.txt

Security vulnerability found in Nginx

Related Stories