Nginx 1.9.11 with Dynamic Modules

http://mailman.nginx.org/pipermail/nginx-announce/2016/000170.html

by Nekit1234007 • 10 years ago

201 59 10 years ago

mailman.nginx.org

Nekit1234007• 10 years ago

Docs: https://www.nginx.com/resources/wiki/extending/

stereo• 10 years ago

This is great news for the pagespeed people.

pilif• 10 years ago

not just for them. This also helps a lot for all OS packagers because right now they have to ship multiple custom nginx packages. As a user you either don't get what you want or you get way too much.

Plus, if there's a custom module you'd really like to have in your nginx you're practically forced into building your own because there's no way to get that module loaded into the distro nginx.

Yes. There's some overhead to dynamic modules. Yes. There's a (minuscule) performance advantage to static linking.

But having the ability to ship modules independently of the web server core is a very important usability feature which, frankly, outweighs the performance overhead for most installs.

If you're really CPU bound on that nginx instance where you need the additional modules and you removing that overhead gives you that little bit extra performance you need, then you always have the option of hard-linking them.

geocar• 10 years ago

There's a (minuscule) performance advantage to static linking.

Is that actually true? I would have thought networking latency overshadows any memory latency introduced in a `call foo@PLT` v. `call *(foo@GOT)`.

Are you aware of anyone who has benchmarked this? i.e. not a microbenchmark e.g. inside something like nginx?

If so, it might be worth fixing the dynamic linker: the cost difference is absolutely fixable, as it's a tradeoff for sharing pages with other processes.

snuxoll7• 10 years ago

This also helps a lot for all OS packagers because right now they have to ship multiple custom nginx packages. As a user you either don't get what you want or you get way too much.

I guess it allows us to split the nginx-included modules into separate packages, but it does nothing to aid with packaging extra modules until nginx provides a stable ABI for modules to use.

cbr6• 10 years ago

Unfortunately, it looks like it may not be something will be able to work with. Instead of an ABI like Apache uses, Nginx went with signature checking. To load a precompiled module, the signature has to match. This means we would have to compile many different binary modules for every configuration people might have, and for every version of Nginx. Like several dozen.

As it stands, though, the module system would be very helpful for OS packagers, because they don't have to worry about signature mismatch.

falcolas2• 10 years ago

I would think that containers would help obviate such concerns, since it would be rather trivial to keep even a hundred images in place for an automated compilation run.

Of course, serving the output would be harder, but still not impossible.

LukeShu• 10 years ago

So does nginx still have to know about all of the modules at compile-time? Or can I compile nginx, then later compile a module for that nginx?

snuxoll7• 10 years ago

Actually, it's still a pain for package maintainers. Take a look at mod_wsgi for apache as an example, it's maintained as a completely separate package (the rpmspec and dist-git trees are totally independent from apache) - when the httpd package gets upgraded mod_wsgi doesn't have to be rebuilt at all. This is important, in fact, because rebuilding a package means updating the spec file, bumping the release, and submitting a new build to koji - not something you want to do for loadable modules every time httpd is updated.

This is going to create headaches for anyone wanting to package nginx, either they will have to compile all the modules when building nginx (creating many sub-packages) or there will have to be coordination between package maintainers to do a mass rebuild of all loadable modules every time the nginx package gets bumped. Neither situation is desirable, and I hope they release a stable ABI because until then nobody is going to waste time packaging loadable modules.

withjive• 10 years ago

Is this going to make it easier to get luajit on a stock nginx server?

I just see a single bullet point regarding "Dynamic modules", anywhere I can get more info?

CrLf• 10 years ago

Probably not. The lua module hooks into internal symbols not considered part of the module API.

cbr6• 10 years ago

Nginx doesn't really have a module API. While lua reaches deep inside, loadable modules already have to be compiled for the specific nginx version, which means this module isn't in a different situation from any of the others.

timdorr• 10 years ago

A little more explanation of what that means from the docs:

In NGINX 1.9.11 onwards a new way of loading modules dynamically has been introduced. This means that selected modules can be loaded into NGINX at runtime based on configuration files. They can also be unloaded by editing the configuration files and reloading NGINX.

https://www.nginx.com/resources/wiki/extending/converting/

tomschlick• 10 years ago

This is fantastic news. I've wanted to use the StatsD plugin for nginx for a while now at the load balancer level but didn't want to compile my own and have to continuously re-compile to get patches.

whalesalad2• 10 years ago

Hugely important. Compiling your own Nginx has never been trivial in modern development pipelines, and until now that's been the only way to use external modules.

derFunk• 10 years ago

Honest question: Why do you think it was non-trivial? What is/was complex?

dujiulun2006• 10 years ago

Good for them.

It's worth to note that an nginx fork called Tengine (by Taobao) has a similar feature before.

http://tengine.taobao.org/index.html

ksec• 10 years ago

Yes and the only reason why Taobao forked it was because Nginix did not allow such feature to be upstream.

pibefision• 10 years ago

Does Dynamic modules will help to enable mod_pagespeed easily?

cbr6• 10 years ago

Sadly, probably not: https://news.ycombinator.com/item?id=11066250

toupeira• 10 years ago

Compiling a module is still a lot more straight-forward than recompiling the whole of Nginx, which is probably the main obstacle to people using it (at least it is in my case).

mixmastamyk3• 10 years ago

    load webserver.nlm

(a reference for the old-timers here)

Seems an odd feature for 1.9.11, why not wait for 2.0?

organsnyder• 10 years ago

As long as it doesn't break API compatibility, why wait?

mixmastamyk3• 10 years ago

Because bugs happen. It's likely a change that reaches into the core of the product. Also, would give a psychological boost to upgrading to 2.0, while also subtly warning that the code will be immature until a few point releases are complete.

uggedal• 10 years ago

The 1.9 series is unstable (mainline) while 1.8 is stable. Classic odd/even unstable/stable versioning.

mixmastamyk3• 10 years ago

This is the answer, I had forgotten about this strategy.

zevrox• 10 years ago

If I could get mod_pagespeed, mod_security and mod_cloudflare configured without recompiling/accidentally breaking everything, I'd be so ecstatic.

frik2• 10 years ago

Is this going to make it easier to enable stub_status module without compiling Nginx? (or buying NginxPlus)

http://nginx.org/en/docs/http/ngx_http_stub_status_module.ht...

akurtzhs• 10 years ago

What problems are you having using stub_status?

I'm using the nginx mainline PPA and didn't have a problem enabling it.

frik2• 10 years ago

  This module is not built by default, it should be enabled 
  with the --with-http_stub_status_module configuration 
  parameter

I had to compile Nginx myself, because that module was absent. As you can read also on various community pages, boards, etc.

Nginx is the only server software that I have to compile myself, just to get a basic status feature. It's a way to upsale people to their NginxPlus which offer a lot more status data in the binary.

SwellJoe• 10 years ago

We support nginx in our products, due to high demand for it, but we don't really like the way it has historically handled modules. Having to custom-build to do interesting things is very far away from the "plug and play" experience that our users expect and can get with Apache.

While I'm not on the bandwagon that says all Apache installations would be better on nginx (and I still run Apache on all of our servers, except nginx test machines), I do think this makes the case quite a bit more compelling, particularly for people who package and distribute the web server (OS vendors, control panel maintainers like me, container builders, etc.).

pnommensen• 10 years ago

We've just posted a blog post "Introducing Dynamic Modules in NGINX 1.9.11." https://www.nginx.com/blog/dynamic-modules-nginx-1-9-11/

We'll take a read through the comments here and are looking forward to answering questions and receive feedback. You may also email the NGINX development mailing list (info in blog post).

(I work @ NGINX).

lumannnn• 10 years ago

Hey :)

I've just created a separate HN submission pointing to your blog post.

I hope you don't mind.

Keep up the amazing work!

vbtechguy4• 10 years ago

Well got nginx 1.9.11 stack installer dynamic module ready almost with ngx_brotli fixed, lua nginx module patched https://community.centminmod.com/posts/26128/ just missing ngx_pagespeed compatibility and i'd be happy as a pig in mud :)

vbtechguy4• 10 years ago

Looking good so far I have enabled a few Nginx dynamic modules including Openresty's set-misc, echo and headers more modules https://community.centminmod.com/posts/26535/

  load_module "modules/ngx_http_image_filter_module.so";
  load_module "modules/ngx_http_headers_more_filter_module.so";
  load_module "modules/ngx_http_set_misc_module.so";
  load_module "modules/ngx_http_echo_module.so";
  load_module "modules/ngx_http_geoip_module.so";
  load_module "modules/ngx_stream_module.so";

Nginx 1.9.11 with Dynamic Modules

Related Stories