Improve your beta-testing with Nginx, Lua and Redis

http://devblog.mixlr.com/2012/06/26/how-we-use-nginx-lua-and-redis-to-beta-ify-mixlr/

by transmit101 • 14 years ago

72 16 14 years ago

devblog.mixlr.com

richardv• 14 years ago

This is a fantastic explanation. We have started to do something similar but your LUA rewrites is a lot more effective. I wasn't even aware that you could do something like that.

runningdogx• 14 years ago

With a modern pcre library version and nginx, instead of

    location ~ ^/([a-zA-Z0-9_]+)/ {
      set $username $1;

you can use named captures

    location ~ ^/(?<username>[a-zA-Z0-9_]+)/ {

transmit1015• 14 years ago

Thanks. I didn't know that. I did strip out a lot of the more advanced/ambiguous Nginx config for clarity though.

newman314• 14 years ago

Newer versions of pcre and nginx also include the ability to use the pcrejit which provides a nice speedup for regex.

grogs• 14 years ago

Good write up for a sensible solution to a problem which should effect lots of sites.

Personally, I would probably take the approach of settings a cookie per user. This would be very suitable when beta testing is enabled/decided by users, rather than a subset being chosen A/B style based on a value in the database. This would have the advantage that nginx would not have to make any database requests; just check a cookie on the request, likely speeding up the speed at which nginx handles requests. However, it would require some changes to the application (versus [small] changes to the database which the slug approach) to set the cookie etc.

I wonder how easy this is to do in apache.

transmit1015• 14 years ago

Cheers. Yeah a cookie approach would be suitable for a lot of uses, including A/B testing. It's easy in Nginx/Lua:

local is_some_cookie_set = nginx.var.cookie_SomeCookie ~= nil;

Then you don't need Redis at all. For us though, we want to separate users by the URL and then a check at an application level variable, so I think this is the right approach still.

projct2• 14 years ago

Couldn't you use a cookie and map, rather than URLs? Sounds like users could end up on different rails back ends unintentionally based on what URL they use, with your method...

http://serverfault.com/questions/268633/controlling-nginx-pr...

transmit1015• 14 years ago

As I explained in the post, that's the intended behaviour.

We want to allow our users to enable beta version for all visitors to their Mixlr page, not just for themselves.

But a cookie approach would be easy too - see my other comment for a simple example.

projct2• 14 years ago

ah, I see. I guess I didn't quite understand the product enough. Neat idea, and thanks for clearing that up.

However I wonder if map is faster than using lua just for cookies. I'll have to test it. :)

jperras• 14 years ago

We've done this before, but by adding an additional load-balancer that forwards requests to a different port (e.g. 81 instead of 80) depending on the subdomain that was accessed. The request is processed by nginx, which has separate server blocks and associated upstreams for processing requests that come in over the configured ports. The only application logic that was necessary to accommodate the changes was a check to determine if the user should be part of the beta, and redirect them to the appropriate subodmain if necessary.

While it's not as clean as having the check done in nginx itself via Lua script + Redis, it's relatively low friction. That said, the Lua script embedded in the nginx config is very slick, and I had been contemplating writing something similar using the nginx memcache module. Nice work.

transmit1015• 14 years ago

Yes that works too as I mentioned in the first couple of paragraphs of the post.

However having a beta.mixlr.com domain a) introduces a lot of friction and seriously reduces the ease with which you can test features b) may alter the perception or expectations of a visitor, making testing less valid.

We've got a lot of users and we want to make the roll-out process completely transparent and inclusive.

Hence this solution.

davnola• 14 years ago

Lovely. We do something very similar in front of EC2 clusters with http://openresty.org/

eli• 14 years ago

I've done very similar things with Varnish reverse caching proxy. And as a bonus, Varnish is crazy fast if your pages or assets are cacheable.

Its scripting language is very simple (not quite as nice as Lua, IMHO), but it supports multiple backends and rewriting arbitrary headers: https://www.varnish-cache.org/trac/wiki/VCLExamples

mqzaidi• 14 years ago

We have also done similar things to test our new platform, using nginx and cookies, by sending part of the traffic to the new platform to be beta tested. Didn't need to use lua though. http://qzaidi.github.com/2012/03/11/nginx-ab/

Androsynth• 14 years ago

rather than: if (-f $request_filename)... you should use try_files

http://wiki.nginx.org/IfIsEvil

http://wiki.nginx.org/Pitfalls#Using_If

transmit1015• 14 years ago

Thanks, it's a good point, I will update the post later.

Improve your beta-testing with Nginx, Lua and Redis

Related Stories