Should Apache replace its current httpd.conf with Lua?

http://thread.gmane.org/gmane.comp.apache.devel/37627

by compay • 17 years ago

58 47 17 years ago

thread.gmane.org

mgrouchy2• 17 years ago

Its a neat idea, but I would echo the sentiment brought up in the thread linked that casual users would have a much harder time with a lua config in comparison to the current one.

Its probably a good idea to move in this direction in the long run, but there is always a risk of displeasing your current users.

evgen• 17 years ago

Given the complexity to which apache and its ecosystem has grown and the consequences for screwing up this configuration task, this seems quite similar to sendmail in that I think a reasonable argument can be made that casual users should not be configuring this particular system. Period.

mgrouchy2• 17 years ago

You can make that argument, but reality is that casual users do configure apache so it ends up becoming a concern of the developers whether they want it to be or not.

Like I said, in the long run, it might be better to make the switch, but a change can cause significant risk/inconvenience to casual users or users who have such complicated configurations it would require significant time to replicate in another format.

scott_s2• 17 years ago

I wonder if the people who made those comments are in denial about the complexity of Apache configuration files. Having never seen one, I Googled "apache sample config file" and I came up on this page: http://www.pantz.org/software/apache/apache13config.html

This is representative of the more complex entries:

  <IfModule mod_alias.c>

    #
    # Note that if you include a trailing / on fakename then the server will
    # require it to be present in the URL.  So "/icons" isn't aliased in this
    # example, only "/icons/".  If the fakename is slash-terminated, then the
    # realname must also be slash terminated, and if the fakename omits the
    # trailing slash, the realname must also omit it.
    #
    Alias /icons/ "/usr/local/apache/icons/"
    Alias /icons/ "/var/www/icons/"

    <Directory "/usr/local/apache/icons">
        Options Indexes MultiViews
        AllowOverride None
        Order allow,deny
        Allow from all
    </Directory>

    #
    # ScriptAlias: This controls which directories contain server scripts.
    # ScriptAliases are essentially the same as Aliases, except that
    # documents in the realname directory are treated as applications and
    # run by the server when requested rather than as documents sent to the client.
    # The same rules about trailing "/" apply to ScriptAlias directives as to
    # Alias.
    #
    # ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin/"
    ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

    #
    # "/var/www/cgi-bin" should be changed to whatever your ScriptAliased
    # CGI directory exists, if you have that configured.
    #
    <Directory "/var/www/cgi-bin">
        AllowOverride None
        Options ExecCGI
        Order allow,deny
        Allow from all
    </Directory>

    #
    # "/usr/local/apache/cgi-bin" should be changed to whatever your ScriptAliased
    # CGI directory exists, if you have that configured.
    #
    # <Directory "/usr/local/apache/cgi-bin">
    #   AllowOverride None
    #   Options None
    #   Order allow,deny
    #   Allow from all
    # </Directory>

  </IfModule>

That is a language, and it's one that exists solely for Apache config files. I think using an actual programming language would be easier for most people.

silentbicycle4• 17 years ago

http://people.apache.org/~rbowen/presentations/apacheconEU20...

Right, and it's a language with a brittle, inconsistently documented, largely ad hoc syntax. While Lua's syntax has its quirks (the syntax for lambdas is a little cumbersome, arrays are indexed from 1), that's because it is designed to be straightforward for simple use by non-programmers. The advanced features* don't get in the way if you're only using it as a log/config/etc. file format.

* Tail-call optimization, closures, coroutines, etc.

yesimahuman• 17 years ago

In your example though, the complexity comes from having to know what things like "Order allow,deny" are. If you moved to lua I assume they would still use the same terminology, you would just have to represent it in lua instead of the current format. I think a file like this is easier for someone who has even a little bit of HTML experience to understand versus lua.

boredguy8• 17 years ago

Programmatic configuration solutions seem easier to me in a Lua world than the current .conf approach, which means it would be easier to develop tools for the casual user.

silentbicycle4• 17 years ago

Small thing, but it's "Lua", not "LUA". It means 'moon' in Portuguese, etymologically related to 'luna'/'lunar' in Romance languages. (Lua is from Brazil.)

garnet72• 17 years ago

Surprised to see so much discussion on that. Instead, would expect someone to simply implement it, throw it on the table, and provide instructions for building it into your own apache.

If it catches on, then you have a replacement to the current config syntax.

That said, in my experience, admins are slow to change. There are still sites running Apache 1.3 out there.

tvon3• 17 years ago

True, though I think it's probably a very large undertaking for a single person. Bringing it up on the mailing list would be a good way to gain mindshare and contributors.

doki_pen• 17 years ago

Open source projects usually fair better with one author until they are somewhat stable, in my experience.

fsniper2• 17 years ago

Admins are not slow to change. Operations and Corporations are slow. Admins generally have more than one version of any software running. More than you can imagen.

Generally they do have development versions daily updated and built and tested under the hood. Generally they do provide support and feedback to these software by bug reports or even with changesets.

But they are not slow.. most of the time they do work out of your eye sight.

I can bet some admins are trying to see if mod_lua can be a good replacement for the current configuration.

For myself I do not like it to happen. Because most of the deployments of apache are huge and this will break too many things.

And yes, if it's working do not change it :)

(edit: gramer and typo corrections)

randallsquared• 17 years ago

As someone who has been an admin (both with and without that job title), I'll say that I was slow to change, and often still am. Why? Because every change is an opportunity for something to go wrong, and in my experience, something goes mildly wrong about half the time, and catastrophically wrong way more often than it should. This is after doing all the testing that can usually be done. Either there's no testing server for this client, or you can't replicate the exact conditions of the change because you don't have another Windows 2003 SBS handy to test interaction with, or you run into lots of issues in the test and can't easily roll back to make sure that you can do it flawlessly this time...

Unless you've got an employer or client that can afford you for an additional 20 hours for each major change, everyone involved just crosses their fingers and holds their breath. I've never been an admin in an environment where there was adequate funding, though, so maybe it's just me.

Anyway, I typically put off upgrades and new installs that affected old servers (new apache modules or whatever) as long as possible.

Edit: this is why I just code, now. It's easier to write stuff that works than to find the magic this-THEN-that incantation to make someone else's stuff work. ;)

snorkel2• 17 years ago

The way to encourage adoption is just provide new functionality only possible using the new config syntax. Need dynamic process pool control? Here's a Lua example...

lamby• 17 years ago

It might be a harsh stereotype, but it has been said that the Apache project is an exercise in bureaucrats doing open source. Alternatively, it's an ideal outlet for bikeshedding.

est• 17 years ago

It would be cool if we can use runtime variables in Apache's config

silentbicycle4• 17 years ago

While this may sound like it's coming out of nowhere, Lua was designed primarily for use in extension/configuration, and has been used in production this way for about fifteen years. It's a scripting engine, config file format parser, JSON-like data format library, etc., all in a library a bit smaller than SQLite.

The lua interpreter is essentially just a "code = read(); f = compile(code); print f()" loop written in the C API, with better error messages. The C API is primary, not the standalone language, and using it with existing C projects is quite easy. (As a language, it's very much like a twin of Javascript that was able to mature for another decade, rather than suddenly getting standardized. Most of the flaws Crockford describes in "Javascript: The Good Parts" were fixed in long ago in Lua.)

More specifically relevant to apache, running code in a sandbox is trivial. You can use setfenv() to control the environment, global packages, etc. available to any code. If code shouldn't access the compiler, OS or debug packages, etc., just pass it an environment without them.

There's also a chapter in _Lua Programming Gems_ (http://www.lua.org/gems/) (it's "Lua as a Protocol Language" by Patrick Rapin, no sample PDF though) that shows how to set up a simple filter at the bytecode level to block loops, guaranteeing that code will always terminate. With a few minor changes, it's completely safe to use as a network protocol language.

pilif2• 17 years ago

I wonder what is wrong with the current way of configuring apache. I might be biased, but I have experience with both lighttpd and apache and I always found the apache way of doing things perfectly fine and actually prefer it to the kind-of procedural style of lighty.

Let's just hope they don't switch to an XML based configuratino file. We don't need even more filler-code in our configuration files.

XML just isn't human readable (and writable! and, yes, you DO write configuration files here and then) enough to be convenient. IMHO at least.

compay3• 17 years ago

As far as I can tell XML config files (at least those intended to be hand-edited) have largely fallen out of favor in non-Java open source projects.

I also don't mind the current config file, but I've been using Apache for more than 10 years so I've probably lost all perspective at this point. Having the config file in Lua would certainly make it eaiser to automate, validate and document though.

pilif2• 17 years ago

well... the XML approach was in fact proposed in the linked thread.

mod_perl already allows for a totally dynamic configuration which should be sufficient in terms of automation.

The documentation is quite perfect already and would IMHO lose quality if the syntax changes this drastically.

The only point I'm willing to hand you is the validation argument, although the current parser seems to be doing a decent enough job already.

No. Discussions like these feel to be motivated by artistic considerations instead of practical ones. Think of all the external documentation such a step would invalidate. Books, Tutorials, Blogs. And think of the external addon modules that would probably break because of this.

Remember the 1.3 => 2.0 switch and how long that took and consider that 2.0 actually had some real advantages compared to 1.3.

dkarl• 17 years ago

As far as I can tell XML config files (at least those intended to be hand-edited) have largely fallen out of favor in non-Java open source projects.

I think it's too bad that people seem to be diverging towards either a crappy, barely readable data format or full-fledged programming languages. The advantage of XML is that it's a well-specified and reasonably flexible way of representing structured data. You can interpret an XML document as code (e.g., XSLT) but basically XML is just data until you say otherwise. Configuration file formats should be dumb. You shouldn't be able to program your configuration file to play chess. The only significant downside of XML is its verbose and barely readable syntax, and you don't need a configuration file that has its own byte compiler and runtime system to provide readable syntax.

simonw• 17 years ago

I've been using Apache for nearly a decade and it still takes me potentially hours of trial and error to figure out how to do something in an Apache config file. I'm pretty sure I'm not a stupid person, but there's something about Apache configuration that just doesn't fit in my brain. The Lua examples in that thread were instantly obvious to me, and I've never written a line of Lua in my life.

mst• 17 years ago

Thing being ... mod_perl let you do most of your configuration in code years ago. Yet I've never done that, and nor have the vast majority of mod_perl users I know (and bear in mind these are people self-selected to already like writing perl, so language wars probably don't make a lot of difference to my argument :).

Configuration isn't code, and I don't really think trying to make it such is necessarily a good idea - I suspect it seems superficially attractive to developers but with my sysad hat on it really doesn't seem wise.

growt• 17 years ago

I prefer the nginx config style over apache2. Why the hell would anyone need a turing complete configuration file syntax?

barrkel• 17 years ago

Turing completeness is a red herring and not an argument against using a language. Turing completeness is usually practically useful only to prove that certain analyses are impossible.

As to why, sometimes you want the flexibility of inferring settings from third-party data sources, or implicitly from the layout of the directory structure, or naming conventions, or any other thing you choose to mention. IMO the power of a programmable configuration language is that the configuration need not actually be written in the language, but stored somewhere else, and just retrieved using the language.

antirez• 17 years ago

I think that config files should be simple to write DSLs, and not programming languages. For automation what is really needed is an API exported by Apache via some kind of networking (a TCP socket could do the work) that allows to configure and change the behavior at run-time via a scripting language. This way you have the best of both the words.

ciupicri• 17 years ago

Have you looked at Augeas http://augeas.net ?

Augeas is:

- An API provided by a C library

- A command line tool to manipulate configuration from the shell (and shell scripts)

- Language bindings to do the same from your favorite scripting language

- Canonical tree representations of common configuration files

- A domain-specific language to describe configuration file formats

jrconlin3• 17 years ago

Wait, what?

key=value is really so hard that we need to replace m4 with lua? Honestly? Before you feel the need to flame, imagine if this proposal had been for implementing the config file in COBOL (a language designed to be code readable by non-technical people). Would you still believe it to be a good idea?

What ever happened to wanting to keep things simple?

gjm11• 17 years ago

m4? key=value? Have you ever either read or written an Apache httpd config file?

jrconlin3• 17 years ago

yep, on a daily basis. I also handle php.ini, java properties files and a slew of others. (Including the ever loving satan spawn of sendmail.cf).

All of which are simple key value pairs. Apache (and a few others) introduce scoping with psuedo XML, but again, it's not complex logic.

I brought up m4 because that was the last time someone tried to solve the configuration problem with a "simple and easy" macro language. Yeah, that worked great.

Layering Lua (or any language) as your configuration option will not simplify things. In fact, I'm willing to say that it will make it much, much harder as folks suddenly become increasingly "clever" about how they construct those config files.

I want my config files stupid. I want them heavily documented so that when I (or someone else) next has to manage them two years from now after working on multiple other projects in various languages and systems, they don't have to spend a week trying to remember what the hell this is doing, deciphering cryptic compile time errors, or wondering what the complex state machine construct someone layered on top of port allocations is supposed to be doing.

tvon3• 17 years ago

I like this:

http://thread.gmane.org/gmane.comp.apache.devel/37627/focus=...

> I would like to see the output of configuration be a struct (of structs), and the actual config files are never looked at again (until reload). At that point we don't need to care how it is built, and we can try using lua, vcl, xml, jelly, groovy, windows ini files, or plists :)

I wonder what's going on with this, I just noticed the whole discussion is from early June...

davidw• 17 years ago

Tcl's nice for this kind of thing too, as it's really easy to make your own "DSL"'s (or is that passe`?)

    directory "/usr/local/foo/bar" {
        options bee bop doo wah
        if { $someotheroption } {
            order allow deny
        }
    }

Lua might or might not be better for processing requests: it's sure fast. Config files don't really need to be fast though.

stcredzero• 17 years ago

Should [X] replace its [Y] configuration with [programming language here]?

Maybe. Depends. Is there a way you can try it out?

Reminds me of Zed Shaw's "The ACL is Dead." http://vimeo.com/2723800

I can also see that turning in to a quagmire.

hypermatt• 17 years ago

Haha I remember Brian from CNN he was talking about this years ago, I guess it takes a lot of momentum to change something as big as apache.

SwellJoe• 17 years ago

OK, for anyone who hasn't seen it before, this is what "second system effect" looks like just before it gets really ugly.

termie• 17 years ago

<Perl>

#!perl

use DBI;

use Blah;

...

push @{ $VirtualHost{'*'} }, {

        	ServerAdmin     => "webmaster\@$domain",
	        ServerName      => "$domain",
                ServerSignature => "Off",
	        ServerAlias     =>  \@aliases,
	        AddHandler      =>  [qw(cgi-script .cgi)],
	
	        DocumentRoot    =>  "/home/$domain/www",

	        Alias           =>  [
			["/cgi-bin/", "/home/$domain/cgi-bin/"],
			["/stats/", "/home/$domain/stats/"],
		],

}, ...

Should Apache replace its current httpd.conf with Lua?

Related Stories