Tir: A Mongrel2+Lua Micro-Framework

http://sheddingbikes.com/posts/1289384533.html

by helium • 16 years ago

120 55 16 years ago

swah2• 16 years ago

"The problem with Lisp is that it is acceptable to metaprogram until the only person who understands what you've created is you and The Flying Spaghetti monster."

Yeah, I guess this is sad but true. But as an alternative should I stop using macros and write lots of repetitive code?

zedshaw8• 16 years ago

The trick is to go far enough to improve readability, but not so far that the programmer has to remember all the metaprogramming or (worse) why you did it in the first place.

drats• 16 years ago

Speaking of "make things as simple as possible, but not simpler" what do you think of Flask?[1]

[1]http://flask.pocoo.org/

rincewind3• 16 years ago

What exactly is your problem with the Arc code? Is it that the code that handles the form comes before the code that writes the submit button?

The problem is not that you have to remember the metaprogramming. I do not know what exactly `aform` expands into, but it works for me and I know what it is supposed to do. (Yay, I can make an argument from ignorance too)

I would still rather use Mongrel2 than Arc in production, but readability is not the problem.

jules8• 16 years ago

I find the Arc version much more readable, even though I have never used Arc. It pretty clear that (aform <x> <y>) is generating a form with <x> as the target page and <y> the form itself. (onlink <x> <y>) is a link with text <x> and target page <y>.

I think you got a WTF because you're thinking about a web application as a bunch of HTTP requests, instead of thinking about it at a higher level. For example you're thinking about a link like this:

1. When the user clicks <a href="url-of-the-target-page">blah</a>, he gets sent to url-of-the-target-page.

2. When the webserver sees a request to url-of-the-target-page, it does X.

Arc does not force you to think at the HTTP level:

1. When a user clicks a link (onlink "blah" X), he gets X.

Also, your code is not 23 lines. You are not counting the templates, which will probably double your code. Does your Tir code handle multiple requests? Does it handle the back button?

greenlblue• 16 years ago

False dichotomy. It's possible to write readable and non-repetitive code. People do it every day without macro magic.

apgwoz• 16 years ago

The Clojure community tends to be in favor of using core data structures and functions to compose, simple, reusable things, instead of using syntactic abstraction because it's there (like some people encourage). Clojure code uses macros, but most of the time it's to tie stuff together into a prettier, more convenient interface--sometimes to remove repetitive code of course.

So, why not use macros all the time? Not because they're hard to understand--you can always expand them to see what they do (or ask the original author to document it), but because they are hard to compose, and reuse. Sure, they can be used in special case a, b and c, but give me a higher order operators, and suddenly I can compose an opus.

rincewind3• 16 years ago

Does this port to an imaginary arc-on-clojure framework look more readable to you?

    (arc/with-page-renderer
      (fn [content & options] (hiccup/html [:body content]))

      (arc/defop "/said" [request]
	(arc/aform
	 :form
	 [ [:input {:name "foo"}]
	   [:input {:type "submit"}] ]
	 :handler (fn [form-result]
		    (arc/onlink :text "click here"
				:handler (fn [_]
					   (str "you said: " (:foo form-result))))))))

swah2• 16 years ago

I was writing functions like this (Clojure):

  (defn set-password [{"login" login "password" password}]
    ...)

Ok, Clojure de-structuring syntax already makes things pretty terse: the function receives and de-structures a map. But for functions with many args it was looking repetitive, so I wrote defn* which lets me write the same as:

  (defn* set-password [login password]
    ...)

So, do you think this is a fair use of macros? It has the disadvantage of hiding the fact that the function actually receives a map, of course (but I'm a solo programmer here, trying to save keystrokes).

jrockway• 16 years ago

This is, of course, a problem with any language. Start naming functions pr and arg and everything quickly becomes unreadable.

The other problem is that there is really no way for the reader to intuit that there is a continuation or that state is being captured and passed around. Using the word "lambda" in the onlink continuation might help in that respect.

Not storing the request state in the global environment would also help readability. One thing that shocks me about programmign is how few people realize that you can pass arguments to functions, and they can return values. Instead of:

   (omg-magic)

Why not:

   (send-response (handle-request req))

Hey, now we know that the parameter is a request and the result is a response. You wouldn't write:

   (let ((a 2) (b 2)) (add-and-print-a-and-b))

You'd write:

   (print (+ 2 2))

So why does this common sense go out the window when the problem is more complex than adding 2 and 2? I wish I knew...

rincewind3• 16 years ago

There is no continuation in the arc example I think. The request is also not a global variable (the aform macro stores a closure in a global hashtable, maybe you mean that).

iampims2• 16 years ago

    I needed something real because you cannot build
    a useful web server in a vacuum.

I’d even go as far as saying that you can’t build anything in a vacuum. This is my main gripe about many sample projects for web frameworks. You get the traditional and useless hello world and the blog engine in 20s. Yet none of them highlights how to decouple your app, how to deal with non mundane stuff (security, error handling and reporting). Most of us, when learning about a new framework, have many years of experience with other frameworks, so why not try to sell us on how your framework is different and better suited to the real world situations we’re facing.

nakkiel2• 16 years ago

I found flask's documentation quite well-written in this regard.

For example, they provide insights at how to grow your application's structure in several ways: http://flask.pocoo.org/docs/becomingbig/, http://flask.pocoo.org/docs/patterns/packages/ and http://flask.pocoo.org/docs/patterns/appfactories/.

Generally speaking, http://flask.pocoo.org/docs/patterns/ gives a good overview of things to explore in Flask.

Edit: I elaborated a bit more. Please note that I'm not advocating anything, just giving my opinion on their documentation.

iampims2• 16 years ago

Indeed, I’m going through the docs as we speak and I’m impressed by the quality of the docs and the examples provided are interesting. Thanks for reminding me about Flask.

chipsy2• 16 years ago

This is true of many(most?) open-source library/framework/engine/API projects. They suck and don't do more than demos because they were written with insufficient perspective for anything beyond that. So do anything more, and they suddenly collapse.

nakkiel2• 16 years ago

Programming languages are about manipulating concepts. Give that Lua code to someone who doesn't have the necessary concepts (views, regexp-based routing, framework, dynamic websites, ...) and he'll be lost.

This rant just shows that the mighty Zed doesn't have the necessary concepts (and neither do I) to understand the Arc bit.

Edit: and obviously, reading the docs would help in this regard.

zedshaw8• 16 years ago

First, it's not a rant.

Second, programming languages are about making things. If I have to learn a ton of concepts before I can make things, then your language sucks.

anthonyb• 16 years ago

Depends on how useful the concepts are. If, say, macros or continuations save you a boatload of work and make your stuff easier, then "you language sucks" is a bit of a stretch.

Obviously there are different trade offs of learning/doing, depending on your tolerance for it and how long your things take to make. For an afternoon framework maybe not so much, but for a longer project learning a bit more upfront makes sense if it'll save you time down the track.

olliesaunders2• 16 years ago

It's actually OK if you need to learn a load of best practice before you're able to be productive. It stops people making a lot of mess.

cageface• 16 years ago

Cool hack. I really like what I've seen of Lua so far. It would be great to see it become more viable for web work.

I think Zed's right that web applications are more and more constructed in terms of interfaces than pages.

jgalvez4• 16 years ago

"NO TESTS. You must work at a startup."

LMAO. You have to hand it to Zed Shaw sometimes...

Now, seriously tho, this has been waiting to happen since forever. I always knew that when the right people started messing around with Lua, awesomeness would emerge.

Lua is the closest scripting language to C. It's faster than Python, Perl and Ruby[1]. It's not just faster, but also uses less memory. It's also extremely elegant (in language concepts[2], at least -- not too much of a fan of its syntax but whatever). But the fact is, apps running in Lua would be hard to beat in terms of single-node performance. Your $20 cloud box that struggles to keep hefty Ruby processes running (or even Python processes, though these are way smaller than Ruby processes) will suddenly be able to handle a lot more than it does today.

I think Lua for instance would make a lot of sense for the big ones: Google, Facebook, Twitter. Google has a lot of C++ code on its infrastructure[3]. Facebook resorted to C++, Java, Python, and Erlang[4]. Twitter took the Java path with Scala.

The more I build applications (and keep them running), the more I respect the Unix nature and the more I want to become proficient in C. Mastering a scripting language and knowing how to glue it to raw C, or something close to C, gives you a lot of power.

Imagine a Python+Lua stack, where you can have handlers written in Python and use small, specific Lua-based extensions to run complex computations. I'm already contemplating using something like Lunatic-Python[5] to accomplish that in an app where I have to create similarity-based clusters. Today the Python code that handles it takes about 15 minutes to run as a queued, background task. I'm very curious to see if I can bring that down with Lua-based code controlled by Python...

[1] http://shootout.alioth.debian.org/u32/which-language-is-best...

[2] http://www.lua.org/history.html

[3] http://www.quora.com/Ben-Maurer/Google-Infrastructure/answer...

[4] http://www.makeuseof.com/tag/facebook-work-nuts-bolts-techno...

[5] https://github.com/dmcooke/Lunatic-Python/blob/master/docs/l...

mhd2• 16 years ago

"Lua is the closest scripting language to C"

Hmm? Neither syntax nor semantics are really close to C, and considering the plethora of C and C-like script interpreters, this strikes me as a pretty odd statement. Or do you mean in regard to binding/FFI?

jgalvez4• 16 years ago

Binding, definitely.

jules8• 16 years ago

If you're just doing Lua to get to C it's much easier to call to C directly from Python.

silentbicycle• 16 years ago

You can call C directly from Lua too, with alien (http://alien.luaforge.net/).

Generating the bindings really isn't that hard, though.

jgalvez4• 16 years ago

Not to get to C, but simply to get somewhat better speed, /somewhere/ between Python and C, which is where Lua stands.

weeksie• 16 years ago

Strange, I haven't done a lot with Python but I've built a pretty big system in Lua and its FFI is incredibly nice, the nicest I've ever used, in fact.

olliesaunders2• 16 years ago

It would surprise me if this was true because C interop and embeddability was part of the reason for the creation of Lua. I don't believe you could say the same was true for Python.

Wikipedia: "Lua is intended to be embedded into other applications, and accordingly it provides a robust, easy to use C API. The API is divided into two parts: the Lua core and the Lua auxiliary library."

mhd2• 16 years ago

Looks interesting. I found Lua to be a wonderfully predictable language, both regarding semantics and syntax. Which is quite nice, in an age of decorators, macros and weird histories (looking at you, JavaScript).

As opposed to Zed, I would say the same about Lisp, though (if you're a mature programmer, who went beyond the first excitement about macros).

jgalvez4• 16 years ago

Dear Zed Shaw, can you change the <title> tag of your posts to the actual post title and not the blog title? Always annoying to have to change it when I save your articles to Pinboard. But most of all, it's annoying because it's like, just wrong, dude! :D You can also tell me to go away and that's it's your blog and you keep it the way you want to. (That's fine too.)

zedshaw8• 16 years ago

Yeah I gotta work on that code some.

dangoor• 16 years ago

The idea of process-per-interface is an interesting one. My first thought was that this has the "PHP problem" which is to say that if you have a bunch of useful utility classes and helper code you have to optimize those to minimize loading. In a framework like Django where an app is reused between requests, you can have a lot of useful framework available at your fingertips.

My second thought was more interesting than the first, though: Tir shows the value of Mongrel2 (or, at least, Mongrel2's approach). You can use the process-per-interface structure in places that make sense, and use Django-esque structures elsewhere.

Of course, Mongrel2 is not the only way to do it and there are sites like Amazon where they take the approach of packing the results of many requests to separate services into a single page, rather than having a single process be responsible for the page.

I find that these days the hardware's good enough that most apps that people create never make it to the "scaling problem" anyhow.

meric• 16 years ago

Very interesting...

Made me jot down a non-mini, non-existent web framework I think I might like to use.... It's only just a tad bit more compact than Tir and will take a lot more effort to implement... but (I hope) it is much easier to tell at a glance what it is doing. Must find time for building my own web framework some day.

Corresponding Arc Challenge: (cheated a bit, if you want to make this 'proper' you'd need to add a cookie, and maybe another function.)

  require "filter"
  require "form"
  require "response"
  require "route"
  said_form = form.default { saying = form.charfield() }
  function said_page(args)
    return response.http(said_form.render())
  end
  function said_click(args)
    return response.http(
      ([[<a href="javascript:document.write(%s)">click here</a>]])
          :format(args.post.saying))
  end
  route.set(filter.get, "^/said/$", said_page)
  route.set(filter.post, "^/said/$", said_click)

  require "filter"
  require "template"
  require "form"
  require "widget"
  require "response"
  require "route"
  require "generic"
  require "css"
  
  login_form = form.default({ 
                  username = form.charfield(), 
                  password = form.charfield{widget = widget.password} 
                  }) 
              
  function login_form:clean(args, data)
      if authenticate(data.username, data.password) then
        return true
      end
      args:set_error(self, {'username or password is incorrect'})
      args:set_data(self, args.post)
      return false
  end
  
  function login_check(request)
      if request.session['user'] then
        return nil, response.redirect("/")
      end
      return filter.get(request)
  end
                              
  function login_page(args)
      return response.http(template.default.render_with {
          head = css.link("my.css"),
          body = login_form.render()
      })
  end
  
  function login_auth(args)
      if login_form:clean(args, args.post) then
        return response.redirect("/")
      end
      return login_page(args)
  end
  
  route.set(login_check, "^/login/$", login_page)
  route.set(filter.post, "^/login/$", login_auth)
  route.set(filter.any, "", generic.not_found)

zedshaw8• 16 years ago

Nice, yeah it's really not hard to crank these little frameworks out. In fact I'm starting to think, if doing one is so simple, then maybe just little frameworks from now on.

One thing though: Part of the point of Tir is that you can read the code to one function and know what it's doing. For example, if I showed this to someone and asked, "What happens after faile to login?" They'd have to trace through routing to figure it out.

Not that I'm saying that's bad, since the advantage is you don't have to save any state. I'm just saying that's what's different about Tir (and Arc or Seaside).

chriseidhof3• 16 years ago

Good stuff, I think Lua might be an excellent choice for this. Now for some shameless self-promotion: a year ago I posted a Haskell solution to the Arc Challenge, which might be even more readable for non-programmers: http://news.ycombinator.com/item?id=1004701

zedshaw8• 16 years ago

Yes, but did it pass the ArcWTF Challenge. You should try it. It's actually pretty funny, but you also learn that maybe a bit more code is better for the brain.

chriseidhof3• 16 years ago

I should try it, thanks. I don't think I agree with that you should optimize your readability for non-programmers, I think you should optimize (mostly) for programmers.

However, I know a guy who is a consultant, and in one project he built a small DSL to describe the domain of the client. Next, he sat down with them and walked them through it, and it was very easy for them to spot mistakes or logical flaws in the code. At first they thought he was crazy, but ultimately they loved it. So I do see your point, I'm just not sure if it's good in every situation.

jules8• 16 years ago

Beating the ArcWTF challenge:

    page '/said' do
      form do
        message = input()
        submit do
          link("click me"){ text("You said #{message}") }
        end
      end
    end

Show this to a non coder and I'm pretty sure they'll recognize more than if you show them the 40+ lines of his Lua version.

zedshaw8• 16 years ago

Well the ArcWTF challenge was sort of a joke, but I think this code isn't better.

The big problem is you (and Arc and others like it) imply too much historical knowledge in the name of terseness. What happens is when someone has to fix the thing they have to dredge up all the knowledge of where things are rather than finding them right there.

For example: If I have to come and fix the template that has the input form in your code where do I go? Do I edit some code deep inside your framework? Is there a page somewhere? Is it in HTML? Oh it's not? So I have to hire a guy who is a designer and a Lisp or Java or Smalltalk guru? Ok there is a template, so where is it? Is that configured somewhere so I have to one more level of indirection to find the thing that configures the thing that shows the page?

If you back off the tersness a bit, you can then have a piece of code that's still small, but also complete by itself. Anyone coming to my code sample and told, "Fix the damn submit button so it's cornflower blue." Can do it no problem because it says right there what file is being used in what way.

Additionally, I don't buy it with these "yours is 40 lines haha" challenges. Who the hell cares? It's more about the usability and ease of maintenance.

jules8• 16 years ago

There is no template. That is the code. All of it.

I do not think it is hard to follow at all. Definitely easier than your Lua code. It translates right into the description of what it's doing:

    On the page '/said' display the following:
      A form containing:
        an input box named message
        a submit button that goes to the following page when clicked:
          a link with text "click me", that goes to following page when clicked:
            a text saying "You said #{message}"

The problem you have with it is likely that it seems too simple. You're looking for additional code that isn't there. This way of coding with continuations as in Arc and in my snippet has many problems, including scalability, memory usage and bookmarking urls, but simplicity or understandability isn't one of them.

Tir: A Mongrel2+Lua Micro-Framework

Related Stories