Lisp for the Web, Part III

by VitoVan • 11 years ago

105 61 11 years ago

eudox4• 11 years ago

You can't build a web application on Hunchentoot -- a basic HTTP server -- and call it modern. Take a look at Clack for the actual modern approach: http://clacklisp.org/

Or one of the frameworks built on it:

http://8arrow.org/caveman/

http://8arrow.org/ningle/

http://eudoxia.me/lucerne/

jlg23• 11 years ago

You can't build a web application on Hunchentoot -- a basic HTTP server -- and call it modern.

The article does a pretty good job of starting with the basics and introducing more complexity step by step. I would argue that the author's approach is better than any attempt to introduce a full-stack framework, which does not help the beginner level lisp programmer (clearly the target audience) understand - one might take away some recipes from that but no real understanding.

Which, if any, "modern" frameworks one uses then finally depends on the project and taste. After all, gluing together the various routing-, persistence-, ORM-choices only takes a few lines of code anyway.

And, after having been consulting as a lisp programmer for some years now, I would even argue that the overhead of learning some full-stack frameworks is usually not worth the benefits they claim to provide.

kerkeslager3• 11 years ago

And, after having been consulting as a lisp programmer for some years now, I would even argue that the overhead of learning some full-stack frameworks is usually not worth the benefits they claim to provide.

I'm not an experienced Lisp developer, but this jives with my experiences with frameworks in other languages. It seems like frameworks cover 95% use cases: they make your life easier for 95% of what you're trying to do. But for the other 5%, you can't just work outside the framework, because frameworks couple you to their paradigm. So you end up trying to solve problems through the framework that the framework doesn't have good solutions for. You're stuck solving problems without the debugging tools or basic building blocks that writing your own in a programming language offers you; it's like trying to tighten a screw with a hammer in the dark. The result is that you end up spending a lot of time on that 5% of problems.

What makes this particularly dangerous is that you typically don't start solving the really unique problems of your project until you've already gotten pretty far into your project: you don't run into the 5% problems until you've already committed heavily to using the framework. A lot of less experienced devs get caught up in this: they start using a framework and everything seems easy. Drunk with their newfound power, they try to do everything in a framework, and each new project reinforces their belief that the frameworks are saving them time. It takes a long time to be on enough long-running projects that you see the pattern and recognize frameworks for the double-edged sword that they are.

To be clear, I'm not saying that frameworks are always bad. For simple or short-lived projects, a framework is probably the best way to go: you'll probably never hit those 5% cases where the framework will hurt you. But if you're doing something at scale, or for a long time, or doing something very complex: a framework might not be the best approach. And large-scale, long-running, and complex projects are where the money and job stability are at.

There is definitely a middle ground to explore here. For example, in the Python world, there are some great minimalist frameworks (Flask, Bottle) which can be tied together with a template engine (Cheetah, string.Template) and SQLAlchemy to give you most of what a big framework like Django or Pylons gives you, but in a much less coupled way that allows you to create your own extensions points. Libraries are much more flexible than frameworks. Sure, you'll feel more pain up front, but you won't spend your time trying to get an ORM to emit a specific SQL query, or waiting for your cache middleware library to accept your PR on a 10-year-old bug in its invalidation.

lispm3• 11 years ago

What makes it 'modern' and not just newer?

eudox4• 11 years ago

Because in the real world of web development applications are hardly ever written directly on top of the server. They are written on a framework, that sits on top of an HTTP server abstraction (like Clack), that has pluggable servers.

That way, if a new server comes out (e.g. Woo[0]), you don't have to learn all of its internals and rewrite the entire application using it -- you literally just change a keyword argument in a single function call, and now your application is running on a different, much faster server. If someone writes a plugin for Clack (e.g. clack-errors[1]), all applications built on any Clack framework using whatever server can use it. A Hunchentoot plugin works for Hunchentoot applications.

Using Clack is sustainable web development.

Sure, for a five minute demo, Hunchentoot is fine. But why teach bad practices?

[0] https://github.com/fukamachi/woo

[1] https://github.com/eudoxia0/clack-errors

copsarebastards2• 11 years ago

If this is true, then this explains why all the "modern" websites are crappy media websites that don't solve any problem except "How do I drive traffic to ads".

jasonm23• 11 years ago

Also ditched reading as soon as Hunchentoot was mentioned.

olewhalehunter• 11 years ago

For anyone interested in having their application and web testing in the same runtime, I'm making a CL web automation library/front-end out of emacs for testing and botting purposes. https://github.com/olewhalehunter/kommissar

616c• 11 years ago

It is not clear from the sample screenshot, but do you know of Conkeror? Sounds like your wet dream!

http://conkeror.org/MozRepl

aidenn02• 11 years ago

This is a reasonable way to get a simple hello world up, but for those new to lisp, know that this isn't how most people develop.

Firstly, it is rare to see the SBCL REPL used directly.

Most commonly, lisp is developed via an IDE that will include a full-featured REPL (there are several; for sbcl SLIME is what is most commonly used. ClozureCL has its own IDE on OS X, and commercial lisp implementations will ship with their own).

GNU clisp has a more feature-full REPL, so you may see that used occasionally.

Secondly, lisp has a build system called ASDF. There is a tiny amount of code needed to setup a simple project, and there is a tool named "quickproject" that will generate it all for you. With all the improvements to ASDF in the past decade, I never do a "write lisp file, then load it" anymore.

CodyReichert2• 11 years ago

cl-project[0] is another nice project generator. It's mostly pretty similar - but I like the default layout and having a built-in test suite.

0 - https://github.com/fukamachi/cl-project

WalterGR• 11 years ago

   GNU clisp has a more feature-full REPL, so you may see that
   used occasionally.

For web programming? My understanding is that clisp doesn't support any kind of threading, which definitely makes it impossible to use at least Hunchentoot. And since this article is about Hunchentoot, my understanding is that using clisp will make it literally impossible to follow this article.

EDIT: It looks like clisp has experimental threading support if you compile it correctly: http://www.clisp.org/impnotes/mt.html

So I guess the better question is: does Hunchentoot work on clisp?

aidenn02• 11 years ago

Regardless of whether hunchentoot works on clisp, the point was if you aren't using CLISP you're probably not developing in the REPL in a terminal.

mck-• 11 years ago

In case anyone might find it helpful to have another example: I wrote a similar post a couple years ago: https://kuomarc.wordpress.com/2012/05/13/12-steps-to-build-a...

Scarbutt3• 11 years ago

Don't know how are things on the Common Lisp side, but on the Clojure side, as much as I prefer it to Javascript I end writing regular web applications in Javascript, because for me, it is the more pragmatic choice when I need to get something done. Javascript/nodejs has lots of good mature libraries with good documentation for anything related to web dev, sponsored by companies that eat their own dog food (ex: the passportjs lib).

TeMPOraL2• 11 years ago

My impression is that CL has much less libraries, but they're generally much better in quality. Having a really powerful language at your disposal also alleviates some of the pains of not having enough "batteries included". But when doing web, it's hard to avoid JS anyway (ParenScript in CL is cool and all, but you still need to make sure the JS it compiles to actually works).

Scarbutt3• 11 years ago

A more powerful language won't help much in getting things done if there is lack of libs in a domain you are not an expert on.

For example: implementing authentication and authorization.

So I agree with everything you said but I'm mostly talking about getting shit done.

kerkeslager3• 11 years ago

Do you have an RSS feed for your blog?

VitoVan2• 11 years ago

Sorry, not yet.

VitoVan2• 11 years ago

Here: http://vitovan.com/rss.xml

It's still hot, I added the code for generating rss one minute ago.

kerkeslager3• 11 years ago

Haha thanks!

hacker_95• 11 years ago

Why Lisp? Again It is awesome.

Why do people who write or talk about Lisp always have to start with something like this? To be honest I still can't see the appeal of Lisp. Yes everything is a function so you can add/remove 'language features' as you want, and the syntax is really simple, and code is data (a rare need of mine anyway). But Lisp sacrifices the most important thing of all; readability.

Everyone tries to read programming languages as close as possible to plain english in their head, but Lisp goes completely against this and makes you write things like (* 1 (+ 2 (/ 3 4))) where you have to glance back and forth to even understand the equation - is 1 * (2 + 3/4) not easier to read? Even in C# the phrase 'if (!(foo is bar))' annoys me because it reads 'if not foo is bar' when I'd like to say 'if foo is not bar'! So I think me and Lisp have no hope.

klibertp4• 11 years ago

Yes everything is a function

No. Read on macros.

and code is data (a rare need of mine anyway)

Rare? Read on macros, then on "Blub paradox".

But Lisp sacrifices the most important thing of all; readability.
Everyone tries to read programming languages as close as possible to plain english in their head

No. That's too wrong to even seriously argue against. The best I can do is to paste this: http://blog.ppenev.com/parens.png without any comment.

The rest of what you wrote is completely subjective and I don't want to comment on it. But I think you should go and take a look at Smalltalk, which is another awesome langauge with 4 decades of history, which pretty much reads like plain English.

hacker_95• 11 years ago

I have read through Paul Grahams essays a few times and do respect his opinion, but I just can't see it myself.

The parenthesis are actually the least bit I have a problem with (as the indentation generally makes flow clear). But the actual function within a function with a function syntax and so on just seems to me to hinder the reading of the code and thus the understanding, maintenance and modification of it.

CodyReichert2• 11 years ago

That is completely subjective, and your example is bad. What's more readable here...

(+ 1 2 3 4 5)

1 + 2 + 3 + 4 + 5

In my (subjective) opinion, the first one is. The extra +'s are redundant and noisy, and sacrifice a meaningful syntax for "trying to read as plain english". The syntax of any language takes practice and lots of reading - that's not unique to Lisp.

hacker_95• 11 years ago

Many languages do follow a similar syntax though, that do relate to how plain english is written. I personally started with C#, and when I look at other languages I don't find them very hard to follow because they also follow the same basic 'read left to right' rule. Even coming across data formats such as html/xml for the first time are easy to follow because of the left to right reading.

In your example, the 2nd one is actually what I'd prefer to read as I know that it will be compiled down to 5 ADD assembly instructions, whereas the first looks like it will be turned into a function call that will then initiate a loop. I'm guessing/hoping Lisp optimises the (+) function though so probably a null argument from me. See my reply to nnq for a better example.

Retra• 11 years ago

This is what I see when I look at lisp code:

    (doof-boof "1 3" f1 3 "f1 3" 'f "f3 " f")

I love lisp, but I love commas too.

renox• 11 years ago

The first one is still flawed: +(1 2 3 4 5) or even better +[1 2 3 4 5] would be better IMHO as it visually separates the function from the arguments.

nnq2• 11 years ago

You're probably a verbal thinker, and English-like code matters a lot to you I guess. Probably lots of programmers are.

For me, a mostly visual thinker that ends up building and visualizing a code tree in his head anyhow, Lisp is very readable because I can more clearly see the tree without all the noise.

And when it comes to math, staying close to standard math notation is a clear winner, and this is why all lisps have ways of writing infix for formula-heavy code, stuff like http://data-sorcery.org/2010/05/14/infix-math/ and others. But lispers don't use them that much because they simply get used to the Lisp notation more.

...oh, and Common Lisp is NOT an example of readable Lisp. It's probably the most unreadable and obsfucation-friendly of them all :)

Take a look at some Clojure code for something that's just as readable as Python after you "get you eyes used to it" (jargon for learning to ignore the parentheses and learning the special meanings of the "funny symbols" like :, ~, # and a few more): https://github.com/weavejester/compojure/blob/master/src/com...

hacker_95• 11 years ago

I agree I am a verbal thinker, but that is only because I have to read the words off the screen to begin with - once I have the english in my head my mind switches between verbal and visual images to figure out the next part to write/how to fix the bug etc.

Anyway to take an example from your github:

if (and form-method (= request-method :post))

am I right in saying this is pseudo-imperative:

if (form-method && (request-method == 'post'))

To me the ability to read it left to right seems like a huge win. With the lisp version I seem to be holding the program in my head until the very last (= ...) just so I can work it all out. Perhaps that is just in-experience though.

lispm3• 11 years ago

...oh, and Common Lisp is NOT an example of readable Lisp. It's probably the most unreadable and obsfucation-friendly of them all :)

Actually typical Common Lisp is much more readable than Scheme or Clojure. 'obsfucation' is built-in in Clojure, where it has to be programmed in Common Lisp. Scheme has great potential for unreadable code, given it is a Lisp-1 and it makes heavy use of recursion and higher-order functions.

Common Lisp OTOH favors: long descriptive names, descriptive keyword arguments, CLOS based data structures (instead of untyped lists and vectors), explicit loops, explicit calls to variable bound functions, ...

ruricolist• 11 years ago

Does anyone read actually read code left-to-right, line-by-line?

I for one start with the shape of the code -- the pattern of color (from syntax highlighting) and negative space (from indentation) -- and only then focus on the different parts.

And Lisp is particularly helpful this way, because instead of paragraph-like blocks of code where you have to reconstruct the state of the code in your head by mentally modeling side effects, you can see the general outlines of what's happening from the levels of indentation.

I want to defend Lisp, of course, but I'm also genuinely curious about different ways people read code.

hacker_95• 11 years ago

I agree that the shape of the code is what I start with too: the syntax-highlighted keywords stick out as anchor points when scrolling, and I know from indentation where sections start and end.

Example:

  if (...)
  {
     ...
  }
  else
  { 
     ...
  }

  loop
  {
     ...
  }

After I home in on the words then yes I will generally read left to right, top to bottom. But another part of imperative style is code is usually split into separate paragraphs of related functionality - so I can usually jump down whole paragraphs too.

The only other obvious variation I can think of is I will read bottom up when tracing the path a variable took to get to where I am currently looking.

loqi• 11 years ago

Same here. It's why I prefer to line-split conditionals in C-like languages like this:

    if (condition0
    &&  condition1
    &&  condition2)

rather than the usual "trailing operator" style. You can immediately parse the left column as a sort of vertical operator that tells you how the individual conditions are being combined.

But then, I also like Lisp. Prefix notation for "and" gives the same benefit directly, no tricky formatting required.

slowmovintarget• 11 years ago

The closer to English the code is, the less precise the syntax can be. In order to reduce the ambiguity, language designers must introduce additional rules, and additional conditions. This often includes context dependent semantics.

All of these things are _complications_ not just in the syntax but in the mental model. In order to effectively manipulate the code, you must hold all of those rules in your head while writing it.

In Lisp, the syntax requires the direct expression of context (parentheses). It also has dead simple semantics: do this to that. This means that Lisps are essentially optimized for authoring the code in the first place. This is what is meant when people say it is easy to reason about.

Of course, this is a trade-off, but arguably one that makes a developer more productive. Yes debugging is more difficult. Yet I'd venture it is a less common activity with Lisp as wrongness in the program is often obvious to the author.

But yes, not for everybody.

cicero• 11 years ago

Why do people who write or talk about Lisp always have to start with something like this?

Because to them it is awesome. People are different, and just because it isn't awesome to you doesn't mean it can't be for others. For me, I was impressed because Lisp overcame so many limitations I found in other languages.

When it comes to readability, I think that a lot of it is what you are used to. I do not use Lisp all of the time, so when I switch back to it, some time is required for me to get back into the groove where it becomes readable, and then it's second nature. (I teach multiple computer science courses using different languages, and switching gears from class to class can be a challenge.)

All of that being said, the greater use of punctuation and syntactic forms in other languages can aid in readability, but I don't think it is significant enough to completely offset the power you gain from the Lisp syntax.

agumonkey2• 11 years ago

Lisps are very abstract[1]. They have no value if you write a tiny mathematical expression or conditional. Who ever had trouble reading these ? This is not were you suffer when you write code IMHO. When you want to compose, rewrap any kind of objects at scale, then it's brilliant.

[1] I believe it's from birth, it was a thought experiment at first, IIRC it didn't even have numbers. Just symbols. It was about writing your own abstractions (see the symbolic differentiator in Lisp first paper). If you don't think about it that way then it surely will look absurdly convoluted.

sjolsen• 11 years ago

Why do people who write or talk about Lisp always have to start with something like this?

Well, the people who write or talk about Lisp generally probably think it's true, and it's the sort of thing that, if true, makes for a fantastic selling point.

you can add/remove 'language features' as you want, and the syntax is really simple, and code is data

I think this is the meat of it: that Lisp offers an exceptionally low barrier to entry to the world of, let's say, "experimental programming." It lets you do things you would never dream of if you've only ever mucked around in something like C or Java. Whether or not those things are actually useful in and of themselves, it's fun and rewarding to experiment not just with programs, but the way you write and even think about them.

Of course, there are people who will tell you that Lisp is also good for solid technical reasons, but I think this is the main appeal of Lisp to most people who describe it as "awesome."

Everyone tries to read programming languages as close as possible to plain english in their head

That's simply not true. You may do this, and it's a perfectly valid approach, but not everyone thinks that way. Personally, I think of code in terms of what it "does," rather than what it "says," if that makes any sense— it's difficult to explain.

As for readability, I firmly believe it ultimately comes down to familiarity. Yes, infix syntax is easier to read for a limited subset of small, operator-dense arithmetic expressions, but I find that kind of code is relatively rare. Broadly speaking, you're going to get most of your structural information about Lisp code from the indentation; the parentheses eventually fade into noise. And honestly, the same is true of just about every other programming language.

gglitch• 11 years ago

I can't speak for anyone else, and I'm a perpetual beginner with languages, but for me, lisp's syntax was sort of a god-send for clarity. Whitespace didn't matter. Indentation didn't matter. There were no symbols to memorize (except I guess precisely one, 'quote,' which could also be done without the symbol). Stringing code together seemed spectacularly simple: Everything's a list, the first atom is always a function unless explicitly declared otherwise with that one simple symbol, and you could always tell exactly how one function fit with the others you were looking at. I'm not a programmer, but with other languages, I feel like even with simple scripts, I'm always having to look up syntax details or pitfalls. Lisp is simple and uniform. It's good for beginners, I guess.

sabya• 11 years ago

You can imagine Lisp grammar like Pac Man eating the dots: ᗧ••••, and there is no ghosts here. Pac Man is the function in Lisp, and the dots are arguments. After Pac Man eat up all the dots (the function is executed with all these arguments), it becames a dot: • . and a dot is able to be eaten by another Pac Man.

Simplest explanation of Lisp grammar I've read!

agumonkey2• 11 years ago

Brett Victor had a similar idea involving something wilder http://worrydream.com/AlligatorEggs/

nickpsecurity3• 11 years ago

It's funny until you look at XML so many love and wonder why they hated LISP syntax at same time.

http://www.defmacro.org/ramblings/lisp.html

e12e2• 11 years ago

People loved XML syntax?

My impression was that the only thing wrong with XML was the syntax (hence the JSON revolution). After all, apart from syntax, possibly some of the ill-advised "external element" stuff ... I think XML is strictly better in every meaningful sense than JSON.

murftown• 11 years ago

Nice tutorial!

Note that your sbcl must be built with threading enabled for the example to work. Otherwise, the initial Lisp landing page will show up at URI /, but the handler for /hello?name=Blah will never be executed because sbcl-without-threading silently fails to use threading and instead runs the server in a blocking way in which later code never gets a chance to run.

This was confusing for me at first, but once I built sbcl from source using "./make.sh --fancy" everything worked perfectly.

Lisp for the Web, Part III

Related Stories