A simple Common Lisp web app

https://www.scotto.me/blog/a-simple-common-lisp-web-app/

by silcoon • 1 year ago

63 21 1 year ago

jimbokun• 1 year ago

Man that seems like an awful lot of ceremony for a trivial web app. Surely the language known for macros and building your own DSLs has a more concise way to get a web app up and running?

nomilk• 1 year ago

This is a pretty great tutorial using common lisp libraries:

https://www.youtube.com/watch?v=A4PzSsOD-CQ

(also, the top comment is gold)

silcoon4• 1 year ago

Josh Betts is a good bloke!

anonzzzies3• 1 year ago

Is it? This translates fairly exactly to things like nodejs/express implementing the same thing?

silcoon4• 1 year ago

Author here: You're right! I did that in the article. First I showed how to use the modern libraries, then wrote a quick custom library that compressed my original app in 29 LOC[0].

[0]: It came out shorter than the guestbook demo in Python-Flask.

vindarel4• 1 year ago

yeah the author does too much (and un-necessary stuff). To keep his stack just use the Caveman library has he does:

    (defparameter *web* (make-instance '<app>))

    @route GET "/"
    (defun index ()
      (render #P"index.tmpl"))

https://github.com/fukamachi/caveman/ (I like Mito and Djula, I'd go with Hunchentoot rather than Clack which is famously undocumented)

anonzzzies3• 1 year ago

I use CLOG these days; it's all so easy to get things done.

nesarkvechnep• 1 year ago

It’s good but has nothing on Phoenix LiveView. I love Common Lisp but for web, I can’t choose it over Elixir.

anonzzzies3• 1 year ago

I agree, but this was about Lisp :) I hope it will get there though; I like CL more... The author of CLOG seems to be interested on how to improve it so.

v9v• 1 year ago

re: common lisp shines at low-level tasks like microcontroller programming

While I couldn't find an explicit declaration of this in the linked article, the tunnel system likely has something on the level of an SBC (think raspberry pi) rather than a microcontroller. Common Lisp makes it possible to do fun low-level assembly stuff but AFAIK none of the implementations can run on a microcontroller.

There's uLisp, which builds on Arduino's libraries but it isn't a Common Lisp implementation.

In terms of bare-metal CL implementations there is Movitz (stopped development) and Mezzano, but I don't think those can run on any microcontrollers.

There are also some libraries to convert CL to C, thinlisp and CLiCC, but these aren't actively maintained currently.

The possibility of porting Clozure CL to the RP2040 was discussed in the ccl mailing list last year, but I don't think anyone's working on it.

praptak• 1 year ago

It probably doesn't make sense to cram a whole Lisp into a constrained environment like a microcontroller. What some people do instead is define a smaller Lisp which acts as a DSL and is compiled to run on the constrained platform.

https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp is an example of this approach.

"GOAL does not run in an interpreter, but instead is compiled directly into PlayStation 2 machine code to execute. It offers limited facilities for garbage collection, relying extensively on runtime support. It offers dynamic memory allocation primitives designed to make it well-suited to running in constant memory on a video game console. GOAL has extensive support for inlined assembly language code using a special rlet form, allowing programs to freely mix assembly and higher-level constructs within one function."

Jach4• 1 year ago

Yeah it's good to call that out. If the author used the term embedded systems there'd be less room for disagreement, as there is a pretty wide spectrum there -- obviously a RasPi that can run Linux is more than enough as far as hardware goes. I also don't think you'll find any CL implementation running out of the box on a $5 ESP32 like you can do with uLisp, but if you hooked up external memory perhaps you could get something going. A fullblown Common Lisp can (with work/money) be made to run in environments probably a lot smaller than most people think... The famous Remote Agent on the DS1 spacecraft was a 4 MB executable and peaked at "32 Mbytes", 20 avg, of memory. (Skimming the linked article, I do see it mentioned that using LispWorks let them run on ARM32 with "128 Mb of RAM". (Presumably that is megabytes.) IIRC that's beyond what you can add to an ESP32. I wonder if they could have gotten away with ECL though, as it has optional and runtime configurable memory limits with a default heap limit of 256 MB.)

Though as the sibling mentioned there are other approaches. Here's a fun article on using CL to write an NES 6502 assembler https://ahefner.livejournal.com/20528.html and demo application: https://www.youtube.com/watch?v=T8qEOG2KUQU

varjag• 1 year ago

While I couldn't find an explicit declaration of this in the linked article, the tunnel system likely has something on the level of an SBC (think raspberry pi) rather than a microcontroller.

I'm the author of that article. It runs on a SOM which is quite downspec from RPi (single core, arm32, 128Mb RAM) but yes it's a full featured system rather than a microcontroller.

vindarel4• 1 year ago

I also hear "programmable chips", does that count? Example:

- [Barefoot Networks](https://www.barefootnetworks.com/)

- "we use Lisp (via Allegro CL’s Enterprise class development environment) in the design of our Intel Tofino series of programmable switch chips"

  - article on [Franz Inc networks](https://allegrograph.com/press_room/barefoot-networks-uses-franzs-allegro-cl-to-drive-artificial-intelligence-in-networks/), some words [on reddit](https://www.reddit.com/r/lisp/comments/oq884e/how_do_you_use_lisp_at_work/h6d8xcw/), and a [youtube video](https://www.reddit.com/r/lisp/comments/oq884e/how_do_you_use_lisp_at_work/h6d8xcw/).

Jach4• 1 year ago

Lisp applications still are used behind the scenes for a lot of interesting stuff. In 1987, "An advanced VSLI design and simulation suite, with programmability and a GUI, used to design actual taped out processors."[0] Screenshots in "The Symbolics Ivory Design and Verification Strategy": https://archive.org/details/ivory-design-verification/mode/1...

[0] Thanks to stylewarning for the link last year: https://x.com/stylewarning/status/1834996048307388889

Jach4• 1 year ago

Thanks for the article! It's good to have more examples of this sort of stuff, especially if people decide to go the lack/clack route rather than using e.g. hunchentoot directly as I like to do. If you ever do a followup to go beyond the simple barebones, I'd like to see more on both development experience and production deployment. Some thoughts:

Better attention to error handling, as well as the difference in developer mode where you might want the error to show up in your REPL and/or shown on a page, but in production mode to just be logged and/or emailed somewhere while the server returns an appropriate error page without too much information on it.

In a more complex application, I would add a way to invalidate the cache automatically if a template changes.

By default, Djula actually recompiles the template if the file does change. What the cache here is doing is caching the association of a path string -> template file object, not the compilation status, as the template object can recompile itself.

You have a lot of top-level forms (path / asdf references, add-template-directory, etc.) that don't fly if you wanted to compile a binary to push up to a server.

Speaking of deploying to a server via binary, it's also easy to compile in an active or toggle-able swank server, so that if you do need to debug or want to develop/test things further "in production", the server doesn't need to know anything about Lisp or Lisp editors, you just use SSH port forwarding on the swank connection and talk to the production Lisp binary with your local editor. (The server does need to have a matching or newer libc than your build system though, and potentially other things like libmysqlclient. I just build stuff in an old debian docker image but Shinmera's Deploy is a great solution with a lot of headaches taken care of.)

I like the @export macro when developing because I can be a bit lazier, but it's better form to just add the stuff to your defpackage :export list eventually. (I finally got around to making my own editor macro in vim to do that for me, so I don't think I'll use the cl-annot package much anymore.) That library is also LLGPL which might not be desirable for the trivial handful of lines of code you can write yourself if you don't need its slightly fancier support for class accessors.

With sqlite, you should probably be using PRAGMA journal_mode=WAL when you create the schema, as well as consider changing its auto vacuuming behavior and a few other features. There are some per-connection pragmas worth looking at as well for a production scenario.

Showing how to send headers like content security policy would be good, and also going beyond simple error pages to including support for live metrics (e.g. https://github.com/deadtrickster/prometheus.cl -- there might be a clack middleware even?)

I wrap my Lisp web stuff in apache but I think nginx and caddy are popular too. Showing the full end-to-end with a let's encrypt SSL cert would be helpful for people getting into web apps who haven't already done it in some other ecosystem before.

silcoon4• 1 year ago

Thanks I've taken notes on the improvements. Also I've checked your website pages on Lisp and I'm discovering so much staff :)

vindarel4• 1 year ago

false premises, false conclusions, and bad good ideas in-between.

I was surprised by the absence of a quickstart page to help me set up a simple server

what about https://lispcookbook.github.io/cl-cookbook/web.html ?

Also I think that the READMEs of Mito (ORM) and Djula (templating) and easy-routes (routing) are very good. Heck, Djula's documentation is excellent. Double heck, even Caveman's README, which the author uses, has a good and sufficient getting-started page.

Oh, I understand, the author uses Clack/Lack… where Hunchentoot is much better documented.

there are of course more blog posts, some videos. I invite any blogger to consolidate community resources.

More recent: https://web-apps-in-lisp.github.io/

ps:

;; If the template is not already compiled, it is compiled, and stored in TEMPLATE-REGISTRY.

won't that prevent the template to compile changes? I suggest to follow Djula's doc.

I would add a way to invalidate the cache automatically if a template changes.

yeah. So don't do that ;)

(also +1 to Jach's remarks and explanations)

I created a macro that ensure the table gets created correctly

hummm… I'd suggest to leave the migration step explicit.

some database helpers to perform a few CRUD actions on the db.

I felt the need for CRUD helpers too when defining Mito classes. However those four ones are only similar functions with same parameters but another name :]

    (defun db-add (instance)
      "Inserts a model instance into the database."
      (insert-dao instance))

> I would say that there are better languages for developing modern web apps.

Burn this article with fire ;)

You don't need to need extraordinary clever needs to use CL. You benefit about it during development, deployment, and the application lifecycle.

time-saving error handling? You can use it 100x times a day. https://www.youtube.com/watch?v=jBBS4FeY7XM

simple "scripts", data-munging applications for B2B? You can do it easily in CL, and save yourself sanity during development, deployment, etc. https://lisp-journey.gitlab.io/blog/running-my-4th-lisp-scri...

One caveat: have some experience with the language.

since hot reloading has already been implemented in all high-level languages

nope you can't compare hot reloading with image-based development.

Please just don't compare anything or induce "better languages for modern web apps" with Python…

shout-out to Alive which is the only Common Lisp extension for VSCode that implements the REPL with features similar to what SLIME and SLY bring to Emacs.

please don't forget SLIMA for Atom/Pulsar which has even more features than Alive, or the Sublime one, or Vim, or the Intellij one, or… https://lispcookbook.github.io/cl-cookbook/editor-support.ht...

silcoon4• 1 year ago

false premises, false conclusions, and bad good ideas in-between.

False constructive criticism (since you've spammed your stuff 5 times), and bad (hostile) behavior which, I found, it's way more common in the Common Lisp community than in other languages.

I just gave my experience of testing the creation of a Common Lisp web app using different libraries, and instead of a guide of >20 pages, I condensed a bit of code in a single article. I could not find a quick tutorial for Caveman so I decided to write a short one. Is it such an issue?

Burn this article with fire ;)

You probably got triggered for my conclusions but I still stand for it. I really like Common Lisp and I'm a Lisp fan in general, but to write generic modern web app I think there are languages that fit better. Most of the web app they would not benefit from having such a powerful language as Common Lisp in the back-ent, I would even think it can be harmful in some cases when you need to hire, and I say this having worked with Lisp dialect in production.

Jach4• 1 year ago

The criticism and response both feel a bit too sensitive, but I do want to support having your own opinion/conclusion on CL probably not being the best language for the job when it comes to web stuff, or as I would phrase it CL not being the most defensible choice for web back-ends. I think it can be defensible, even just on the grounds of direct web application concerns, independent of what the actual purpose of the application is, and I'll continue trying to write everything in CL, but I sympathize with the opposition and recognize a strong argument has to be made for CL. Other options enjoy popularity and default-choice status.

My primary reason is like: there's a lot of complexity in the modern web, and it really is quite convenient if you can lean on some ecosystem that pushes things under the rug for you and is trustworthy enough to keep doing so for years to come as things change. All the basic table stakes stuff like CSRF protection, sending custom CORS/CSP headers, database interfaces, logging, metrics, bcrypt hashing, selenium webdriver automation, etc. etc. all have their core pieces available in CL, but you still have to know about them and opt-in to them and occasionally roll up your sleeves to do things your own way from scratch. They also don't all have great documentation, maintenance, or integration with each other. (There's even some dumb github drama I happened to notice being mentioned on discord the other day around a breaking change related to websocket code. Even if it doesn't affect me it's discouraging.) I don't mind rolling up my sleeves (and it can be a way to keep up to date), my own blog I haven't gotten around to rewriting in CL is still just raw PHP I wrote the majority of in 2009, with its own custom micro-framework and template system. Back then I was pretty anti-framework (I still am to an extent) but frameworks are pretty convenient, whether they're micro ones like Flask, slightly larger (if now outdated) ones like Yii, or the big boys. (I liked how for a CRUD project I did for a client in 2012, Yii had the ability through a developer mode web page to generate a bunch of code for you, rather than a separate command line tool or just shrugging and telling you to write your own macros: https://www.yiiframework.com/extension/yiisoft/yii2-gii/doc/...) All that to say that the framework options for Lisp aren't really there if you have expectations from other ecosystems like PHP, Python, Ruby, or Java, and when it comes to ever changing web stuff, ecosystem has more impact than the core language.

A secondary reason is even more my own perspective and perhaps isn't entirely fair, but it seems a lot of people doing web stuff in Lisp are doing it mostly because we love Lisp and find its workflow and feature set the best, the language is a great fit for any actual purpose/problem the application aims to solve regardless of the web bits. (I don't think it matters much if your "needs" are just modest or generic CRUD level, like a certain defunct license contract subscription and renewal tracker, or something more complex or long-lived like google flights or grammarly.) To put it another way, you don't tend to see a lot of super passionate people for the web platform itself. Like check out this Rails keynote from last year https://www.youtube.com/watch?v=-cEn_83zRFw dhh is if anything passionate and has been doing this for 20 years. Even if one might disagree with his solutions (I don't particularly like Ruby itself) there's clearly a lot of value there and real problems being solved for people at a level prior to their actual application's main concern.

vindarel4• 1 year ago

oh, hello. Yes, you write "I was surprised by the absence of a quickstart page to help me set up a simple server", that's spreading FUD. Many such tutorials have been written.

I could not find a quick tutorial for Caveman

But what about Caveman's own README? It's a tutorial.

You would phrase differently I could not discuss it (like your first sentences on CL's documentation state in general). The "absence", I can. Words are important.

triggered

yeah obviously, and by the fact that I put efforts to improve CL's (web) documentation situation.

hostile

abrupt and impatient, yet relevant? I make some points.

A simple Common Lisp web app

Related Stories