Ooh Ooh My Turn Why Lisp? (2008)

http://smuglispweeny.blogspot.com/2008/02/ooh-ooh-my-turn-why-lisp.html

by bootload • 10 years ago

99 157 10 years ago

smuglispweeny.blogspot.com

asimuvPR• 10 years ago

The post linked has another interesting post in the comments: http://www.defmacro.de/?p=13 . Its about an interview McCarthy gave in the year 2000 to infoQ.

catnaroek10• 10 years ago

object identity for everything but numbers and characters

Since when is this supposed to be a selling point? Not having compound values (not the same thing as compound objects!) is a pain.

oconnore3• 10 years ago

You seem to be very confused. This is referring to #'eq (which is why #'eql exists). What lack of compound values are you lamenting? Lisp has built in classes, structs, arrays, lists, and maps. What is missing?

An implementation is permitted to make ``copies'' of characters and numbers at any time. The effect is that Common Lisp makes no guarantee that eq is true even when both its arguments are ``the same thing'' if that thing is a character or number.

http://www.lispworks.com/documentation/HyperSpec/Body/f_eq.h...

catnaroek10• 10 years ago

Lisp has built in classes

Objects aren't compound values. Objects are compound, well, objects. And pointers to objects are primitive, indivisible values - not compound!

lists

Racket and ML have lists. Lisp has mutable linked data structures, built out of `cons` cells, whose value at any given point in time may be a list. But the value itself isn't first-class, because you can only bind the identity of the cons cell to a variable.

einhverfr4• 10 years ago

It's a nice perspective. Very practical and down to earth. There is another side though, and that is that learning to think of code as data is extremely empowering and translates well when working with a fairly large set of other languages (Perl, Python, Ruby, and now even Java). Learning Lisp turns you into a better programmer.

agentgt2• 10 years ago

I like Scheme/Lisp but I don't know if I buy the whole "learning Lisp turns you into a better programmer". Maybe more knowledgeable but not necessarily a better a developer.

In some ways learning a really flexible language like Lisp can turn you even into a really bad developer. I say "developer" instead of "programer" because I want to emphasize working with others and thus sharing code with others. Of course this is based on some past observations working with MIT grads and various other academia so take my opinion with a grain of salt.

IMO the language that really changed everything for me was the ML family of languages and maybe C. I would say knowing C and ML is more worthwhile than Lisp (Lisp is not exactly hard to learn anyway... the basics that is).

ohyes4• 10 years ago

I'm biased, but there's a big difference between "learning lisp" (well enough to transliterate that python program) and really getting the code as data concept.

I'd say that learning X really well is always a positive because whenever you learn something in depth you can then apply the concepts elsewhere. And there are still lisp-only concepts, so learning lisp is positive. ML is good too. Also spending time with a well designed concurrent language is beneficial.

einhverfr4• 10 years ago

It is also worth noting that failure is often a step on the path to success. Learning to apply the "code is data" principle in other programming languages is not without pitfalls as you say and falling on one's face sometimes is to be expected.

However, that's part of the process too as is learning when to use a hammer vs a wrench vs a screwdriver. There is the point in learning a new tool where everything looks like a nail. Yes, people will really mess things up by applying the wrong concepts but how else do we learn?

lispm6• 10 years ago

There are some Lisp-related things which provide developers with new tools and new ways to think, maybe making them better:

a) code as data

b) interactive software development

c) user-defined language extensions where the developer is part-time language designer, syntactic abstraction

d) dynamic object-oriented programming

e) meta-object programming

hellofunk16• 10 years ago

I think that more specifically, "Learning FP makes you a better programmer", and you often learn FP when learning lisp, so the original statement is often true as well.

cheez6• 10 years ago

Did Kenny ever release his educational software?

tsuru• 10 years ago

It appears so, yes: http://tiltonsalgebra.com/#

cheez6• 10 years ago

This is amazing. Kenny, please fix the fonts.

mark_l_watson3• 10 years ago

Classic Kenny!

Common Lisp used to be much of my programming world also.

platz2• 10 years ago

most Common Lisp implementations are native compiled. ... isn't it nice to compile down to the metal?

Is just the interpreter native or does cl compile your app to native as well?

pmalynin• 10 years ago

SBCL compiles to native. In fact thats how you check if the function was tail-optimized -- it will have a JMP opcode instead of CALL

ohyes4• 10 years ago

To add to this, Most of them compile to native. I think the exception is CLISP. Certain implementations have both because compiling was costly at one point.

snaky3• 10 years ago

To native with elegance [1] and even closer to the metal "because sometimes C abstracts away too much" [2].

[1] https://www.pvk.ca/Blog/2014/08/16/how-to-define-new-intrins...

[2] http://www.pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly...

oconnor663• 10 years ago

he simply asked if Python could gracefully manipulate Python code as data

Have there been any big LISP macro code injection vulnerabilities in the wild?

kiiski• 10 years ago

What do you mean with "macro code injection vulnerabilities"? Macros are expanded before the code is compiled; by the time someone is running the program there are no macros. Well, technically, someone could call `EVAL` in a production app, but that's strongly discouraged.

Johnny_Brahms2• 10 years ago

As the other comment says, that doesn't really make sense. However, I have had to debug a codebase where lisps unhygienic macros led to extremely weird errors. The second time that happened in a big project I decided that schemes hygienic macros are superior. The extra cruft writing them is worth it. I have literally spent weeks debugging weird macro errors introduced years after the macro was first written.

wtbob2• 10 years ago

You're not wrong: simple macros can be written poorly. Macros which don't properly use GENSYM should be rejected in review. Also, it's entirely possible to write a hygienic macro implementation with DEFMACRO: nothing stops a team from doing that (or using one off-the-shelf).

But sometimes you do need the full power of DEFMACRO, and it's not possible to write DEFMACRO with DEFINE-SYNTAX. So one tool enables you to do anything you need, and the other only some of what you need. I know what I prefer.

krupan• 10 years ago

"The question is where will we be in five years, the answer is using Common Lisp, because of language features" says the author in the comments. This was 8 years ago.

Kind of rude of me to take that potshot, but it's there. I've been reading these "Why Lisp?" Arguments for years and yet it still remains pretty niche. The longer this goes on the more I am unable tell Lisp advocates apart from Perl advocates. They both sound like write-only languages. They make a lone programmer feel very productive and Free, but it results in essentially unmaintainable code. I've written and attempted to maintain much more Perl than Lisp, many years ago, so please tell me if I'm wrong.

meric2• 10 years ago

I'm developing a language that's a superset of Lua and Lisp. It's still in its infancy, but here's an example of a Lisp/Lua function in a module that can be imported as a (+...) macro that expands into Lua `a + b + c + ...` expression.

https://github.com/meric/l2l/blob/rewrite/l2l/macro/arithmet...

The idea is programmers will write in Lua whenever readability is prioritised, and when homoiconicity is required, for example in the use of domain specific languages, lisp can be switched to. The lisp part can be used to produce macros that expand into Lua.

Would be great to know your thoughts!

zvrba2• 10 years ago

You know about metalua, right? http://metalua.luaforge.net/

Frondo3• 10 years ago

Heh, yeah. Lisp has powerful features, I'm sure, but something that's always stumped me is, if that power mattered for delivering better code, would we have to strain so hard to find examples of why it's better?

Paul Graham's viaweb story, well, that's great, but why wasn't there a whole cohort of super-powered startups winning the day with lisp in the original dot com boom? Why are there so few big wins with lisp? The "lisp is powerful" story just doesn't seem like the whole story, when there are so few successes on the ground compared to C, C++, Python, Perl, PHP, Ruby, VB, you name it.

jfoutz• 10 years ago

Eh, i think pg's strategy is sound for a startup. If you know lisp inside and out, absolutely use it to build your MVP. but once you get traction, you need a plan to transition to something you can actually hire for, just like reddit. wasn't orbitz lisp? there are a few that kicked around.

On the other hand, if you don't know lisp, you should use what you know instead. it's just not worth learning a language and solving your startup's problems at the same time.

gkya4• 10 years ago

You confuse powerful with popular. Many startups don't need powerful tools, other than a CRUD framework and JQuery. Also, existence and abundance of libraries is a factor, and Lisp didn't get much love from the OSS web folks til recently.

lispm6• 10 years ago

DWAVE uses Lisp / SBCL.

http://www.dwavesys.com

Some company doing mobile satellite-based Internet streaming used Lisp / Clozure CL.

martinflack2• 10 years ago

Hmm. In many walks of life, popular != best. In fashion, cars, investing, and to some extent cooking, for example, the top performing adherent is doing things differently than the crowd.

Lisp code can be very readable, but it takes a few years to learn how to write it that way. I spent many years in Perl 5 myself, and I found Common Lisp tricky at first but ultimately very satisfying.

Two jedi mind tricks for the beginner:

1. Don't think of Common Lisp as the language you want. The language you want is one layer above CL, and fits the business domain you operate in. Build that. (Yes, this requires a little investment up front.) Then solve your problems in that "language". Because Lisp macros are powerful fully-transformative engines, you can do almost anything (less breaking a few simple syntax rules... but newbs often underestimate what can be broken/bent).

2. As silly as it sounds, the parentheses coming first does bother folks. Every editor does syntax highlighting now. Do two things: (a) color the parens close enough to your background color that they visually fade and "back off", but are still visible when you need to look at them - your code instantly looks a bit more like python, indentation sticks out; (b) used a structured editing mode that inserts closing parens, keeps parens balanced automatically, and offers neat structured editing hotkeys to copy/cut/paste/delete sexp's (expressions) as units. Both steps are extremely comfort-building and the ubiquitous parens remove a class of bugs.

The greatest compliment to Common Lisp that I've seen over the past 8 years is the constant pilfering of language features to other mainstream languages.

As for readability - like Perl, TIMTOWTDI, and even moreso in Lisp because one can write macros as utilities to greatly simplify the actionable code and have it blow out into lower-level code that does the mechanical dirty work.

As a result of what I've said above, people working in CL do tend to write it slightly differently. You can see some evidence of that if you study different open source libraries, etc. It's not so much a rigid code style as an interest and ethos that binds the community.

I'll leave with a little example of macros at work, via the old and venerable SERIES library, that allows one to specify loops in a semi-declarative style, that if executed literally would cause multiple passes; but which compiles to a single pass over the data sequence.

The input is stock market "bars":

  (defvar *spy2006*
    '(("29-Dec-06" 142.08 142.54 141.43 141.62 45461200 141.62)
      ("28-Dec-06" 142.41 142.70 141.99 142.21 37288800 142.21)
      ("27-Dec-06" 141.87 142.60 141.83 142.51 39727100 142.51)
      ;; ...
     ))

This is what you might code, which I humbly submit to you is very readable:

  (defpackage iteration-testing-series (:use :cl :series))
  (in-package :iteration-testing-series)

  (defun combine-bars (bars)
    "Summarize a sequence of BARS into one bar."
    (series::let ((zbars (scan 'list bars)))
      (list (first (first bars))
            (second (first bars))
            (collect-max (map-fn 'float #'third zbars))
            (collect-min (map-fn 'float #'fourth zbars))
            (fifth (car (last bars)))
            (collect-sum (map-fn 'integer #'sixth zbars)))))

That function is very easy for me to read. Partially expanded, it becomes the following, which is itself further expanded. Note that it was trivial for me to request this expansion, and that multiple macros are composable so that many features can be engage at once. I can write something high-level, but also observe the expansion down to low-level code all the way (SBCL can even trivially show me the resulting assembler code for my CPU). I am freed up to think at a high level - but I don't lose control over the low level.

The following block may be daunting in size but if you know that SETQ is an assignment, and within the TAGBODY there are labels for GO which is like a goto, it should be possible for a programmer of other languages to follow. Not also the type declarations carried over for me automatically, which will allow for speed and debugging optimization, which in CL is a knob that I can turn - something I don't know exists in other languages.

  (LET* ((#:OUT-1014 BARS))
    (LET (ZBARS
          (#:LISTPTR-1012 #:OUT-1014)
          (#:ITEMS-1019 0.0)
          (#:NUMBER-1017 NIL)
          (#:ITEMS-1026 0.0)
          (#:NUMBER-1024 NIL)
          (#:ITEMS-1032 0)
          (#:SUM-1030 0))
      (DECLARE (TYPE LIST #:LISTPTR-1012)
               (TYPE FLOAT #:ITEMS-1019)
               (TYPE FLOAT #:ITEMS-1026)
               (TYPE INTEGER #:ITEMS-1032)
               (TYPE NUMBER #:SUM-1030))
      (TAGBODY
       #:LL-1035
        (IF (ENDP #:LISTPTR-1012)
            (GO SERIES::END))
        (SETQ ZBARS (CAR #:LISTPTR-1012))
        (SETQ #:LISTPTR-1012 (CDR #:LISTPTR-1012))
        (SETQ #:ITEMS-1019 (THIRD ZBARS))
        (IF (OR (NULL #:NUMBER-1017) (< #:NUMBER-1017 #:ITEMS-1019))
            (SETQ #:NUMBER-1017 #:ITEMS-1019))
        (SETQ #:ITEMS-1026 (FOURTH ZBARS))
        (IF (OR (NULL #:NUMBER-1024) (> #:NUMBER-1024 #:ITEMS-1026))
            (SETQ #:NUMBER-1024 #:ITEMS-1026))
        (SETQ #:ITEMS-1032 (SIXTH ZBARS))
        (SETQ #:SUM-1030 (+ #:SUM-1030 #:ITEMS-1032))
        (GO #:LL-1035)
       SERIES::END)
      (IF (NULL #:NUMBER-1017)
          (SETQ #:NUMBER-1017 NIL))
      (IF (NULL #:NUMBER-1024)
          (SETQ #:NUMBER-1024 NIL))
      (LIST (FIRST (FIRST BARS)) (SECOND (FIRST BARS)) #:NUMBER-1017
            #:NUMBER-1024 (FIFTH (CAR (LAST BARS))) #:SUM-1030)))

flukus• 10 years ago

This is what you might code, which I humbly submit to you is very readable:

It's not, if that's a real life example. There is no context for what the data is, like a variable names.

happyslobro4• 10 years ago

Regarding the hate for parens: rainbow coloring helps out big time. When each pair of parens has it's own color, it is very easy to see exactly how they are grouped in the more complex statements. For Vim, that would be Rainbowtags. It's actually an improvement for any languages, imo.

mbrock2• 10 years ago

I like niche stuff.

What language doesn't result in unmaintainable code, really?

And let me just say... it's a bit rude to spread this rumor that Lisp is anti-social and bad for business when you've barely even tried it. Seriously.

"Prove me wrong" is not a good way to discuss. See for yourself instead: learn Lisp and try to use it to write maintainable code.

tempodox2• 10 years ago

I think you touch an interesting area. For me, figuring out how to make code maintainable (including error handling) is still an area of research, and I'm not a newbie. At least Lisp, with its metaprogramming facilities, gives you some good tools for experimenting. And I do think that the experimental character of the original problem space when Lisp was discovered plays an important role there.

Grue3• 10 years ago

Well, I'm still using Common Lisp. If there was anything better, I'd have probably switched.

effie3• 10 years ago

Curious, what are you using it for? Would you say it is well suited for writing operating system helper tools (working with files, pipes, kernel api, calling other programs)? I like bash expressiveness a lot but sometimes I feel I could use a more powerful scripting language.

posterboy• 10 years ago

Clojure got reasonably popular around that time.

unpop-opinion5• 10 years ago

Okay, I've spent a lot of time writing Common Lisp and even more time reading about Common Lisp, and to be honest, I'm just a little tired of the cult around it. People who know it well gloat about how great Common Lisp is, which just so happens to make them look great too. And people who don't know Common Lisp all talk about the little Common Lisp they know, because they don't want to seem like they aren't in on it, and because they don't know better.

Let's talk about this fabled exchange between Norvig and McCarthy, where McCarthy asks if Python can gracefully manipulate code as data, and Norvig said no, and supposedly a thousand words were said in the ensuing silence.

Here's the thing; a thousand words weren't said. I don’t know what Tilton thinks was said there, but it certainly wasn’t a one sided silence in which only McCarthy’s side gets to smugly smirk as if it proves anything.

Let’s talk about code as data, shall we? What enables that? Of course, it’s the s-expressions and prefix notation. So let’s write a macro. But we won’t want to write a macro that simply sits at the beginning of an s-expression and takes in a bunch of arguments, because then we could just write a function. No, we want to do a transformation on the code that creates a domain-specific language, because that’s the power of macros, right?

So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions. Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following. Every future macro must account for every previous macro. The more you use the capability to manipulate code as data gracefully, the less graceful it becomes.

And, in my opinion, far more damningly, macros make your program harder to reason about. Before macros, a reader could assume that the first thing in the s-expression was operating on everything else in the s-expression. But no longer. We can’t even assume that sub-s-expressions in the s-expression are evaluated first.

This isn’t a hypothetical problem. Common Lisp programmers spend a ton of time talking about how to write macros so that they’re not going to come back and bite you in the butt when they get used in an unexpected situation. And the reason is, nobody really knows how to do it.

So, with a great deal of respect for McCarthy based on his many other achievements, I have to say I don’t care about macros, and actually think we’re better off without them. First-class functions are a much more coherent, consumable way of using code-as-data.

Most of the features of Common Lisp I actually want are in other languages now. Garbage collection? REPLs? First-class functions? Higher-order functions? They’re all in other languages now. Python, for example has all those things.

And to be honest, other languages have done a lot better things with the functional programming aspects of Common Lisp. McCarthy gets credit for inventing a lot of stuff, but we don’t fly Wright-brothers style planes today and we shouldn’t use Common Lisp just because it was first to have those things. More sophisticated type systems make lambdas more powerful (and incidentally, a lot of the problems with macros can be seen as type problems).

People on this thread are claiming, “Learning Common Lisp turns you into a better programmer”. But I tend to think that learning functional programming is the part that people are referring to, and frankly, there are better languages in which to learn functional programming. Haskell, Standard ML, or OCaml would be a better choice.

And sure, there are other features that only Common Lisp has, but nobody is talking about those. Restarts? I’d love to see more people experimenting with those. Then again, Erlang has a way better threading model than anything else and much more sophisticated pattern matching. Scheme has call/cc. Standard ML has a powerful type system. Haskell has functional purity. Prolog has unification. A great many of these are more interesting than restarts.

I don’t hate Common Lisp; if nothing else, Common Lisp has a few very solid implementations out there and lots of existing libraries that make it a very useful tool. There are some programs which I wouldn’t consider writing in another language. I just don’t really think it’s the be-all and end-all of programming languages any more, and I’m kind of tired of the cult that has formed around it.

EDIT: s/Lisp/Common Lisp/g because not everyone takes “Lisp” to just mean “Common Lisp”.

catnaroek10• 10 years ago

So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions. Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following. Every future macro must account for every previous macro. The more you use the capability to manipulate code as data gracefully, the less graceful it becomes.

You should check out Racket's macro system. It's a lot more sophisticated than Common Lisp's. Common Lisp macros are to C (e.g., gensym is a macro-level malloc, you need to manually destructure S-expressions, etc.) as Racket macros are to ML and Haskell (syntax objects are aware of which variables are in scope, so automatic fresh name generation is possible; user-defined syntax classes and patterns let you process arbitrarily complicated structures in a sane way, etc.). If you like the idea of metaprogramming, but `defmacro` left you with a bad taste in the mouth, Racket is totally the language for you.

First-class functions are a much more coherent, consumable way of using code-as-data.

First-class functions are easier to use than macros, but they are not “code as data”. Furthermore, “code as data” itself is only true with a caveat: the full version is ”code in an object language is data in the metalanguage”, which is obvious to anyone who has written a compiler. Of course, macros make it easy to use Lisp as its own metalanguage, but there's still a phase distinction between macro-expansion time and when the generated code is actually used.

And to be honest, other languages have done a lot better things with the functional programming aspects of Common Lisp.

Common Lisp is a ridiculously powerful language, but it isn't a functional language. It fails to meet the zeroth nonnegotiable requirement in a practical functional language, namely, a notion of compound value: https://news.ycombinator.com/item?id=12199981

unpop-opinion5• 10 years ago

You should check out Racket's macro system. It's a lot more sophisticated than Common Lisp's. Common Lisp macros are to C (e.g., gensym is a macro-level malloc, you need to manually destructure S-expressions, etc.) as Racket macros are to ML and Haskell (syntax objects are aware of which variables are in scope, so automatic fresh name generation is possible; user-defined syntax classes and patterns let you process arbitrarily complicated structures in a sane way, etc.).

Agreed. I did play around with this part of Racket quite a bit and I'm convinced that it's the best system if I wanted to create a domain-specific language. But it still runs into the problem where you're defining a new language with new syntax, which forces you to define even more new language with more syntax in order to make that language useful. It's the best way to build a DSL

But given we already have a pretty good multi-purpose language with a lot of work put into it (Racket) the number of situations where it's worthwhile to create an equally-well-thought-out DSL is pretty low. Racket makes it easier, but it's still not easy. Add to this the fact that other people are going to write half-assed DSLs in my code, the net tradeoff is still usually negative, even with Rackets clearly superior macro system.

Furthermore, “code as data” itself is only true with a caveat: the full version is ”code in an object language is data in the metalanguage”, which is obvious to anyone who has written a compiler.

Uh, I've written a compiler and that's not obvious.

If you want to disagree with me on what "code as data" means, you're welcome to do so. As long as you understood what I said I don't care which words got me there.

Common Lisp is a ridiculously powerful language, but it isn't a functional language. It fails to meet the zeroth nonnegotiable requirement in a practical functional language, namely, a notion of compound value: https://news.ycombinator.com/item?id=12199981

How about we assume when I said "functional programming" I'm using the Wikipedia definition[1].

[1] https://en.wikipedia.org/wiki/Functional_programming

qwertyuiop9243• 10 years ago

You should check out Racket's macro system. It's a lot more sophisticated than Common Lisp's. Common Lisp macros are to C (e.g., gensym is a macro-level malloc, you need to manually destructure S-expressions, etc.) as Racket macros are to ML and Haskell (syntax objects are aware of which variables are in scope, so automatic fresh name generation is possible; user-defined syntax classes and patterns let you process arbitrarily complicated structures in a sane way, etc.). If you like the idea of metaprogramming, but `defmacro` left you with a bad taste in the mouth, Racket is totally the language for you.

Better yet, check out some other schemes. ir, er, and sc macros has the raw procedurual power of defmacro, but with the hygene and safety of syntax-case/syntax-rules, without the declarative syntax of syntax-rules, and the disadvantages of syntax-case (stupidly complex, breaking the standard macro abstraction with syntax/datum distinctions, etc.).

Given, syntax-case has some advantages, but I don't think it carries its own weight from a programmer's perspective.

zvrba2• 10 years ago

Let's talk about this fabled exchange between Norvig and McCarthy, where McCarthy asks if Python can gracefully manipulate code as data, and Norvig said no, and supposedly a thousand words were said in the ensuing silence.

Python has this: https://docs.python.org/2/library/ast.html so it is possible to manipulate code as data. It just happens that it's not homoiconic, but so what?

unpop-opinion5• 10 years ago

Well, the key part is gracefully. Lots of languages can manipulate code as data, but so for for me only in homoiconic languages does it end up feeling graceful for a while (although as I noted, it stops feeling graceful pretty quickly).

TeMPOraL2• 10 years ago

Python has this: https://docs.python.org/2/library/ast.html so it is possible to manipulate code as data. It just happens that it's not homoiconic, but so what?

The difference is that when you can do it gracefully, you'll end up doing it and using this possibility to build your project. If you can't do it gracefully, you'll dread using it and will not consider it a possible solution to your problem unless all other options are exhausted.

lispm6• 10 years ago

Let’s talk about code as data, shall we? What enables that? Of course, it’s the s-expressions and prefix notation.

s-expressions and prefix notation are orthogonal concepts. Nothing in the definition of s-expressions says it uses prefix notation. S-expressions are just a data format and its external notation.

But we won’t want to write a macro that simply sits at the beginning of an s-expression and takes in a bunch of arguments, because then we could just write a function.

Functions and prefix notation are also unrelated concepts. Lisp syntax uses prefix notation, but not everything is a function call. There are also Lisp variants, which don't use prefix notations, but still have s-expressions and even macros,

Take a lambda expression: (lambda (a b) (* a b (+ a b)))

It has a lambda symbol in prefix position, but it is not a function call.

So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions.

Which is nonsense. Macros don't break s-expressions.

Every future macro must account for every previous macro

Which is also nonsense. The typical macro expansion mechanism takes care of that most of the time.

Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following.

Well, now we not only have special forms (!), functions, lambda expressions, etc., but also macros. The main difficulty now is: more syntax and even user-defined syntax.

First-class functions are a much more coherent, consumable way of using code-as-data.

Which are unrelated concepts.

Most of the features of Common Lisp I actually want are in other languages now. Garbage collection? REPLs? First-class functions? Higher-order functions? They’re all in other languages now. Python, for example has all those things.

There are cars which have wings, can swim, etc. But that does not make them especially useful, say, to bring the kids to school every morning. The raw assembly of features is not the point, it's their integration for certain use cases.

Wright-brothers style planes today and we shouldn’t use Common Lisp just because it was first to have those things.

Common Lisp wasn't first. CL was defined 26 years (in 1984) after Lisp (1958) was invented and standardized after ten years more work (1994).

But I tend to think that learning functional programming is the part that people are referring to,

Not really. One can learn functional programming with much simpler languages. Legions of students used simple Lisp dialects/subsets to learn some FP concepts. See SICP (and many other books/courses) - which doesn't use macros, btw.

and frankly, there are better languages in which to learn functional programming. Haskell, Standard ML, or OCaml would be a better choice.

Since Common Lisp was never designed to enforce or advance statically-typed Functional Programming, it's only logical that it is not particular good at it.

wtbob2• 10 years ago

So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions. Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following.

That's just completely wrong, and makes me wonder if you have actually spent much time writing Common Lisp at all. Macros are orthogonal to S-expressions: they manipulate code (which is just lists) in-memory, after it's been read in from S-expressions by the reader. They don't break S-expressions at all (and indeed, the output of a macro is necessarily PRINTable as an S-expression, e.g. (macroexpand '(with-open-file (foo "/tmp/"))) expands to:

    (LET ((FOO (OPEN "/tmp/foo")) (#:G637 T))
         (UNWIND-PROTECT (MULTIPLE-VALUE-PROG1 (PROGN) (SETQ #:G637 NIL))
                         (WHEN FOO (CLOSE FOO :ABORT #:G637))))

When Lisp programmers talk of implementing a DSL with its own syntax, we almost always mean an S-expression-based DSL, where the fundamental syntax still uses S-expressions and the DSL is built atop them. An example would be CL-WHO[1], where HTML tags are represented e.g. as (:p "Some text " (:em "emphasised")).

Maybe you're thinking of read macros, which really can be used to implement other syntaxes, e.g. CLSQL[2], which uses custom reader syntax to implement inline SQL expressions? In that case, you're not wrong: implementing your own reader syntax does, indeed, break S-expressions. But it doesn't matter: due to the specification of the reader algorithm, neither elements containing nor elements within custom syntax care. The reader algorithm is quite elegant that way.

I have to say I don’t care about macros, and actually think we’re better off without them. First-class functions are a much more coherent, consumable way of using code-as-data.

At the cost of being much more verbose. I've taken a look at trying to implement restarts in Go, and the sheer number of times I'd have to type 'func' is insane.

But I tend to think that learning functional programming is the part that people are referring to

Once again, I disagree. IMHO writing Lisp makes one a better programmer because one starts to think of code as clay, which can be sculpted and crafted, moved around, deleted, generated &c. The worst good programmers I know of simply aren't comfortable with that concept; the best programmers I know are.

Compared to that, functional programming is merely interesting IMHO.

[1] http://weitz.de/cl-who/ [2] http://clsql.kpe.io/

cronjobber• 10 years ago

a lot of the problems with macros can be seen as type problems

If it had a type, it would be something like code->code. You seem to hold one of those "not even wrong" kind of notions about types and/or macros.

qwertyuiop9243• 10 years ago

I'm just a little tired of the cult around it

Me too, but you seem to not be entirely clued in to how macros work. So there's that.

So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions. Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following. Every future macro must account for every previous macro. The more you use the capability to manipulate code as data gracefully, the less graceful it becomes.

While perhaps more true in Common Lisp than in scheme, that's still fairly untrue. IIRC, macro expansion is innermost-first, so macros don't have to worry about tripping over each other, but even if that's not true, you're still wrong. Macros are sexprs, just like anything else: they break the semantic rules of lisp, NOT the syntactic rules of lisp. Since macros operate primarily on a syntactic level (their job is to sugar over Lisp), and when they operate on a semantic level, they're being used to write DSLs, which aren't lisp, the problem you describe is rare to nonexistant.

macros make your program harder to reason about.

Yes, but not for the reasons you think. Macros make your program harder to reason about for the same reason functions do: they're an abstraction. The higher level the abstraction you're using is, the harder it is to reason about your code, because by definition, you can't see what it's doing.

Common Lisp programmers spend a ton of time talking about how to write macros so that they’re not going to come back and bite you in the butt when they get used in an unexpected situation. And the reason is, nobody really knows how to do it.

However, us schemers (NOT racketeers, Racket using syntax-case instead, which while similar seeming, is an entirely different ball game, although it also fixes this) have had hygenic defmacro-style lowlevel macros for years now, and that fixes the worst of it.

Python, for example has all those things.

...Not actually true. Python's functions aren't first-class. They try very hard to be, but they're not. Which is why idiomatic Python doesn't use a lot of higher-order functions. Ruby does better, but of the trinity of popular high-level scripting languages, only JS has true first-class functions. In fact, for a while, JS had no distinct syntax for function declaration.

People on this thread are claiming, “Learning Common Lisp turns you into a better programmer”. But I tend to think that learning functional programming is the part that people are referring to

It's not what I'd refer to. FP makes you a better programmer to, but learn than in Haskell or ML, or even Scheme, though it's not as good for that purpose. Learning lisp exposes you to code-as-data in a very viceral way, and can help you understand a variety of programming concepts, but more importantly, it exposes you to new, different ways of programming, which always makes you a better programmer.

And sure, there are other features that only Common Lisp has, but nobody is talking about those. Restarts? I’d love to see more people experimenting with those. Then again, Erlang has a way better threading model than anything else and much more sophisticated pattern matching. Scheme has call/cc. Standard ML has a powerful type system. Haskell has functional purity. Prolog has unification. A great many of these are more interesting than restarts.

I agree, but restarts are the one I want most. It might seem like we schemers got a better deal with call/cc, and I love call/cc, but sometimes I look at restarts, and wish we had gone the other way (call/cc and functioning restarts are effectively mututally exclusive). Trust me, call/cc looks cool, but so does self-rewriting asm. Both come in handy from time to time, but you rarely want either in production (and especially not if your scheme doesn't have cheney on the mta compilation, because then call/cc is criminally slow)

I just don’t really think it’s the be-all and end-all of programming languages any more, and I’m kind of tired of the cult that has formed around it.

That's true. No language is, or ever can be the be-all, end-all. Rust and FP taught me that: some paradigms and ideas require resrictions, others require freedom. There's always something new to learn. And the cult of common lisp is not one you need to join. In fact, if you're a good lisper, you can't really be part of it.

hahooooo• 10 years ago

Is there any advantage in using a functional language when programming non-mathematically related code?

I see how it can help write Neural Networks, AI, financial code, etc. But if you know Python,Ruby,PHP,Java and Lisp (well), why would you choose to write a blog, webstore, webmail client or social media* app (which probably is about 90% of what people actually do) in Lisp over the four three?

I'll tell you why I would chose the first four:

1. Large community. 2. Many more libraries. 3. Easier to hire.

*Except for (perhaps) a small spam-block/feed ai module

nv-vn• 10 years ago

Yes. Functional programming (although it isn't necessarily a trait of Lisp) has, in fact, little relation to math (despite what many people believe). It's also very far removed from the field of AI right now (not for any particular reason, but there's no more specific reason to switch to Lisp for AI than there is for desktop or web apps). The reason you'd use Lisp is quite simple -- it saves time and makes programming easier. Macros are the biggest time saver in the world, and you don't realize how much time you waste writing repetitive code until you get to use them.

As for your 3 reasons in favor of the other languages, I think the first means almost nothing at all. Lisp is much easier (once you grasp it) than many other languages, so I think the fact that you'll get less answers on StackOverflow is irrelevant. As for the second reason, as long as you have one library that works for what you want -- say some library for writing web servers -- it doesn't really matter after that. There's rarely any reason to reinvent the wheel here, and there's always someone that attempted such a common task before you. Just because you don't have 3,000 different choices like in Java, doesn't mean that you won't find high-quality code for what you need to do (in fact, I'd argue that Lisp code has a much higer quality on average than Java code). As for hiring, I think Paul Graham has already given the best comments on this. Simply put, a good programmer can be taught to write good Lisp, even if they don't know it by the time you hire them.

GlennS• 10 years ago

I'd say web's request -> response model is pretty well suited to functional programming.

It seems popular amongst JavaScript programmers at the moment as well. React, which appears the current hotness, is based on making a functional interface to the DOM.

Although actually, I'd say the main reason to use a Lisp for web gunk is the macros rather than the functional programming. The tree structure maps very naturally onto HTML generation. There are some really nice libraries (in particular I'm familiar with Clojure libraries) which make the server-side part of a web app really neat. You can basically skip the whole 'templating' bit.

nurettin2• 10 years ago

For UI application programming You can do without FP language features. Hell, you can even do without smart pointers checked arrays or GC.

But you cannot do without a decent desktop or web development experience. I got involved with pretty large desktop applications written in delphi. The design-time, the debugger, native compilation without having to install some msvc runtime, deriving and combining visual components, it just gives a smooth experience for desktop apps. If lisp has such well thought-out frameworks, I would jump ship just to get at the language features it offers.

junke5• 10 years ago

Would you pay for it?

http://www.lispworks.com/products/capi.html

nurettin2• 10 years ago

I did buy frameworks for work, and this does look OK, but coding gui by hand is not something our developers can handle in large scales.

hellofunk16• 10 years ago

For UI application programming You can do without FP language features.

Wow, do I ever disagree with you on this. The absolute best UI development process I've ever had -- super easy, fun, fast, painless -- was using Clojurescript's Om, which is functional to the core. Doing anything else, like going back to the polished UI API's of Apple still feels arcane by comparison.

cheez6• 10 years ago

You should never use CLJS because then you'll hate going back to JS.

cheez6• 10 years ago

Funny, the hottest new thing in UI (React/Redux) loves to be functional.

For what it's worth, nothing matches the Delphi experience, not even today so I don't imagine CL would be any better. I worked for the Delphi guys for a bit and their maniacal focus on the end user is something that few development environments can claim as a philosophy.

ww5202• 10 years ago

I want to see people's list of reasons for why not lisp.

Edit: Ok, this is not trolling. I quite liked Lisp and actually had implemented a version of common lisp from scratch based the Guy Steele's CL reference back in the undergrad time. I just found that beyond academic and a few Emacs packages, I didn't use Lisp at all, for one reason or the other. I just want to hear people's reason for not picking Lisp for their work.

kqr• 10 years ago

I dislike Lisp (and Lisp-inspired languages like Scheme and Clojure) for two reasons:

1. their weak and dynamic type system, and 2. they don't control side-effects.

happyslobro4• 10 years ago

Haskel programmers: you can spot them by that constant pained look, as if they have to step in horse poop whenever they go outside.

diminishedprime• 10 years ago

I'll agree that clojure doesn't enforce control of side-effects. But with the recent work on clojure.spec, your first complaint has become (even more of) an asset.

ubertaco• 10 years ago

I've tried on a few occasions to really like Lisp. I love it's simplicity and recursiveness. The "code as data, data as code" thing is really cool.

But I love static typing. Not only for "safety-net" reasons, where the compiler lets you know that you've broken stuff, but also for documentation reasons (I don't have to guess at the shape of each "c" in "cs" in "(mapcar some-fun cs)", I can just look at its type and know its shape), and I really like it when that type system lets me be more expressive, like in MLs (where I can pattern-match by the various members of an ADT and destructure them as I go).

I also find that idiomatic Lisp likes to nest things too much to be really readable for me. If I don't have types to tell me how the data's being transformed, at least give me named functions and well-named variables instead of very-deeply-nested anonymous functions everywhere.

And finally, I've never been completely convinced that macros are a great idea. They're far too easily abused to create "the Scala problem" of having code in language X that is still unrecognizable to someone else who also writes in language X.

joshmarlow• 10 years ago

I totally agree with you here; statically typed languages (OMG OCaml) are great for making code very readable/documented, IMHO. I just have so much more confidence in my code when the compiler has my back.

Another commenter pointed this out, but Typed Racket is a Racket variant (which is a scheme-ish variant) that implements "gradual typing". I've played with it here and there and am about to dive in again.

https://docs.racket-lang.org/ts-guide/

ww5202• 10 years ago

That's a very good point. Type information certainly help reading. That might explain why Lisp is easy to write but hard to read. I did find myself having to go into a function's code to figure out what it does and what the parameters are. Hard to read code make sharing difficult and hinder widespread language adoption.

hellofunk16• 10 years ago

I'm of the opinion that there is only one lisp right now with some serious potential to dominate in the future, and that is Racket. Why? For one very simple reason: unlike all other lisps, there is serious, ongoing, and lengthy research into correctly bringing a static type-checking process to the language. Clojure's core.typed doesn't count here, as it is full of significant holes that invalidate its entire point -- though, those holes are very possibly getting filled in the coming years as they attempt to refactor core.typed following the Racket model. But the Racket team is doing something with lisp that is true to its core, not just with types, but with many other features as well.

Interesting piece of trivia: this website you are reading right now is built on top of Racket.

vertex-four• 10 years ago

Isn't HN on Arc?

emidln4• 10 years ago

Arc runs on mzscheme which turned into Racket. The last official release of Arc requires an older release of mzscheme from before lists were immutable. There is a community-maintained fork called Anarki [1] that runs on modern Racket.

[1] http://arclanguage.github.io/

emidln4• 10 years ago

What makes the Racket model more correct than the system based in Sequent Calculus [1] provided by Shen?

[1] http://www.shenlanguage.org/learn-shen/types/types_sequent_c...

hellofunk16• 10 years ago

This is the first I've heard of Shen, so I can't say.

However the Racket engineers, particularly Matthias Felleisen, are doing cutting-edge research into types. Typing is not a solved problem, because if it was, there wouldn't be a static vs dynamic type debate that lingers ad nauseum.

Racket is in an interesting sweet spot for typing research because it is a traditional lisp with dynamic types, to which gradual typing that works both in typed and untyped land has been introduced and continues to undergo extensive attention.

hashkb• 10 years ago

I believe YC builds their new apps in Rails now.

mordocai• 10 years ago

It might be good to point out to you that the general industry cares very little about things like this.

Racket could have literally the most advanced and robust type system out of all programming languages in existence, and still be used by the same amount of people as today and have a hard time convincing anyone else to use it.

What makes a language popular from what I have seen are libraries, frameworks, and community.

Edit: Racket has all of the above, so obviously there are other factors as well.

hellofunk16• 10 years ago

The tide is turning. Naughty Dog video game company (Uncharted, Crash Bandicoot, Jak and Daxter) uses Racket as its scripting language now.

galdosdi• 10 years ago

The general industry is in fact starting to care about static typing; I've seen it happening lately. The last batch of hot languages included a lot of untyped languages like Python, Ruby, and JS. But the latest batch of rising star languages are typed, eg Go, Rust, Scala. And C/C++, C#, and Java are still all going strong.

People are no longer buying the myth that the benefits of static typing can only be gained at the expense of heaps of boilerplate. The gospel is slowly spreading.

tempodox2• 10 years ago

A great read. Few words, and everything relevant mentioned.

cwmma• 10 years ago

The whole manipulating code as date just always seems like such a rich target for exploitation like that ruby yaml bug from a while ago [0]. It seems like a features that would be great if your code never needs data from anywhere else but as soon as you gave data coming in from other sources it would be a bit of a nightmare.

0: http://blog.codeclimate.com/blog/2013/01/10/rails-remote-cod...

Ooh Ooh My Turn Why Lisp? (2008)

Related Stories