Lisp is Too Powerful

https://c2.com/cgi/wiki?LispIsTooPowerful

by gnosis • 15 years ago

128 96 15 years ago

sjs2• 15 years ago

You could make the same argument about Unix, but I still say that with great power comes great responsibility. If you can't handle the responsibility maybe you should use watered down tools. That's somewhat bleak though and following that logic you end up with Java. Java is not the worst language on Earth but I doubt it's your favourite and no one writes it w/o a code generating and refactoring IDE.

arkitaip• 15 years ago

This is your selection bias speaking. Tons of people LOVE to code in Java or the similar C# for that matter because that's the most suitable tool they have.

RodgerTheGreat2• 15 years ago

This is both glib and anecdotal, but Java is one of my favorites and I never write Java with an IDE. It has an extensive standard library that I can count on having available without requiring users to install optional packages, it's easy to write portable code and I can write software for everything from a cheap feature-phone to a high-end server.

There's a lot of horrible Java code on the internet, mostly because there are tons of people writing Java. Nevertheless, in good hands, Java can be quite elegant and succinct.

chimeracoder• 15 years ago

> Nevertheless, in good hands, Java can be quite elegant and succinct.

Could you elaborate, perhaps with an example? Not trying to start a flamewar or anything - but I honestly can't imagine a case where Java is succinct relative to other languages. And as far as elegance goes, Java is so overly verbose and fixated on classes ('too many classes? there's a class for that!') that I have a hard time thinking of it as elegant.

But then again, maybe I'm just used to reading bad Java code everywhere, and you've been lucky enough to find the good stuff!

palish• 15 years ago

Hey, that is pretty awesome. I had zero Java experience, but I felt inspired by your post, so I searched "java without ide" and got an XML parser up and running in approximately 20 minutes.

For the curious reader, here's how to do Java entirely from the command line:

1) install the JDK and add its "bin" directory to your PATH. For me this was "jdk1.7.0_02/bin"

2) copy and paste http://pastebin.com/R47bwDZx and save it into a new file called "test.xml"

3) copy and paste http://pastebin.com/Ut5dKBkn and save it into a new file called "ReadXMLFile.java"

- note, this is not my code. It's just a snippet I found from the 'net. Feel free to clean it up. =) My goal was "to get up and running as quickly as possible". Production code would be cleaner than this.

4) run these commands:

  javac ReadXMLFile.java
  java ReadXMLFile test.xml

And the XML file is parsed!

So why go through all this trouble, when you could just write it in Python?

Deployment comes to mind as one reason to go with this approach; fewer systems have Python installed than Java runtime.

So therefore when you deploy your app, you'd have to either 1) make customers install Python, or 2) (more likely) ship the contents of the "Python27" folder along with your Python app, and invoke your app via a shell script, or something. I'm not sure.

machrider• 15 years ago

You may want to link to http (rather than https), since this site has a bad certificate. Big scary warnings on Firefox here.

sp332• 15 years ago

I was expecting a self-signed certificate, which I usually accept, but this one shows up as "localhost.localdomain". No thanks, I don't want to trust this cert to sign for my localhost :)

watmough• 15 years ago

http://c2.com/cgi/wiki?LispIsTooPowerful

Yep, came here to post that too.

spacemanaki2• 15 years ago

There's some definite trolling down at the bottom half of that page: "Huh? I've seen no objective evidence that BrainFsck is more challenging for business applications and systems software programming than is Lisp. I invite you to provide clear evidence that it is more challenging."

Leaving that aside though, other people smarter and more experienced with Lisp than I have suggested that this "problem" is not unique to Lisp, and may not be a much of a real problem. I really think it's just a question of having good documentation and sensible style which explains what you need to know to use the magic even if it doesn't explain what's underneath. It's true that a lot of "lone hackers" aren't going to be writing good (or any!) documentation, but that's not a Lisp problem.

Ruby's metamagic is a common example. I already know Ruby and recently I've been learning Rails. I know there's a lot of magic going on behind the scenes, and occasionally I'll read some example or something and know that underneath there's magic that I do not entirely understand. I can continue studying the framework as a user and probably even get away using it for a while without completely understanding the magic underneath. I doubt that every person who has used Rails commercially completely understood every part of the framework that they use, since that would be a huge drag on their ability to ship their thing.

Another interesting example: This past week I talked to someone at a nearly pure Scala company and he described this DSL that they wrote (or just use? I can't remember) to interface with SQL databases. It exploited the fact that infix operators are really just methods and that you can define implicit conversions on existing library types that allow you to sort of extend them (I don't actually know Scala, so this may be slightly off base). The snippet of code he showed me, while it looked like Scala, would be turned into an SQL clause. Programmers who use this don't necessarily need to understand all of the gritty details so much as they need to understand what the designer intends and understand the semantics of SQL and of the DSL.

In Common Lisp, "lambda" is a macro. I don't really know how it works, but I know that it expands into lower level code defining a function and a closure. It's the same way with standard macros (in some Lisps) like let, letrec, cond, if, and, or etc... Most Lisp programmers who are not experts can use them understanding that they are macros but maybe not knowing their complete implementation because they are very well defined (and because they are pretty simple).

More complicated macros like "with-open-file" can be used by programmers who probably have some idea of how the macro works, but not a complete understanding. As long as your own macros and Lispy magic are documented sufficiently and designed sensibly programmers should be able to use code in your "world" without understanding it completely. At least to start.

N.B. I haven't used Lisp in a commercial setting, so it's entirely possible all this is bunk and I'm just being naive. Wouldn't be the first time.

edit Upon re-reading this monster comment I realize it might come across that I'm advocating some kind of "cargo cult" or copy-and-paste style of ignorant programming. I'm really not. My point is that while understanding your tools is important, so is knowing when you don't need to peel back the curtain and instead need to address the problem at hand.

LearnYouALisp• 15 years ago

I think they are poking fun at the often-misused technicality that "all Turing-complete languages are equivalent", which is obviously beside the point.

breckinloggins• 15 years ago

Lisp, more than any other class of language, is what you make of it. Also, it is just as susceptible to the whims of culture as other languages (perhaps MORE so). Take the Clojure world, for example...

Because it is being used by a generation of programmers who cut their teeth on Rails and were frustrated by J2EE verbosity, popular Clojure code tends to be written such that APIs are quite readable and bereft of too much cleverness (the cleverness is usually hidden in the implementation of the API, rather than the interface).

Examples: Compujre, Ring, ClojureQL, Encanter, etc.

S-Expressions don't befuddle people. People befuddle people.

billrobertson42• 15 years ago

C/C++ can be taken to extremes with preprocessor abuse. If you've never looked at the obfuscated C programming contests, then you should. As far as Scala DSL's go, didn't somebody post one the other day that made an ascii art picture of a Christmas tree into a valid scale program? Isn't there also a Scala DSL that lets you make it look like basic?

Sure, there are languages that don't let you get at the meta, but just because the ones that let you do can be abused does not invalidate the usefulness of the notion.

tikhonj• 15 years ago

I don't know about Scala, but somebody did embed basic into Haskell: http://hackage.haskell.org/packages/archive/BASIC/0.1.5.0/do...

However, it's at least partly taking advantage of the fact that normal Haskell actually looks (in shape, anyhow) vaguely like basic as is. At the very least, Haskell doesn't force you to use parenthesis or braces everywhere.

Here's an example: http://augustss.blogspot.com/2009/02/is-haskell-fast-lets-do...

vorg• 15 years ago

Someone also wrote a Groovy DSL to read Java bytecodes and translate them into Groovy AST nodes, so they could eventually become Java bytecodes.

draegtun• 15 years ago

> Isn't there also a Scala DSL that lets you make it look like basic?

Yep, it's called Baysick - https://github.com/fogus/baysick

motxilo• 15 years ago

bitcracker2• 15 years ago

If Lisp is too powerful then you are too weak :-)

Lisp requires a new way of thinking - in recursion, lambdas, mapcars etc. - to write good code which reflects the awesome abilities of Lisp. Unfortunately many people don't grasp it. They don't want to think or to learn superior ways if they just can use a language which makes them able to solve their problem. The way to the solution doesn't matter a lot if the solution itself works.

Btw the same phenomenon happened with Ada. The Ada 95 language is awesome. I admired it, it was real fun to use it. But average programmers are simply overwhelmed. That's the reason why Ada died.

Many people also complain about Unix and Linux but if you take the effort and learn it seriously you will love it.

stevecooperorg5• 15 years ago

What awesome abilities are you talking about particularly? Recursion, lambdas, and mapcar are available in just about every modern languge, be it VB, Ruby, JavaScript, or PHP.

pavelludiq4• 15 years ago

Here is a short list of awesome lisp abilities, some of them not unique to lisp, but still interesting read: http://random-state.net/features-of-common-lisp.html

algoshift• 15 years ago

How about FORTH then?

RodgerTheGreat2• 15 years ago

Forth can be a pretty good way to go about implementing a Lisp, actually. I have a book[1] that uses Forth to build a set of list manipulation facilities inspired by Lisp and then in turn use those to construct a Prolog-like DSL for expressing expert systems. I've actually been tinkering with a pair system in my own Forth dialect that could be interesting if you've ever wondered what recursive list operations would look like in a concatenative language.[2]

[1] http://www.amazon.com/Designing-Programming-Personal-Expert-... [2] http://pastebin.com/q9hMS1rE (warning: WIP)

ohyes2• 15 years ago

I use lisp in a commercial setting.

The main point about lisp is 'there is no accounting for taste'.

Lisp leaves most things up to the programmer's taste.

You can write Lisp that looks like Fortran, or C++/Java, or Scheme.

You can make a DSL that directly models the problem.

You can use objects or no objects, do everything in CLOS or do everything with structs.

You can write your own object system.

You can make it fast or slow, you can use correct data structures or you can do everything with lists.

You can use macros for everything or you can never touch macros ever.

You can make your program one big macro.

None of these things are 'bad taste.'

Most people have different taste from you.

Most people only end up learning the part of the language consistent with the paradigm that they like.

If you only know that piece of the language, you will have difficulty working on someone else's project when they are working in a different paradigm.

None of the individual pieces of the language are particularly difficult.

People complain macros are difficult to understand. Macros are easy. If you can understand a program that concatenates lists to make a new list, you can understand a macro. Macros are quite literally 'just lisp code'.

Are some macros written in a way that you can not personally understand? Most likely, but that is not an issue inherent to macros. You have to be careful about checking inputs, and creating the proper debugging and type checking at macro-expansion time. Just like any other program. There does seem to be a stupid tendency for people to cram an entire macro into a single function. This is foolish, the whole point is that you have the entire power of this lisp runtime. There is no reason to write it like c pre-processor garbage.

So, how does one write good lisp code? Well, one way is to pick some standards. This is as difficult has having someone in charge willing to gently say 'this doesn't really match up with the style of the code around it'. If I've inherited some code, when I do a bug fix, I'm going to do my damnedest to stick with the style that it is written in (unless it is truly tagbody/go awful, in which case I might rewrite).

This kind of turned into a rant, I apologize. I guess my point is that sure, lisp is powerful, but the real issue is the number of options that it provides. At some point, you have to pick a subset and a style, and go with it. And then you have to be comfortable learning if you inherit something that you don't know yet.

ScottBurson• 15 years ago

Lisp leaves most things up to the programmer's taste.

I don't entirely disagree, but I think there's more to it than that. While the language certainly does offer some free choices, more often there are advantages and disadvantages to each, so that in any particular situation, some choices are better than others. Becoming an expert Lisp programmer requires learning about these tradeoffs, which takes experience and, usually, guidance from existing experts.

That's true of any language, of course, but some of the facilities Lisp offers are rare among other languages, so that people coming to Lisp from some other language are unlikely to have experience with them.

Oh, one point about macros in particular. If you have to resort to reading the implementation of a macro to understand what it does, the person who wrote it screwed up. Macros should always have documentation strings explaning their syntax and semantics. If you find yourself in that situation, the best thing to do is to go to the REPL and use `macroexpand' interactively to see the expansions of the macro calls you're interested in.

ohyes2• 15 years ago

There is a saying 'perfection is the enemy of done.' In any particular situation, there is probably not a solution that is both optimally efficient and also optimally elegant. (This is true in any programming language). If you work towards that goal too much, you will likely miss your deadline.

But the point is, the little advantages and disadvantages don't matter until they do. There isn't going to be a big difference in most programs between using a loop to iterate a sequence, and using map nil with a lambda, and using do* (for example). In fact, whether there is any difference at all in the resultant assembly or byte code will depend entirely on the compiler implementation. It is a style thing... so try to be consistent, and work within what you are comfortable with.

Currently, the only thing I can really think of that is actually really unique is the macro facility (and that is only because, as soon as a language adopts the macro facility, it becomes a lisp.

As a programmer, the focus of my job is learning new things. I am basically a mechanism for translating the new things that I have learned into computer code. If I can't learn a few measly language features, what good am I going to be as a 'thing I just learned to computer' translator. And like I said, use what you are familiar with, until you are faced with something someone else wrote, or you have the time to learn new things. But don't punt.

If someone doesn't have experience with a given piece of the language that has been used, I expect them to pick up a book or online resource about it (and then, possibly most importantly, play with it). It is not hard, but it does take effort. There is nothing in common lisp that requires genius level intellect. (God knows, I'm certainly not that bright). No one requires that fresh-faced C interns be pointer arithmetic gods, but i'm sure they are expected to learn it if it is part of the job.

Oh, one point about macros in particular. If you have to resort to reading the implementation of a macro to understand what it does, the person who wrote it screwed up. Macros should always have documentation strings explaning their syntax and semantics. If you find yourself in that situation, the best thing to do is to go to the REPL and use `macroexpand' interactively to see the expansions of the macro calls you're interested in.

I'll add that in addition to doc strings for syntax and semantics, there should also be assertions written into the macro about the syntax and semantics. If I am passing a number or list where the macro is expecting a symbol, an error should get thrown during the macro-expansion phase. Macros are programs like anything else. Validating inputs and throwing an error at the earliest possible time is a good rule to go by.

So, macroexpand-1 is a good start, if the macro is implemented correctly, does what its documentation says, and you are simply flubbing the syntax. (Of course, it should be yelling at you for flubbing the syntax).

However, when there is a bug in a macro, the only thing that macroexpand-1 will tell you is that the macro doesn't work. You'll macro-expand it and say 'yup that's the wrong generated code.' It doesn't really tell you anything about how to actually fix the macro unless you are already familiar with the macro's code. Having examples of inputs with bad outputs will aid in pinpointing the problem, but not unless I already understand how the program works.

Macros are lisp programs, and can be as complicated as any arbitrary lisp program. Writing a more complicated macro is not screwing up (I think this is an important distinction to make)... inadequately documenting, explaining, and bulletproofing it is. Someone might have to debug it later, so strive to write readable macro code. It isn't hard as you are just constructing lists and writing normal lisp code with minimal efficiency requirements.

So I guess that was a roundabout way of me saying "I agree, mostly."

stevecooperorg5• 15 years ago

That's really helpful -- nice to hear from a pro. Thanks.

sigil• 15 years ago

Always lacking from these kinds of debates about code: actual code.

Does anyone have examples of crazy unmaintainable Lisp code we could look at?

On the other side, what examples of powerful / elegant Lisp code do you feel best make the case for Lisp?

I fully realize this is subjective ("no accounting for taste"), and that a handful of anecdotes doesn't really settle anything. Nevertheless, I'm interested in what the failure modes might be for Lisp, and whether they have analogs in the languages I'm more familiar with.

I'm also bothered by a tendency in the Lisp community to say, "Lisp is the best, and I have all this awesome Lisp code, but no, I'm not going to show it to you." If someone asked me for great C, C++, Python, or Perl code, I have favorite examples I'd point them to without hesitation. What gives Lispers? Is your Lisp code so personalized or specific to the problem that you fear it really wouldn't make any sense to an outsider? If so, how come this doesn't translate into maintainability problems?

pavelludiq4• 15 years ago

A well engineered lisp(and any other language actually) program isn't a stream of beautiful lines, but a set of components, any of which might have a complicated implementation, but all of which have a good clean interface. Lisp is beautiful not because it allows you to write beautiful 20 line programs, but because it allows you to design large systems that still have a chance to be maintainable, despite the complexity of the problem they are solving.

I can still show you examples of beautiful and horrible 20 line lisp programs, but I'd rather show you examples of large scale design. The most popular might be Emacs. Emacs has a million lines of elisp, imagine if it was written in java or C++, scary thought :)

In a large system like emacs you'll find many examples of beautiful and ugly code, but the overall system is still beautiful and maintainable. This talk by Stuart Halloway might explain what i mean by that: http://vimeo.com/1013263

In a nutshell, emacs is big, but small for its size, meaning that those 1000000 lines of lisp do much more than a million lines of java will ever be able to do. That property comes in part by using a lisp as an implementation language(and not a very good lisp at that :)

dmansen• 15 years ago

Great talk, thanks for the link. The power of emacs isn't immediately obvious, until you try to use it for something it wasn't explicitly designed for.

spacemanaki2• 15 years ago

I don't think my own Lisp code is really worth showing off so I'm going to cheat.

Here's some impressively awesome Lisp code, that leverages features that might be considered "too powerful". It's some Common Lisp from Peter Norvig's Paradigms of AI Programming: http://norvig.com/paip/auxfns.lisp

Checkout any of the macros in that file (search for "defmacro"). If you need a primer on Common Lisp macros I think Peter Seibel's intro is sufficient: http://gigamonkeys.com/book/macros-defining-your-own.html

The memoization stuff is pretty cool and so is the stuff for managing resources (defresource and with-resource).

That's one of the best books on programming I've ever read (I mean started and set aside then picked up again and again for many months and haven't finished), and there's a lot of really nice and elegant code in it. The way he describes searching graphs and trees is really nice but that code doesn't use any of the really Lispy stuff. It would look pretty similar in Python or Ruby. The rest of the code is here: http://norvig.com/paip/README.html

mnemonicsloth• 15 years ago

Progress.

The standard answer to If-Lisp-Is-So-Great used to be "It isn't."

hluska• 15 years ago

For the most part I enjoyed reading this article, however I'm uncomfortable with one assertion:

"This is why Lisp is a HackerLanguage instead of a commercial language: hackers are generally loners who don't care if others can figure out their code (at least while they are in the mode or role of hacking)."

I don't know many hackers who this applies to. Rather, the DRY ethos seems to extend into readability and accessibility - most acknowledge they'll one day pass the code to someone else to maintain and too much complexity makes that near impossible.

raganwald7• 15 years ago

The notion of programming languages being “too powerful” rests on fallacious assumptions.

The faulty reasoning is this: If Hacker Hortense does something with powerful language L that Newbie Nathan finds confusing, we assume that language J won’t permit Hortense to do that, and therefore if we standardize on language J, fewer bad things will happen.

This reasoning is faulty. First off, if Hortense and Nathan don’t see eye to eye on how to write programs, no language will solve the problem, because you have a disparity of experience and/or education. If what you want is code Nathan will like, and you don’t want to raise Nathan to Hortense’s level, you have to drag Hortense down to Nathan’s level with coding standards. And you could have done that in Language L just as easily as J. If you ban L, what will you do when Hortense implements Parser Combinators and Monads in J?

The first flaw is the assumption that language features introduce complexity, when in reality it isn’t the features, it’s the solution to the problem that confuses.

This indirectly leads into the second fault with the reasoning. There is a hidden assumption that Hortense is going to write so much code, and there will be so many “problems” Nathan will have with his code, and every problem we eliminate is one fewer problem in the result. This is not how software works.

Let’s say Hortense writes some code in L, and Nathan complains that there are seventeen things he doesn’t understand. Ten of them rely on features of L, seven are independent of the language. If we tell Hortense to rewrite things in J, we cannot assume that the result will have seven things Nathan doesn’t understand. It might have even more as Hortsense works around J’s deficiencies.

For a demonstration of this, look at any modern Java Dependency Injection Framework. These complex and opaque software machines are made up of of XML, interfaces, and classes. They exist in no little part because Java lacks the reflection and meta-programming of languages like Lisp or even Ruby. Are these thing simple by virtue of being written in Java instead of Lisp?

Imagine for a moment that DI frameworks don’t exist, and that Hortense had built something in Lisp with macros for dependency injection. Nathan is nonplussed, so Hortense rewrites it in Java and rolls her own DI infrastructure. Will the result really have fewer things that Nathan doesn’t understand? Or will it be even more inscrutable thanks to the accidental complexity of working around Java's limits??

The “problem” is not that some languages are too powerful, it is that people imagine that teams of programmers can work together on complex problems without communicating, and some people assume that when there is a disparity of experience of knowledge, that people can work together without educating or learning from each other.

sirclueless• 15 years ago

I'm not so sure I agree with you, because there is something to be said against your counterexample. In java, I can reason as follows: the Java Dependency Injection Framework is gigantic and convoluted, therefore it is going to be complex and difficult to manage.

In Java there is a sort of ceiling, one might call it an upper bound on the mindfuckery per square inch of code. In 100 dense lines of lisp, there is no a priori reason to assume that no one has written a recursive descent macro-expanding s-expression handler for arbitrary routing of network packets in a rules-based DSL. In the same amount of dense java code there are rarely more than 200 method calls, all statically checked, and rarely more than 100 variables, typed and scoped.

From John Carmack's recent article on static checking: "If you have a large enough codebase, any class of error that is syntactically legal probably exists there." Now, he is concerned with actual defects, but the same rules apply even more strongly for stylistic rules. As the size of a codebase increases, the probability that any particular language feature is absent approaches zero. And in a language like lisp, where you can write your own language features, this is moderately horrifying.

The fact that you can replicate java's dependency injection in a few orders of magnitude less code in lisp is not a comfort to me. Because the 10,000 lines of code in Java's Dependency Injection Framework is a red flag to me. The chances that someone who writes the same thing in lisp has drastically simplified the implementation are not so high.

j_baker5• 15 years ago

Perhaps I'm missing something here. Argument 1 seems to basically be that Java code is bigger than Lisp code. Argument 2 is that big code bases tend to use all of a language's features.

Why couldn't you solve the second problem with the first? As you've already pointed out, Lisp code is smaller. Thus, you don't have to worry about the "large code base" problem like you would with java.

raganwald7• 15 years ago

the 10,000 lines of code in Java's Dependency Injection Framework is a red flag to me.

We agree.

The chances that someone who writes the same thing in lisp has drastically simplified the implementation are not so high.

We disagree.

First, when I roll my own, I scratch only my own itch. I don’t need to build something that works for everyone, everywhere. It’s like Microsoft Word: MSFT boasts that most users only need 5% of what it does, but every niche of users uses a different 5% of the whole thing.

But can I roll my own? Well, I suggest that the answer is more likely to be “yes” in Lisp than in Java. First, folklore suggests that defects are constant per line of code. Therefore, if I need fewer lines of Lisp than of Java, I should have fewer defects to contend with. I assume three things. First, I need much less than the full framework’s functionality. Second, Lisp is more expressive than Java, so I need fewer lines of Lisp for any functionality than of Java. Third, I suggest that languages with meta-programming support are particularly well suited for tasks like dependency injection, reducing the amount of code I need to write even further.

Now, the big DI framework is written by someone else. So is it free? No. I need time to learn it, time to use it, I can make mistakes using it, I can get an XML configuration wrong, I can implement an interface when I am supposed to extend an abstract class, I am not immune from defects just because I am using a library.

So the net question for me is whether the chances of successfully rolling my own feature in Lisp for my project’s specific needs are greater than the chances of successfully using an existing framework in the Java world=, and parallel to that, the question is whether someone else working with my code will find it easier to decipher the XML configuration and interfaces and classes I have written to work with a Java DI framework or will find it easier to work with the smaller, simpler and more compact Lisp code written for this specific project.

Reasonable people can go either way on this, I find it hard to believe that Java is “obviously” a win, especially if they’ve used one of these big frameworks with the many gotchas (as I have).

p.s. Of course, the wild card is that there are plenty of libraries in Lisp as well. I reject the notion that every Lisp programmer reinvents everything from scratch: https://github.com/lprefontaine/Boing

nwbrown• 15 years ago

While I don't agree with everything you say, I'm definitely reusing the term "mindfuckery per square inch of code".

stevecooperorg5• 15 years ago

Isn't his point rather that Lisp goes beyond the normal definition of 'programming language'?

"Lisp is almost an interpreter builder rather than just an interpreter/compiler"

So most languages have a fixed grammar, and programs are always written in that grammar. Java functions, whatever problems they have, are at least always declared using a fixed grammar -- something like

    function ::= returnType Identifier parameterList;

So even if you don't know what a function does, at least you (and tools) can easily see functions in the source code. The same applies to loops, classes, conditionals, etc. (Same for python, BASIC, whatever.) What Lisp does goes beyond that; I can define a new conditional language construct, or looping grammar, or object system...

"This means that one can make it be just about anything they want. Thus, its culture is almost a lack of culture."

The fear is that you'll end up having to join a project using "Dave's Object System" or "Nora's Conditionals Library". And that's where Lisp has the Aw(esome|ful) power to write in wildly divergent styles -- in fact, written in whole new grammars.

I suppose there's an idea that code can be written in a 'natural' way for the language. So you have this idea in python programming of something being 'pythonic' -- not just corresponding to the the python grammar, but also in a particular, respected programming idiom. So the question is, when you have the power to write your own grammar, can you then still write in a good, predicatable style that makes the code a good form of communication for other programmers? Is there a 'lispish' way to write code? (I suppose the same question applies to things like internal Ruby DSLs.)

Honest questions; although I've given Lisp a good shot, I've never written anything longer than a few hundred lines...

raganwald7• 15 years ago

Well, it’s true that Lisp has meta-syntactic programming and Java doesn’t, however since Java doesn’t have homoiconicity, you end up with people building things where a lot of the meta-programming is baked into XML files.

So, you could argue that a large, complex Java program can’t be understood just by reading it a line at a time, you have to figure out how it is all wired together via the XML files.

I think what I’m trying to say is that large and complex systems are, well, large and complex whether in Lisp or java. But Lisp has standard libraries just like Java, e.g. CLOS. So it’s true Dave could write his own objects, and Nora her own conditionals, but I wonder whether that is more folklore than practical reality on commercial projects.

pavelludiq4• 15 years ago

Is there a 'lispish' way to write code?

Yes. If you read enough Clojure, Scheme and Common Lisp code you'll see that they have their own different styles. On a few occasions I read Common Lisp code written by schemers, it was somewhat ugly and inelegant, although the same code would have been elegant if written in scheme. Here's an example from an Erik Naggum article( http://www.xach.com/naggum/articles/3195982690560792@naggum.... ). First the Scheme in common lisp:

  (defun make-account (balance)
    (labels ((withdraw (amount)
                       (if (>= balance amount)
                           (progn
                             (setf balance (- balance amount))
                             balance)
                         (format t "Insufficient funds")))
             (deposit (amount)
                      (setf balance (+ balance amount))
                      balance)
             (dispatch (m)
                       (cond
                        ((equal m 'withdraw) withdraw)
                        ((equal m 'deposit) deposit)
                        (t (format t "Unknown request -- MAKE-ACCOUNT")))))
      dispatch))

Perfectly reasonable way to do it in Scheme, not in Common Lisp. Heres how you do it in idiomatic Common Lisp:

  (defclass account ()
     ((balance :initarg :deposit :initform 0 :accessor balance)))

  (defmethod withdraw ((account account) amount)
     (decf (balance account) amount))

  (defmethod deposit ((account account) amount)
     (incf (balance account) amount))

  (defmethod withdraw :before ((account account) amount)
     (unless (<= amount (balance account))
       (error "~A has insufficient funds to withdraw ~A." account amount)))

I'll let the code speak for itself :)

Lisp allows you to turn code like the first example into code like the second.

edit: formatting

wpietri6• 15 years ago

The more interesting situation to me isn't novice vs expert. It's the same one that bothers the person who wrote the top of the original article: pro vs pro.

A former boss of mine said that programmers are like hairdressers: whenever she changes to a new stylist, the first thing they say is, "Oh my god, what did that last person do?" That's certainly what I say when I have to examine an existing codebase. More expressive languages allow for more stylistic variation, and one's style is deeply rooted in one's values and experiences. One person's brilliance is another person's WTF.

I think the right solution to this involves developing a house style, which for my team involves zero code ownership and a lot of collaboration. Eventually the team converges on a set of idioms that work for everybody, so style issues are minimal.

But that's not always the working reality. If I have to inherit a bunch of code from a random developer, I'd much rather it be in Java than, say, Ruby. Java definitely limits brilliance, but it also limits too-clever-by-half idiocy. The downside risk with a Java code base is smaller.

raganwald7• 15 years ago

I agree about developing a house style. It is not my Java programming experience that the Java language limits the too-clever-by-half idiocy.

If you want to say that the kinds of companies that hire Java developers don’t hire clever idiots, that’s a different discussion. But I’ve seen so many convoluted architectures built out of Java that it feels to me like too-clever-by-half is a constant across all languages, and the very best thing for me is discouraging cleverness, with second best being let people do their clever things in a language that helps them, rather than making them write even more inscrutable clever code just to get away with being clever in Java.

This is my point above, and perhaps it doesn’t match your experience, namely that some clever things will be done, and Java will not prevent them from being done, but merely prevent them from being done simply and cleanly. So why not have this other “pro” do what she is going to do, but do it simply and cleanly thanks to a powerful language, rather than resorting to XML and other shenanigans?

But honestly, I fear we are now having a weird discussion where our working assumption is inheriting code from people we don’t respect. It’s far, far easier to simply work with reasonable, sane people, than to make all your choices predicated on the idea that your colleagues are psychopaths.

It makes for witty sayings, but terrible careers.

raganwald7• 15 years ago

Argument for the prosecution, “The Rule of Least Power:”

http://www.w3.org/2001/tag/doc/leastPower

postfuturist• 15 years ago

The paper describes a completely false dichotomy between powerful and simple. In the context of an extremely simple and powerful language like Scheme or other Lisps, the argument is almost meaningless.

gruseom10• 15 years ago

How is it not an obvious oxymoron to call a language "too powerful"? Imagine a physicist or a mathematician calling a theory "too powerful". It explains too much! You can prove too much with it!

We're not even close to understanding how to reliably make good programs, so naturally we don't understand how to do that with Lisp either. Lisp is just a particularly pure medium for programming. Most of the things people are saying about Lisp ("it allows people to invent their own little worlds") are really just statements about programming.

Oh and one other thing. A pox on the terms "readable" and "unreadable" floating around as free variables. They are hopelessly relative and their prominence in any discussion of programming (or should I say every discussion of programming) renders said discussion pointless. We literally don't know what we're talking about.

As pnathan brilliantly put it in another thread (http://news.ycombinator.com/item?id=3387432): "Intuitive is just what you've seen before." Bravo, pnathan.

j_baker5• 15 years ago

Even worse than "readable" and "unreadable" are "maintainable" and "unmaintainable". At least I know what "readable" means at some level. But "maintainable" usually turns into a generic catch-all term for "this doesn't scare me too much".

wpietri6• 15 years ago

It's not an oxymoron because programming languages are tools for accomplishing specific purposes, and power is not the only thing that makes a tool good.

For what it's worth, one can easily call a theory "too powerful" for practical purposes. E.g., one often uses Newtonian mechanics to analyze something rather than Einstein's more powerful approach. Human brainpower is a sadly limited resource, so "easy" often trumps "powerful".

gruseom10• 15 years ago

It's funny you should cite limited cognitive capacity, because the same starting point leads me in exactly the opposite direction.

"Human brainpower is limited, so we should use weaker languages" seems a strange argument. The whole point of using more powerful languages is that they can express more with less, allowing us to fit more into limited human RAM. It's because our brains can't manage millions of details that we don't build systems directly out of machine instructions in the first place and resort to cleverer strategies like higher-level languages. At what point along the way does that process stop being valid?

ezyang• 15 years ago

'Imagine a physicist or a mathematician calling a theory "too powerful". It explains too much! You can prove too much with it!'

Actually, I can perfectly reasonably imagine a physicist complaining that a theory is too powerful. If a theory has the capability to explain any possible observation, it similarly does not have the capability of being falsified...

gruseom10• 15 years ago

That seems like a degenerate trivial case. A better objection would be Occam, that a simpler theory is preferable where adequate. But even that breaks down here, because we don't have any adequate "theories", only the hard problem of how to build complex software systems.

I can see refusing an approach on the grounds that it doesn't work, but to refuse it on the grounds that it works too well?

6ren2• 15 years ago

Instead of comparing Lisp with other languages, let's consider the problem stated, of people inventing their own little worlds - i.e. DSLs.

Brooks said that a "programming product" (meaning one that can be used by other people) takes x3 the work of a "program" that works. He talks about documentation, testing, generalization and "can be run, tested, repaired by anyone". I think this means careful API design for usability, discoverability and understandability - not just efficiency - is important.

So, inventing new worlds (DSLs) is not a problem; inventing your own little worlds that are hard to understand and use is a problem. But it takes x3 as much work to do it right, and it usually isn't worth it unless it is explicitly intended to be used by others (e.g. it's a library for sale; or a utility for use within a large organization; or a web API).

Secondly, an example from the history of relational databases. Codd had the idea of relations plus a high level language. He designed a couple of high level languages, but no one liked them. Instead, Boyce and Camberlin came up with SQL (originally "SEQEL"), that was usable by mere mortals. "Since Codd was originally a mathematician (and previously worked on cellular automata), his DML proposals were rigorous and formal, but not necessarily easy for mere mortals to understand." http://webcache.googleusercontent.com/search?oe=utf-8&rl... Sometimes, designing a DSL for others is so hard that it takes a different person to do it.

This isn't specific to Lisp. It's an issue for designing DSLs and APIs (and languages in general), which can be done in any language. Lispers may invent more often and with more variation, because lisp is more powerful. Power --> Responsibility

6ren2• 15 years ago

Summary: poor abstraction is worse than no abstraction.

In Java, the "DSLs" you tend to get are at the class level, in different files. Like any abstraction, these can be well- or ill-designed. There's some differences to lisp: (1). the abstractions are less flexible/powerful, so there's less to understand; (2). having them in different files makes it harder to grasp the whole than if all in one file (or one screen); (3). the syntax is fixed, so you can at least understand the symbols without understanding anything else.

I think that inventing a new language has the best chance of making something that is a genuinely better solution. But if you want it to be understandable, it's helpful to link it to existing concepts that are already known, understood, and with known modes of use and application, perhaps via metaphor - i.e. adoption through familiarity.

But the general case for adoption seems to be that something must be x10 better (or compelling in some way) for people to go through the pain of adoption, of learning new syntax, new concepts, new ways of working, new infrastructure, new tradeoffs, new gotchas, new shortcuts, new consequences, new policies, new standards, new training, new suppliers, and so on.

If you can reduce the pain of adoption, adoption is more likely.

Put another way: there are two kinds of pain: the pain your solution addresses, and the pain your solution creates. The reason to adopt your solution is to reduce pain, but if the solution itself brings too much pain, it's simply not worth it.

Relational databases are an example of this. The relational concept solved the pain of storage change, but also created pain (difficult to use; x10-x100 slower). As those secondary pains were solved (with SQL; with optimization stategies and Moore's Law), its adoption accelerated.

So, sometimes a fundamental improvement needs to place power and flexibility over ease-of-use - but to be adopted, sufficient ease-of-use is essential. And your abstraction has to be, not just good or better, but x10 better.

j_baker5• 15 years ago

This is why Lisp is a HackerLanguage instead of a commercial language: hackers are generally loners who don't care if others can figure out their code (at least while they are in the mode or role of hacking). Thus, they build their own little world in it that fits themselves nicely so that they can hack fast, but the rest of the world be damned.

I think calling this a straw man would be generous. Hackers use lisp because they feel it makes code more easily readable. You don't have to agree with them, but at least take time to understand their arguments before you refute them.

ken• 15 years ago

Besides the superficial ("parentheses!"), I've heard two major complaints about Lisp. One is that it's just too powerful. The other is that macros don't really let you do anything you can't do with lambdas in other languages, just with (much) easier quoting.

I don't think you can have it both ways. (I'm not saying that any one person is making both of these points, but person B's anti-Lisp argument is arguing against person A's anti-Lisp point.) If you can't use Lisp in a team because somebody might write a macro that you don't understand, how can you deal with Python or Ruby or C# code that inevitably tries to fake it by taking as parameters functions-that-returns-other-functions?

(The other common alternative I see these days is to make your DSLs in XML. That means they're more verbose, you can't step through them in your debugger, and so on -- plus you still have the problem that allowing arbitrary DSLs is too powerful! I suppose there's also a third alternative: don't even try to use higher-level abstraction, and build giant apps out of low-level spaghetti code.)

I've seen perfectly readable code in Lisp, even making extensive use of macros. (There are conventions, and good programmers do follow them.) I've also seen perfectly undecipherable code in every language I've ever seen. When I've watched individuals write code in multiple languages, those that write bad Lisp tend to write bad anything. Nothing that I've seen leads me to believe that there is any significant set of programmers who can only write bad Lisp code, but can write great code in lower-level languages. We can give them Lisp, though, and they will at least write less of it!

langsamer• 15 years ago

I like how most languages now a days are trending back towards LISP-like functional languages by supporting constructs like closures, lambda expressions, etc. It's all about modularity and making the programmer most productive, which LISP seems to do quite well. I guess John McCarthy was on to something.

tete• 15 years ago

Perl has a similar (non)problem with TIMTOWTDI.

Lisp is Too Powerful

Related Stories