A man, a Plan. No canals (2011)

http://davidkendal.net/articles/2011/11/a-man-a-plan

by randallsquared • 14 years ago

81 92 14 years ago

niggler2• 14 years ago

The real problem is that many of those functional programming features touted by languages like Arc and Lisp have made their way into more mainstream languages like Python, and the types of problems that can better be solved with manipulating S-expressions are dwindling in cardinality.

stcredzero2• 14 years ago

Can you give three Examples?

chimeracoder2• 14 years ago

I disagree. Homoiconicity and the metaprogramming it allows you[0] can never be present in a non-Lisp.

[0] Deterministically, I mean - heteroiconic languages do provide for metaprogramming, but it is inherently less robust than metaprogramming in a homoiconic language: http://lists.warhead.org.uk/pipermail/iwe/2005-July/000130.h...

slurry• 14 years ago

Your comment is correct but it doesn't contradict what you were replying to. An Algoloid language with closures and lambdas and tail-call optimization etc. can't do 100% of what a Lisp can do and maybe not even 80%, but it can still do "enough" for many definitions of "enough".

draegtun2• 14 years ago

Smalltalk, Io and (possibly) Perl6 & Rebol are good examples of non-Lisp homoiconic languages which have full & robust metaprogramming.

icebraining• 14 years ago

Well, there's Prolog.

berntb• 14 years ago

I don't believe the explanation in the article.

Lisp variants can be seen as a scripting language equivalent -- with efficient compilation. It was (and still is, to a large degree) an obvious win compared to Perl/Ruby/etc.

Academic stigma and/or culture clash with scripting users is what I'd guess. Or maybe it just is lack of hype?

But I don't expect to see an explanation. It is a larger mystery than dark energy to at least me -- why didn't lisp take over the world 10-15 years ago when the scripting languages started going?!

Edit: pjmlp, I talked about the last 10-15 years and compared with scripting languages, so hardware/AI winter/IDE are earlier/later problems. (Today, a lack of libraries might be the worst problem, except for Clojure(?).)

pjmlp2• 14 years ago

- Parenthesis

- Lack of a proper IDE

- The AI Winter

- Mainstream hardware wasn't powerful enough

- Most blue collar developers don't understand FP concepts

Just some possible reasons out of my head.

andyjohnson0• 14 years ago

Interesting. I didn't realise that I'm a 'blue collar' developer. What do I need to do to upgrade to white collar?

TylerE• 14 years ago

Another I'd add is lack of a clear execution model.

There is something to be said for being able to just run 'python foo.py'

Where as in many lisp variants there is no clear entry point, and often not really an "interpreter".

wmil• 14 years ago

- The fact that most tutorials cover topics like "rolling your own object system" before topics like "writing a string to a file".

Lisp example also always cheat at things like Quines... the program will return itself instead of writing itself to stdout.

ccdan2• 14 years ago

"- Most blue collar developers don't understand FP concepts"

HAHAHA! The vast majority of FP advocates are either unemployed or work as math teachers. Like it or hate it, FP is absolutely nothing more than a pseudo-programming paradigm (largely emulating some concepts from astract math and using notation somewhat similar to math notation) that attracts people who can't wrap their heads around OOP, rich frameworks and other associated stuff. Sorry folks, computers are neither abstract nor stateless. And the same holds true for software, which often deals with real world stuff, which again, is neither abstract nor stateless. Virtually everything that can be done in a functional language, can also be done in a procedural or OOP language. The opposite on the other hand is totally untrue. It's really funny to see how FP advocates struggle even with some extremely basic things. Using languages/platforms like C/C++, Java,.Net - there's always an increase in performance compared to any functional language (yeah, including scala, f#, clojure ocaml an so on) The "elegant code" argument is one of the most ridiculous things FP advocated come with, since it's almost always synonymous with crappy, cryptic code that no one wants to read besides its authors (maybe not even them after a few weeks or months) :D So I'm afraid that the FP advocates are far worse than real blue collar workers.

fusiongyro2• 14 years ago

Without paying too much attention to the essay's content, the simplest explanation is that Common Lisp, Scheme and Clojure are more compelling.

Lispers are always tempted to blame Lisp's niche status on being too advanced, but it always sounds a bit like a smug non-answer to the job interview classic "what's your biggest weakness?" One friend who likes the syntax plus the existence of reader macros does not prove the syntax is not a big issue.

That said I think his conclusion is pretty sound, though you'd be hard-pressed to find much evidence of Unix-friendliness in C# or Java.

chalst• 14 years ago

Maybe "friendly to a sufficiently powerful runtime" should replace "Unix-friendliness"?

fusiongyro2• 14 years ago

That's good. I mean, people complained about using Git in Windows for ages because it was too Unix-friendly. At the same time, almost nobody actually likes SML/NJ, and I think part of that is because it has its own strange build system rather than autoconf/make/cmake etc. and until a few years ago really couldn't produce standalone executables.

jherdman• 14 years ago

Totally off topic, but if the author of the post is reading this, please reconsider the fonts you've used on your blog. It's quite difficult to read as it. Compare and contrast your current design with this http://www.readability.com/articles/4xlr1wov.

leephillips2• 14 years ago

While I didn't think much of the article, I found the font to be attractive and easy to read.

mark-r• 14 years ago

I wonder if you were seeing the first choice "Mrs Eaves" or the backup Georgia? Or perhaps you're on a high resolution display? The combination of font and size does seem a bit extreme to me.

jgrahamc2• 14 years ago

The discussion of PG's decision to use closures for web development seems off topic. That doesn't directly affect the popularity of the language; it's quite possible to do things in Arc without using closures for state on the web.

I played around with Arc quite a bit and the biggest problem was that it did not have a standard library with enough stuff in it. In contrast (and talking about a totally different type of language) one of the reasons Go has been successful is the excellent set of standard packages.

If you start your language with sufficient standard library kindling then others can build on that and write more and more libraries. But start with insufficient libraries and it's hard to take off.

So, the initial releases of Arc were constrained by things that PG had already written. If you wanted to do the same then it was just fine.

As as concrete example I wanted on my old UseTheSource site (which was in Arc) to go grab a URL from a third-party site inside the code. There wasn't an easy way to do that so I wrote an external script in Perl and did the following in Arc:

  (def hn-valid (user)
    (if (uvar user hn)
          t
          (let v (system (+ "perl check-hn.pl " user))
                  (if v (= ((profile user) 'hn) t))
                  v)))

The Perl script grabbed a page on Hacker News, did some light parsing and returned either 0 or 1. That really put me off making any extra effort to use Arc (and I like Lispy languages) because I was either going to shell out to something else, or have to write everything from the ground up.

A similar thing happened with UseTheSource's Twitter integration where I could use Perl's Net::Twitter::Lite to trivially integrate.

canttestthis• 14 years ago

If a language's popularity was closely dependent on library support, then (in your opinion) why hasn't Clojure gained mass adoption?

sigzero• 14 years ago

I would say "can be closely dependent". And for a LISP, it is doing pretty darn well.

jgrahamc2• 14 years ago

My comment boils down to the logical statement

  NOT has large initial library => NOT gain widespread use

You seem to be making a logical error based on a common problem with people's understanding of modus tollens: http://blog.jgc.org/2009/05/frequently-misunderstood-logic-m...

It is NOT the case that from my statement that it's possible to derive:

  has large initial library => gain widespread use

jimbokun• 14 years ago

Relative to Arc, it certainly has.

aidenn0• 14 years ago

It's easily the most successful lisp in over 20 years. How popular do you want it to be?

stesch• 14 years ago

People who would like to use Clojure or Scale aren't fans of Java. And if you aren't a fan of Java, chances are good that you aren't a fan of the JVM, too.

I'm already using something else. Why should I use the Java eco-system?

jasim• 14 years ago

Clojure IMO is gaining great mindshare as a practical Lisp, especially given how recent it is. I don't think anybody expects it to become as mainstream as Java. But it does attract programmers unfamiliar with FP, but whose appetite for languages higher in the power continuum has been whetted by languages like Ruby.

My colleague Steve presented a talk at RubyConf India titled 'Why Clojure is my favourite Ruby' which might appeal to such people. http://www.youtube.com/watch?v=PCdEbUBk6a0

abqarc• 14 years ago

I bypassed the library problem by calling Python from Arc and returning the results as s-exps. It took 40 lines of Python code.

scott_s• 14 years ago

I find it strange to read an essay on the (un)popularity of Lisp which does not mention Clojure.

kanaka2• 14 years ago

The essay isn't actually that recent.

rodrigoavie• 14 years ago

Are we in some king of Lisp day or something? Hahaha

retube• 14 years ago

Altho in reality it probably has no bearing, but most people's casual exposure to Arc is the HN website - which at the best of times is slow. So the mindshare being developed here is "arcane AND slow" - not a great advert.

michaelochurch2• 14 years ago

Arc is beautiful, but Paul Graham was wrong about one thing. For language adoption-- at least circa 2013 (which he couldn't have foreseen)-- libraries do matter. This is to Clojure's advantage and Arc's detriment. A typical IT boss is not going to let you spend 6 months writing libraries, so a lack of libraries is a problem. The people who pick languages in all but the most daring startups tend to see "lack of libraries" as more of a problem than it actually is.

Paul Graham was building a web company in the mid-1990s, so rewriting a bunch of libraries was not an impediment. First, they had top-notch programmers who were willing to work with FFIs, write tools they were used to in other languages, and understand technology deeply enough to write good libraries. Second, the state-of-the-art for libraries in the 1990s was, if nothing else, less Big. Third, they didn't need to sell their language choice to a boss; they just needed to sell the product to the market and investors, neither of which cares what language you use as long as it works.

Arc is prettier than Clojure but, in my experience, Clojure is more than attractive enough... and it has the familiar JVM libraries.

PaulHoule• 14 years ago

A big reason why PHP was as successful as it was is that it came with a fairly complete and correct standard library for writing webapps.

This way you could spend zero time writing your own urlencode function (and screw up a corner case) or searching through maven to find functions to do very simple things and finding later other people on your team imported similar functions from four other projects.

guard-of-terra• 14 years ago

"with a fairly complete and correct library for writing webapps"

Some of which were dangerous and unwieldy actually. Then proven dangerous and unwieldy. And then most of things that made idiomatic PHP were removed from the language or discouraged and nobody writes that way anymore.

peatmoss• 14 years ago

And of course Clojure addresses another issue raised in this article of regex as a first-class citizen. I'm pretty new to Clojure, but that has already given me a few "ah" moments.

emmett• 14 years ago

Where on earth did you get the idea PG thinks libraries don't matter? I don't think I've ever seen PG say anything like that. Here's where, in fact, he says the exact opposite:

Of course the ultimate in brevity is to have the program already written for you, and merely to call it. And this brings us to what I think will be an increasingly important feature of programming languages: library functions.

http://www.paulgraham.com/popular.html

In fact, read the whole section 6, on Libraries.

michaelochurch2• 14 years ago

Great link. I remember him saying something to the effect of libraries not mattering in his decision to use Common Lisp for Viaweb, but that's not the same thing as him saying that they're unimportant for language adoption. So I guess my memory is inaccurate.

Where there is disagreement is that most CTO's are afraid to adopt "weird" languages out of library FUD, which PG asserts doesn't matter. On that, I think PG is right.

qznc• 14 years ago

I wonder if the embedded-language angle of attack is more promising today. Lua and Javascript did not need bug libraries. They are embedded into larger programs and invoked useful functionality there instead. Nevertheless, in both communities people try to write standalone programs nowadays and standard libraries will emerge at some point.

justincormack• 14 years ago

Lua more so I think as it has easy C interfaces built in.

jiggy2011• 14 years ago

Well , there's a few things.

Not all "hackers" are language geeks, plenty of smart people will not necessarily invest their time in learning new exotic programming languages as language learning for the sake of it is not interesting for everyone.

Other considerations for language choice will certainly include documentation and the accessibility of that documentation. Looking up a few pieces of code to do familiar tasks is much easier than reading a grand language design document as a way to get a feel for a language.

Having usage in some large commercial setting is also a consideration. For example knowing that Google uses python extensively can give one some confidence that python is unlikely to suddenly die out one day because Google wouldn't let that happen.

lobster_johnson• 14 years ago

Languages need momentum to gain foothold. It needs a community of dynamic, experimental, evangelistic pioneers to gather around your language and start building.

When I go to the Arc site [http://arclanguage.org/], it seems pretty obvious to me why Arc has not "taken off": A drab HTML 2.0 site with a tiny font describing how it's unfinished, and the only way to install is through another Lisp version, and no indication that there is a community of developers behind it.

It's almost like they don't want anyone to use it. To be fair, it seems Paul Graham does not care about popularity. But not caring about popularity means it's dead in the water.

It's not like lots of people don't want a Lisp these days. Clojure has become very popular, after all; it has been able to hit the sweet spot in terms of modernity and lispiness; good, practical technology with a solid community.

sramsay2• 14 years ago

I really think the biggest problem that new languages have is contending with the colossal hegemony of three stacks: C, Java, and Javascript.

There are a few exceptions, but for the most part, your language is going to be successful to the degree that it can interoperate with these environments. Common Lisp, Racket, Haskell, OCaml -- all of these languages have ffis, but they're "begrudging" ffis. They don't really care about your "legacy" systems or the titanic number of C and Java libraries that already exist.

Clojure and Scala do care about that, as does Lua. And Ruby and Python (and Perl) care as well.

I see beautiful new languages every day. I can tell whether they're going to live or die primarily on the basis of their attitude toward the dominant library environments in modern programming.

kanaka2• 14 years ago

And Clojure can target and interoperate with at least two of those stacks: Java (standard Clojure), Javascript (ClojureScript). And there is experimental support for C/C++ (clojurec, Ferret, etc). Not to mention in progress support for targeting CLR/C# (ClojureCLR) and python (clojure-py).

sramsay2• 14 years ago

Right. Clojure isn't the first Lisp for the JVM, but it's definitely the first one to care so deeply about other languages and existing libraries.

If you ask about C interop or creating standalone binaries on a Common Lisp board, you'll get an answer (because you can do this). But you'll also get this whole, "Oh, but why would you want to infect our beautiful language/runtime with the fallen world of imperative code, UNIX conventions, etc. Free your mind!"

That. That right there. That's the problem.

bitcartel• 14 years ago

Hands up if you saw the headline and thought about Objective-C Automatic Reference Counting...

stcredzero2• 14 years ago

ARC vs. Arc. (Which is why it should be Lua and not LUA)

shaunxcode• 14 years ago

I think (and this is total subjective speculation) one of the problems is that Arc is so beautiful in its simplicity that the sort of people attracted to it think "man that looks like fun to implement, I think I will work on a lua/js/python/jvm port! then it will REALLY take off" instead of building things (apps, tools, libraries) WITH it.

michaelfeathers• 14 years ago

I think that when you design a language you can take the low road or the high road. The low road is to design your language to explicitly interop with an existing language/platform so that you can hijack its libraries.

C++ took the low road by co-opting C. Scala and Clojure took the low road in co-opting Java. Languages like Arc, Smalltalk, Haskell and Eiffel took the high road and as a result they don't have as many users.

Having said that, I don't want to imply that language designers are doing a bad thing when they take the "low road." They get adopters and they bring powerful tools into the world. It's just that when they do that they have to sacrifice coherence a bit.

DigitalJack• 14 years ago

If you didn't want to imply something bad by your analogy, you should have chosen a different one. The analogy of the High Road and the Low Road has implicit that the High Road is the good path and the Low Road is the bad.

orangethirty3• 14 years ago

It hasn't taken off because it has not been marketed. Languages are products which need to be marketed like everything else. If PG wanted it to take off, then he could market it without much problem. But he has better things to do.

npsimons• 14 years ago

This, sadly, may be true. I remember reading PG's essays, thinking, "huh, I'll have to look into arc" then never hearing about it again. It's not packaged for Debian stable (one of my litmus tests), and just about every week I'm hearing about clojure, go, scala, etc, etc, but nothing about arc.

MrMan• 14 years ago

arc would not take off even if it were more-aggressively marketed, IMHO. it has too few differentiating features compared to other lisp dialects. 'I made a LISP' is not enough, even if it is a nice elegant one.

orangethirty3• 14 years ago

You are right. Though Javascript with its weirdness took off...

huherto• 14 years ago

Curiously, this is one the things he disliked about Java. http://www.paulgraham.com/javacover.html

jsnell2• 14 years ago

This article is starting from completely the wrong point. The default state of a programming language is somewhere between "dead" and "niche". You wouldn't expect a language to take off unless it's got something special going for it. And Arc had very little. Basically lots of hype built around a cult of personality.

And in the other column, the Arc had significant things going against it. It was a toy implementation by somebody who really had more important things to do with his time. It didn't bring anything substantial to the table -- to a first approximation it felt like a basic Scheme with a idiosyncratic function naming scheme. It didn't have any killer features. It didn't have a significant standard library. Compare that starting point to e.g. Clojure or Go. It's like night and day.

The question isn't why Arc didn't take off. It's why anyone would expect it to take off in the first place.

pg5• 14 years ago

You're being a bit uncharitable. Even though I haven't spent a huge amount of time on Arc, there's more to it than just hype. I know Common Lisp and Scheme really well, and I would much rather program in Arc than either of those. Now that I'm used to Arc, I find writing code in CL or Scheme painfully clumsy.

And though language design (as opposed to implementation) may not seem "substantial," it does matter. Scheme itself was initially an exercise in language design-- a cleaning up of MacLisp. It didn't only become "substantial" when people started writing complicated compilers for it.

jsnell2• 14 years ago

I don't think the convenience of Arc is a very widely held view among Lisp users. But that's setting also setting the bar way too low. If Scheme or CL didn't exist, and somebody tried to launch them as new languages today with a first generation implementation, they wouldn't stand a snowball's chance in hell. Most probably CL or Scheme as new languages wouldn't get any traction even with a mature implementation + good library coverage right at the start. It's much easier for a language community to be self-perpetuating at a certain size than to actually grow to that size.

I agree that Scheme didn't only become substantial once the implementations became mature. But that's because it had a few fresh big ideas (lexical scoping/closures, CPS), not because it was new. Likewise it didn't become popular (as much as it ever became popular) just due to existing, but largely thanks to being used as the teaching language for a legendary intro course at a prestigious university.

As for the "hype" comment, that wasn't intended as a negative attribute. It's very hard for a language to succeed purely on its own merits. Tens of man years of effort had gone into Go before it was launched. It might still have failed had there not been such powerful marketing hooks in place (created by Bell Labs legends, having the appearance of being backed by Google). Arc had a good marketing hook as well. That would have made bootstrapping an initial community much easier than e.g. for Clojure.

And just to be clear this isn't intended as a personal criticism! There's just one YC, but hundreds of new languages and implementations appear every year, slowly advancing the state of the art.

pg5• 14 years ago

What a complicated answer to a question with a simple one. I spend all my time on YC now. The last release of Arc was years ago. So this case doesn't prove anything one way or the other about programming languages.

orangethirty3• 14 years ago

Do you ever stop and think about working on it? After all, a language is a very personal project.

akkartik2• 14 years ago

At the very least his point about continuation-based webservers is a good one.

I recently realized why the arc webserver needed to track IPs to ignore almost from day 1: everytime somebody requests a page, memory usage goes up. This includes crawlers, drive-by users, everybody. The standard way arc encourages webapps to be programmed (http://paulgraham.com/arcchallenge.html) is by leaking memory. Every load of the front-page creates 30 closures just for the 'flag' links. Ignoring IPs is just a band-aid to cover up this huge sink[1].

I've been programming in arc for 3 years and am still active on the arc forum. I love that arc is a small and simple language without lots of bells and whistles[2]. I really couldn't care less that it's 'not taken off'. But there's a different between toys that encourage exploratory play, and painting yourself into a corner design-wise. I now think of continuation-based webservers as an evolutionary dead end.

[1] Another ill-effect of the ip-ignoring is that every newcomer to arc who tries to run it behind a proxy server immediately runs into a gotcha: The arc webserver now thinks all requests are coming from the same IP address and so throttles everybody. http://www.arclanguage.org/item?id=11199

[2] If you spend any time with it it literally begs to be tinkered with. And the experience of programming in a language while referring to its compiler in another window is something everybody should experience.

mcartyem• 14 years ago

His point on continuations is invalid.

"The problem: because closures can’t be stored in databases, you really have to use a hash table on your web daemon."

This is not true. Closures can be stored anywhere you want.

pg5• 14 years ago

I'm not sure what you mean by continuation-based. The Arc server can use closures to store state, but it doesn't use continuations.

Using closures to store state is like using lists as data structures: it's a rapid prototyping technique. You're in no way painting yourself into a corner, and the current news software is proof of that. You have a very old version of it. In the years since we released the version you have, we've gradually squeezed most of the closures out of the more common things, and we do have a proxy in front of the Arc server. That's how we manage to serve 1.7 million pages a day off one core.

whyme2• 14 years ago

Would it be fair to say that if Arc had more success when you first released it that you would have been less willing to let the effort die? If so then the full analysis could still have merit.

pg5• 14 years ago

In retrospect YC would have taken over my life no matter what. It has pushed out essays too, mostly.

I don't consider Arc to have died, incidentally. You used it to say that, and I'm using it to reply. If I ever retired from YC I'd probably start working on it more actively again.

tomcam• 14 years ago

Agree with the conclusion, but some of the reasoning is unsound. This statement is patently untrue: "This doesn’t mean you can attract smart hackers to your language just by running an advertising campaign though. Sun tried that with Java, but a: their marketing was too explicit, and it just seemed like one company trying to push its own technology, and b: Java itself is a detestable language. So, in addition, it seems like language quality is not something you can make up for in marketing."

It implies that Java failed in the marketplace. It did not. It has taken and kept huge market share. Sadly, it is the language most employers are hiring for. I would rather poke myself in the eye with a pencil than program in Java, but it has most certainly succeeded in the enterprise.

johnx123-up• 14 years ago

FWIW, the modified title "Why Arc hasn’t taken off" was more informative than the original.

A man, a Plan. No canals (2011)

Related Stories