Clojure from a Schemer's perspective (2021)

https://www.more-magic.net/posts/thoughts-on-clojure.html

by lispybanana • 4 years ago

168 66 4 years ago

lionsdan• 4 years ago

Previous discussion https://news.ycombinator.com/item?id=26356367

kccqzy• 4 years ago

A quick look at the code implies that various data structures are used under the hood for what looks like one data structure in the language. That's a lot of complexity! I'm not sure that's a tradeoff I'd be happy to make. It makes it harder to reason about performance.

I don't think that's true. A persistent array map is a single data structure. It's just a persistent data structure optimized for non-destructive (functional) updates. On a high-level it's not too different from finger trees which may be more famous. When I used to write Clojure, I didn't feel any need to reason about its internals for performance reasons or any other reasons. It simply feels natural.

lgessler• 4 years ago

I think the author's referring to the multiple implementations of persistent maps (i.e. APersistentMap). Clojure's generated docs[1] say there are four implementations (PersistentArrayMap, PersistentHashMap, PersistentStructMap, PersistentTreeMap), and this Stack Overflow answer[2] explains that the latter two are rarely used, and the former two make up the bulk of actual APersistentMap instances, with the language's core functions for persistent maps changing implementation used under the hood as it likes in order to do what it thinks is best for performance (basically, using PersistentArrayMap if the number of keys is small, and using PersistentHashMap otherwise). You can see this, for example, in PersistentArrayMap's implementation of assoc[3], which replaces itself with a PersistentHashMap if the number of keys has grown over the hardcoded threshold of 16.

So like you say, your experience as an "end user" makes it seem like it's just one data structure at least in interface, though in implementation it's actually doing some performance optimization stuff which, I think I'm willing to grudgingly grant to the author, does make performance harder to reason about in principle. Though in my (very unserious) experience, I think I've enjoyed languages that take a possibly incomplete stab at getting performance right over ones that draw a line leaving it entirely out of scope and declare victory.

I'll equivocate a little and share that I have been bitten by these internals before, though the fault was mine: one time, I was unwittingly relying on an implementation detail of PersistentArrayMaps, which is that they preserve insertion order. PersistentHashMaps don't, and my code was mysteriously breaking when number of keys grew beyond 16 until several hours later I learned much of what I've just shared above.

[1] https://www.javadoc.io/static/org.clojure/clojure/1.10.1/clo...

[2] https://stackoverflow.com/questions/16516176/what-is-the-dif...

[3] https://github.com/clojure/clojure/blob/5ffe3833508495ca7c63...

eru• 4 years ago

CPython also switches out what internal implementation it uses to represent your dicts based on a few heuristics.

chii• 4 years ago

does make performance harder to reason about in principle.

the thing is, you don't reason about performance. you measure it.

Once you've obtained measurements, you can make decisions based on the outcome of said measurements.

If the Runtime (and not just talking about clojure here) makes a decision on performance, and it's suboptimal, your measurements _should_ reveal it - for example, by increasing your input size, the performance graph should show a clear inflexion of where this happens.

The real question is whether you _can_ force the Runtime to do something at the programmer's behest, rather than a hidden decision that you cannot control or change. I believe in clojure, you can change the backing data structure, but i'm not sure if you could do it after the fact?

rippeltippel• 4 years ago

Isn't it similar to what happens in other languages?

In example, in C++ std::maps are implemented as RB trees, are::vectors instantiate 50% (sometimes 100%) more memory than requested, etc. All of that happens under the bonnet in order to fulfill the performance guarantees of the language.

IIRC, one of the most widely quoted memes in Clojure space goes along the lines of "simple != easy". Or better (not= :simple :easy)

pjmlp3• 4 years ago

Traditionally most C++ implementations use RB trees, the standard doesn't require it to be so, all data structures requirements are described via big O notation.

pluijzer• 4 years ago

Lisp programmers know the value of everything and the cost of nothing.

hxegon• 4 years ago

The actual details of how the underlying datastructures work isn't that hard to grasp (see: https://youtu.be/wASCH_gPnDw?t=1817), but in practice you don't even need to go that far.

At a high level there's really just a few things you need to keep in mind, the biggest being the big O guarantees of the core DS interfaces, which are listed here https://clojure.org/reference/data_structures. Those guarantees hold across the different underlying implementations, and there are plenty of things you can tune to get a 2x, 10x, 50x improvement in performance before resorting to things that depend on which specific DS is being used under the hood

Edit: Fixed timestamp for video link

pmoriarty• 4 years ago

One of the main reasons I don't even want to touch Clojure is because I don't want to be trying to figure out Java stacktraces when things go wrong.

fuzzythinker• 4 years ago

Clojure doesn't need JVM. You can run it with GraalVM either directly [1] or via Babashka [2]. Both are great options for get into Clojure.

[1] https://github.com/oracle/graal

[2] https://github.com/babashka/babashka

selimco• 4 years ago

Technically, graal is also a JVM. Do you mean native image by any chance?

kaba02• 4 years ago

What’s bad about Java stacktraces?

vaylian• 4 years ago

The problem is that the stacktrace shows you all the gory JVM details that are irrelevant to your Clojure code. Even when your code is 100% Clojure, the Java classes, that are the foundation for Clojure, are shining through when you get the stacktrace.

It's been a while since I last used Clojure, so I don't know if the situation has improved in the meantime.

vkazanov4• 4 years ago

strict answer: they are very deep.

informal coffee break answer: abstracted layers of abstracted abstraction within parametrized abstractions just don't feel right to lispers.

AtNightWeCode• 4 years ago

The page count

wiseowise• 4 years ago

It’s quite silly to complain about such minor “issue”, don’t you think? Compared to what being hosted language gives you.

Bost4• 4 years ago

I'm sorry but this as an argument is weak. Yes, it is true, the stacktraces are terrible, bordering on useless noise, but that doesn't mean that "exceptions & stacktraces" is the only possible approach when it comes to error handling.

kubb2• 4 years ago

If this is one of the main reasons, then the case against Closure that you're making doesn't sound very strong.

lgas• 4 years ago

I don't think the person you're responding to is attempting to make a case against Clojure in general, just saying why it's not something they are personally interested in it.

didibus6• 4 years ago

Ya, if you've never programmed in Java, Kotlin or Scala on the JVM before, this is definitely an additional learning that all JVM languages have.

But if you're already familiar with any of those, it'll be the same as what you're used too.

roguas• 4 years ago

To a degree. Interop is useful, but you can getby with most of things without it. Long term you will learn interop as it's often times easier to write small piece of code to take java lib, rather than trusting in community wrappers.

rawoke0836002• 4 years ago

Recently I tried to jump into scheme (I'm okish in clojure) man what a mission to install scheme on linux.

I first tried todo gerbil scheme, then Ubuntu seems to have the wrong version of gambit (gerbil-scheme runs optop of gambit-scheme for those that don't know). That took a few hours of compiling, reading, trying diff versions always ended in some deadlock-issue ??

Then I tried Guile,was a little better and easier, but soon got stumped when I wanted to access mysql-db yes there are packages but those packages I couldn't install for the life of me !

In the end chicken-scheme was surprisingly easy and complete solution for a hit-the-ground-running scheme.

Now I'm prob a noob as it comes to scheme, but not to fiddle and massage packages and programs on linux. Trying to learn scheme but spending a few hours just to get it installed was what killed it for me.

Clojure is certainly not perfect, but you could get up and running much faster in my opinion. Oh and it uses square brackets for parameters :)

pjmlp3• 4 years ago

Next time just install Chez Scheme or Raket, they work perfectly fine straight out Ubuntu repos.

medo-bear• 4 years ago

besides having the best mascot, just curious why you would start with gebril? esp for a newbie. surprised you didnt just pick racket

not an expert but i believe you should use guix for anything guile related

Oh and it uses square brackets for parameters :)

for many lispers this is a negative because they see as a big positive the manimal (no) syntax value proposition of lisp. besides if you use emacs you can easily (if you are a lisper) make it do something like having parameters highlighted without introducing additional syntax in the underlying language

bakul• 4 years ago

Gambit scheme also just works. That's what I use on FreeBSD, MacOS (m1 and x86-64), Ubuntu and Raspbian. Had a version running under plan9 as well but that was a long time ago.

vkazanov4• 4 years ago

One (non?) problem with Scheme is that it is has a long history of being a language implementation research device. This is why it is relatively easy to implement, and this is why there are so many implementations.

At least it didn't go the way beautiful Standard ML went!

bmitc2• 4 years ago

At least it didn't go the way beautiful Standard ML went!

It blows my mind how under appreciated MLs are.

dig1• 4 years ago

Not sure why you went with Gerbil/Gambit - Gambit is usually marked as a hard-core Scheme, and you usually go with it if you need an excellent optimizing Scheme compiler without too much dance around it (it has a minimal interface). For Guile, you usually rely on OS package management to get modules or go with Guix.

But if you want a "download, unpack it, run" Scheme, you go with Racket [1]. Tons of documentation, a good-enough JIT compiler, and usually everything works. You get an amazing IDE with it (DrRacket) with many advanced features yet to be found in modern IDEs for popular languages.

Oh and it uses square brackets for parameters

Most modern Scheme implementations make square brackets and parenthesis equivalent. E.g.

   (let ((a 1)
         (b 2)) ...)

   ;; is the same as
   (let ([a 1]
         [b 2]) ...)

[1] https://racket-lang.org/

bmitc2• 4 years ago

Installing Clojure, because of its dependence on Java and plethora of IDEs, is a major pain in my experience. I have tried several times and usually just give up because I find it difficult to get up and running with a simple project that allows me to write code. I don’t understand the seemingly complicated project systems.

It takes literally just a few minutes to download, install, and get going with Racket, Elixir, and F#. It’s the main reason I have not got into Clojure yet, despite trying.

salad-fan• 4 years ago

Yep your experience is (sadly) not even remotely unusual, and Cognitect have repeatedly demonstrated that they don’t care about this issue (which will come back to bite them as the community stagnates).

The “best” advice I have for Clojure beginners is to follow this guide: https://calva.io/get-started-with-clojure/, which will ultimately land you in a solid VSCode-based IDE environment for Clojure.

That’s not how I personally like to approach a new language mind you (REPL from the command line plz), but I’ve pretty much given up trying to get Clojure beginners started there as there are just too many moving parts that can go wrong, and unjustifiable frictions.

likeclockwork• 4 years ago

pacman -S clojure ?

Just install it with your distro's package manager.

I have no idea why the official instructions only involve brew (wtf?) but it's in the repo of most distros now.

germandiago2• 4 years ago

Try this: https://janet-lang.org/

didibus6• 4 years ago

I think it's just bad online tutorials and guides. It's not any more complicated in practice than any other language I've tried, maybe with the caveat that getting a REPL running and connected is more work than an only static analysis based IDE.

mhitza• 4 years ago

If you've went with chicken scheme, I recommend installing it from source code (has very few dependencies, so it isn't a time wasting process). The reason being that with the version that you might get from the OS package manager static libraries aren't shipped (at least that was the case in Fedora) and you won't be able to build statically linked executables.

The other upside of using chicken is that their FFI interop with C is really streamlined. From one of my projects; in order to expose the `getpass` C stdlib function in my program I only had to write the following line

    (bind "char* getpass(const char *prompt);")

weatherlight4• 4 years ago

Clojure is a nice well thought out language, I wish it wasn't on the JVM.

snikeris• 4 years ago

Curious when you formed your opinion on the JVM. It's been under active development for 28 years.

Steltek• 4 years ago

The JVM ecosystem pioneered a lot of what I see as awful anti-patterns in language development:

- Language-exclusive build chain: Not that Make is the be-all and end-all of build tools but it worked with everything without prejudice. To understand the basics of a build system, you only needed to know one tool. All attempts at fixing Make's flaws fail to understand this fundamental recipe for success. Now when you switch languages, you need to relearn that layer on top of the language itself. And the tools all do the same thing but with their own syntax, commands, and eccentricities.

- Language-exclusive packages and repositories. Once upon a time, you learned your OS's package manager and you were done. Now you need to learn a new package system for every language out there. Again with their own syntax, commands, versioning, and eccentricities.

- Needing language-aware parsers to determine dependencies. You can't depend on spatial locality or compiler-output dependency trees to know whether to rebuild, reinstall, etc. You had to parse out the org.whatever's and resolve that back to a jar or class, which was clumsy and mostly wrong.

- The OS doesn't know how to run your program. What's so hard about building your executable so it can be natively run by the OS? Everyone has to write the same `#!/bin/sh exec java -jar ...` wrapper script. Java programs worked best by assuming it was going to be a pariah hanging out in /opt and never actually integrate with anything.

- Runtime versioning and installation was painful. I disliked having to suddenly manage environment variables just to run something. It continues today with python2 vs python3, GOPATH, etc. On top of that, JRE's weren't a simple "curl|bash" and no one was allowed to repackage the JRE to make it more user friendly.

cube2222• 4 years ago

Well, you can run it on Node.js, too.

didibus6• 4 years ago

But then you get the terrible JavaScript stacktraces !

markc• 4 years ago

Well it’s not only on the JVM: https://en.m.wikipedia.org/wiki/Clojure#Other_Implementation...

weatherlight4• 4 years ago

Sure, and thats good thing :)

wellpast• 4 years ago

Clojure marries the elegance of Lisp and functional programming with an extreme practicality. The practicality of it is precisely _why_ it is hosted on the JVM ecosystem.

There is this very common reaction to Clojure (also reflected throughout the original post): a balking at its pragmatic decisions. But Clojure's pragmatism is precisely what makes so significant and wonderful.

weatherlight4• 4 years ago

My issue isn't that. its that every Clojure project I've seen has always been at Java Shops, where eventually some java dev decides it too hard/ doesn't have the time to learn Clojure, and it just gets rewritten in Java. I've seen this happen with a few Scala projects as well.

Clojure from a Schemer's perspective (2021)

Related Stories