REPL vs CLI: IDE wars

https://vlaaad.github.io/clj-vs-cli

by vlaaad • 5 years ago

147 105 5 years ago

vlaaad.github.io

tomconnors• 5 years ago

This post could be summarized as "write as much of your project's tooling as you can in the project's main programming language and the project's main programming language should be Clojure" and I agree wholeheartedly.

blacktriangle3• 5 years ago

I think their point is bigger than that. Historically one of the major points against using Clojure to write your tooling was the slow startup times which are just painful from the CLI. It looks like the clj-exec idea linked to in the article is the secret sauce that makes moving to writing your tooling in Clojure a workable idea, since now you have a unified calling convention for both calling tooling from the REPL during development but also activating your tooling from the CLI in some CI or build pipeline where the massive Clojure startup times don't matter.

This article comes at a perfect time, we're just starting a new Clojure project and were looking into how to automate tooling since we'd been burned by Clojure startup times before. Looks like clj-exec means we can now unify our work on Clojure.

Borkdude• 5 years ago

Also check out https://babashka.org: it offers Clojure scripting with very fast startup time. It also has a task runner (similar to make, just, etc.) that can be used to store long invocations (like clj-exec tends to have).

utxaa12• 5 years ago

what about the horrible debugging experience?

kinjba11• 5 years ago

Does GraalVM work with Clojure? I would assume that would solve the startup time issue for a project like a CLI tool.

toomanybeersies• 5 years ago

The general gist of the article also applies to Ruby too.

smw• 5 years ago

I think you might be missing the main "aha!" of clojure REPLs vs REPLs in non-homoiconic languages: you can very easily execute small parts of the program you're editing without re-typing the code.

In emacs, for instance, you often use `eval-last-sexp`, by default bound to C-x C-e. This lets you move your cursor to a particular point in the file, often deep in a function, and get the results of just the form(s) you're pointing at.

This is a superpower! It lets you test small pieces of your code without writing any test harnesses or scaffolding. Try it with an editor that embeds the repl and has these commands, and you'll never want to develop any other way.

It does cause you to want to structure your code in 'repl-friendly' ways, so that you can, say, restart your main server or reset your state without restarting the whole process.

lennoff• 5 years ago

When you evaluate a form deep in a function, how do you handle variables that are defined "outside"? How can you evaluate such expressions?

synthc• 5 years ago

Cider (the clojure plugin for emacs) has a variant that prompts you for the the values for those variables.

weavie3• 5 years ago

You can essentially give values to global variables with same name as those outside variables.

Graziano_M• 5 years ago

The repl maintains an environment.

dgb233• 5 years ago

You can use a REPL powered debugger. You can interact with the debugger to a higher degree than with most mainstream languages. It’s not quite as powerful as CL though, from what I’ve read.

Or you pull out expressions and test them in isolation with given (assumed) inputs.

Scarbutt• 5 years ago

By evaluating outer forms first.

slifin• 5 years ago

Cursive has a repl powered debugger just put your breakpoint there use the repl to trigger the call then use intellji's debugger to step through

If you want to run expressions in context just use the expression editor like a normal repl

Something I haven't tried yet is redefining functions at debug time can't see why it wouldn't work though

notriskfree2• 5 years ago

I suspect you are right. Rebinding an inner function inside another function that contains variables probably will not work. But you could still alter and rebind the entire thing. And that is still flexible; you can still change a running program on the fly with only that ability. And you can always reference objects at the top level.

dragonwriter• 5 years ago

Homoiconicity isn't required for that, all you need is to be able to map between source and AST enough to identify expression boundaries.

Plenty of development environments for non-homoiconic languages provide the ability to evaluate a selected expression (not just the last one) in a linked REPL on demand.

I love the Lisp family, but I don't know why its advocates often sound like they haven't seen a dev environment for a non-Lisp language since the early 1980s when pointing to “unique” advantages of Lisps.

marcosdumay2• 5 years ago

Homoiconicity is an ill-defined concept. If you take every subjective feature from it, all it implies is that you are able to cut your code into well defined AST structures (what all languages have, but for some it's easier than for others).

So, yeah, anything people do with Lisp could be done for any other language, it only requires more complex tooling. How much more complexity depending on the language, and it varies from "barely perceptibly more" into "that's completely not practical".

leephillips3• 5 years ago

Indeed; I do this every day with Vim and its term command, with Julia, Python, bash, Elixir—it works with any language with a REPL of any kind.

dgb233• 5 years ago

There is a significant, qualitative difference between using a language that designed for REPL use and one that isn’t.

Those boundaries you talk of are the crux of the issue. A highly dynamic, completely expression based language is going to enable a much different experience. Homoiconicity also plays an important role here, because you can ispect and parse code within the language, with the same functions and algorithms as everything else.

elamje• 5 years ago

Maybe it's not required. However, I can confidently say Clojure in emacs feels like an entirely different development experience than my daily Python work in PyCharm/VSCode and C# in VS. I recommend trying it out if you haven't already.

pjmlp6• 5 years ago

Not only that, Smalltalk, Mesa and Mesa/Cedar workstations, all shared the same interactivity as Interlisp-D at Xerox PARC.

afiori• 5 years ago

a comment here some time ago explained nicely what most of these "unique" advantages of homoicinicity are actually about; it was essentially another step on the same spectrum of statement-languages and expression-languages.

in particular this does not mean that it is better but the same advantages of being able to put anything into anything that you find in expression languages like rust in lispy languages are also syntactically in editing source code.

since I started using shrink/grow selection in vscode I have much increased interest for simple syntax aware editing, something simple would be moving blocks of code token by token

tomrod• 5 years ago

This is the draw for Jupyter Notebooks as well. Execute code blocks. I remember when I learned this in Matlab.

REPL are super nice for this!

I've had a hard time in Java, Fortran, etc. in determining the right development patterns that are used in this space to be productive.

bookofsand2• 5 years ago

Step1: Write a test.

Step2, IDE: Right click + Run.

Step2, CLI: $ run <test name>

Bonus: The test is now part of your automated test suite, and will be ran many times to ensure things don't break.

cbm-vic-20• 5 years ago

Modern Java (11+) has jshell, which has interactive goodies like tab completion, documentation display, paren/bracket matching, etc.

winrid3• 5 years ago

You can do this in IDEA's IDEs too, just place a breakpoint and then you can "evaluate expression" in that context to call functions, read values, etc.

fnord772• 5 years ago

I remember doing this with emacs and C in the 90s

BoxOfRain• 5 years ago

Kotlin and IntelliJ IDEA really are a joy to work with in my opinion, they're very much my primary tools at the moment. I'd never really been exposed to functional programming features until I got into Kotlin and now I'm learning Scheme of all things, it's been a bit of a gateway drug for me.

keymone• 5 years ago

you just have to try to understand the difference. the friction is zero in a proper repl, unlike breakpointing and evaling.

simiones2• 5 years ago

The homoiconicity of Lisp is not what gives it this superpower. It is rather Lisp's dynamic nature that actually allows you to keep running your project while also developing it that is crucial - something that Clojure running on the JVM can actually only approximate.

For example, in a full CL, you can modify a class that is currently being used, and objects of that class will adjust accordingly (for most modifications). This is not possible for the JVM - to modify a class, you would have to unload it, which can only happen if the Class object is garbage collected, which requires all instances of the class to be GCd, and the ClassLoader that loaded that class as well - only then could you add a field with a default value to the class.

utxaa12• 5 years ago

i get the magical aspect of repls in scheme and CL. not in clojure. not sure why. so i don't like clojure, but i feel like i should ?? why is that?

fnord772• 5 years ago

I can do this in Java and C with the right IDE.

utxaa12• 5 years ago

i don't love java but no one can argue with the fact that it has fantastic tooling.

properly written java is a joy.

utxaa12• 5 years ago

In emacs, for instance, you often use `eval-last-sexp`, by default bound to C-x C-e. This lets you move your cursor to a particular point in the file, often deep in a function, and get the results of just the form(s) you're pointing at.

but that depends on how the environment is looking at the time.

during development, in scheme, i just have a function (reset) that reloads all the files of an app. sometimes i create a thread and poll the filesystem and reload everything that changed in the last second. that way i never have to C-x anything. if i need to eval a subexp i just jump on the repl and do it.

nerdponx• 5 years ago

Python can hot-reload an application from scratch, too. The idea here is to not reload everything from scratch, but to redefine functions incrementally/iteratively as you work.

nixpulvis• 5 years ago

Well, a UNIX REPL (read sh, bash, fish, etc) is pretty much "homoiconic", since everything is just a text stream. Of course the value of ARGV[0] will be some executable, written and possibly compiled, but that's no different than any other language which needs to eventually make it to hardware instructions.

dgb233• 5 years ago

Some Clojure frameworks and libs do provide REPL support for things that we would otherwise expect to be CLIs, such as running migrations, (re-)starting services and so on.

The startup time criticism is valid, but in context of development you typically don’t restart it except you pull in deps (very rare) or something terrible happens (rare).

Then, there is also borkdude/babashka which is a Clojure powered scripting tool with fast startup times due to GraalVM.

geokon4• 5 years ago

Thanks for introducing me to `add-lib` This is going to be a huge time saver :) More info here: https://insideclojure.org/2018/05/04/add-lib/

Hopefully there will be some way to just "reload" your whole `deps.edn` though

"If you get an error, the execution stops by default and you get a stack trace."

I'd say the other missing piece of REPL development is that while you get a stack trace, you don't get a program state like you do with GDB or ELisp. Maybe I'm "holding it wrong" but this causes a lot of friction and lost time. I'd be curious how others approach this. And that all being said, the CLI doesn't relaly offer a better alternative here.

For entirely replacing the CLI I think that since Clojure is a general purpose language there ends up being a tad more boiler plate than you'd like.. It's not at the point where you're gunna just run `clj`, load in some library with `add-lib` and start messing around b/c things are just a tad too clunky.

For instance if you wanna read in a CSV file (I had to look this up)

    (-> "my-csv-file.csv"
    (io/file)
    (.getCanonicalPath)
    (io/reader)
    (csv/read-csv :separator \,)
    (#(into [] %)))))

Uhh.. so you're prolly gunna want to wrap that up in a helper function. I personally end up making a dummy "project" where I keep a bunch of helper functions and then doing my REPL "scripting" and messing around in that. It feels a bit wrong.. but at least to me it looks like a solvable limitation. Given a nice set of helper libraries you could probably get to a point where a bare `clj` REPL would be as ergonomic as a more explicitely interactive language like R/MATLAB/etc.

marcosdumay2• 5 years ago

Maybe I'm "holding it wrong" but this causes a lot of friction and lost time.

It is a completely non-problem for functional code. It is a big problem for imperative code.

You can't just write all of your code in a functional style, but depending on what you are doing the limit gets larger or smaller. So it's normal that this will be a showstopper for some people, and irrelevant to others.

geokon4• 5 years ago

I don't really follow.. How is it a non problem in functional code? For instance you have some recursive idempotent function that blows at some point. Wouldn't you wanna see the state at which things broke?

Just because things are functional doesn't mean you always knows the inputs at all times

simiones2• 5 years ago

What does this have to do with imperative state?

Take this program:

  compute x = 1 `div` (x - 3)
  map compute [1,2,3,4]

Would seeing that it crashed in `compute` in `map` be enough info to debug, or finding out that `x` was 3 when it did also help?

Edit: fixed to use integer division so we actually crash .

dghf• 5 years ago

This should work just as well:

    (csv/read-csv (slurp "my-csv-file.csv"))

Or if you prefer the threading-macro style:

    (-> "my-csv-file.csv" slurp csv/read-csv)

geokon4• 5 years ago

Oh thanks. Yeah. I thought maybe I was noobing it up :)

Always nice to learn something new

thom2• 5 years ago

CIDER ships with a perfectly adequate debugger if you want it. Might require a bit more manual labour than some environments but you’re never stuck just staring at a stack trace if that’s what’s bothering you.

I think using external dev dependencies and rich dev/user.clj files is pretty common on Clojure projects. Your REPL isn’t just somewhere to interact with the current codebase directly, it’s a framework for building that software and managing its environment more generally.

geokon4• 5 years ago

I need to play with the debugger again.. Does it give you the state along with a crash stack? I remember that unfortunately the debugger couldn't be turned on globally. You need to instrument particular functions. And this turned out to be a major inconvenience in my usecase (a GUI app)

But I'll take another look in my next project

lateusername• 5 years ago

Considering its ecosystem, what do most here use Clojure for or what do you think is its sweet spot? I used it in a project with pg/ring/reitit and while some things were nicer it was not better enough to make me switch from pg/nodejs/express for new projects. Used emacs with inf-clojure (found cider too bloated and buggy, was tempted to port some features of cider to a fork of inf-clojure) and macros were helpful in a couple of places, but other than that, you can write JS almost as how most use Clojure. Also found JS to be faster unless you got out of your way to write Javaish Clojure, which reminds that is was a time sink having to deal with Java libs(it's really not as "seamless" as most advertise) because of no Clojure equivalents.

bcrosby95• 5 years ago

Nowadays I use Clojure as my primary general purpose programming language.

As far as sweet spots go, due to pervasive immutability, I think it can be a good choice anytime you're dealing with concurrency.

If you're dealing with relatively straightforward web applications, outside certain specialized scenarios (and certain really bad choices) I don't think language choice matters.

blacktriangle3• 5 years ago

Clojure's sweet spot is very much the "situated program" Hickey likes to talk about. Clojure is a great fit for line of business apps where maps really are the right choice for modeling your domain.

We recently evaluated Clojure vs node for an upcoming project and went with Clojure because while most of the work really is just querying a db and writing JSON, we also do a fair amount of heavy reporting and pdf generation which will bring node to a crawl relatively speaking. Rather than have to take on the operational complexity of microservices for the slower parts of our app we just went with a Clojure monolith. The draw of a single language to work with for both client and server was very tempting, but ultimately the JVM won out on the server.

I'd be amiss if I didn't also say the whole Deno situation has us worried about the long-term implications of spinning up a brand new node project, where the JVM seems far more reliable logistically.

ithrow• 5 years ago

It comes down to the JVM vs nodejs and which is better for your project. The JS ecosystem is way richer than clojure's though.

Can you share your approach to generating PDFs in Clojure?

a-dub• 5 years ago

protip; you can bring this flow to any language with a repl using vim-slime and a terminal manager like tmux.

revscat• 5 years ago

That was my impression. I’ve been doing this for years with Ruby, tmux, and some custom zsh widgets.

https://github.com/jchilders/dotfiles

godshatter2• 5 years ago

Don’t forget the set -euo pipefail at the beginning of your script. What does -euo means? I don’t know, I copy-pasted it and man set said there is no manual entry for set.

Use "man bash" to find the list of built-in commands. Scroll way down, or search for "SHELL BUILTIN COMMANDS".

The "e" command-line switch to the set command tells the script to exit immediately if there is a non-zero return value. The "u" switch tells the shell to treat unset variables as errors when performing parameter expansion. The "o" switch enables the following option (in this case "pipefail"). "pipefail" tells the script to return the value of the rightmost command that returned with a non-zero value in a pipeline. This is all paraphrased, the details are in the man page.

ajkjk• 5 years ago

It's kinda nonsense that `man set` doesn't provide that information. But it's kinda nonsense that those flags aren't the default, anyway.

aidenn0• 5 years ago

"man set" can't provide that information trivially without knowing your shell; dash/bash/ksh/zsh will all be subtly different.

However, if you are running bash, "help set" is fine.

FWIW, I really don't like pipefail as a default (e.g. piping to "head" causes random pipefails) and -e also has some confusing semantics. Also either failglob or nullglob are more important than either and -C is useful for some scripts as well.

[edit]

Simple example of confusing "set -e" semantics:

The following does not print "hi" and shows returns error status, as expected:

  (set -e; echo hi); echo $?

But put it in an if statement, and it's suddenly success, and does print hi:

  if (set -e; false; echo hi); then echo hello; fi

The same problem applies to functions. The below function will remove all files in the current directory if it's called from a conditional, but not otherwise!

  foo() {
    set -e
    cd /some_directory # set -e means we exit if this fails
    rm -rf *
  }

I think just defining a die() function and using it after any command that must succeed is more verbose, but less error prone:

  cd /some_directory || die "chdir failed"
  rm -rf *

AceJohnny23• 5 years ago

Off-topic, but Bash's manpage is awful and a great demonstration of the limitation of manpages.

Not that the content is bad (it's great!), but as OP (and you) demonstrated, finding relevant information is close to impossible in the humongous document.

In Bash's defense, its manpage is a concession to people's habits, and its documentation really rests in its Info page. Except that doesn't seem to be popular either (I'll admit to being still inexperienced with its UI, after decades of sparse usage)...

AceJohnny23• 5 years ago

Perl (and Git and TCL) seem to have found a middle ground, where they broke up their manpages into different ones that refer to each other (via "SEE ALSO" section at the bottom). But it's sad that, in 2021, we still have such a poor terminal documentation system, and the improvement invented 40 years ago (Info) gained no steam...

godshatter2• 5 years ago

I always forget about the info command, even though it's mentioned in most man pages. So I typed "info bash", and went looking for the "set" command with my almost null knowledge of the info command. It took me a while, but I eventually found it's description under indexes/built-in index/set. It was surprisingly more difficult to find than just looking in the man page, but did take me much less time since I didn't need to skim the whole document to find it. Typing "info info" was actually really helpful, though. Info has a lot more power than I knew about. Typing "i set" while in info brought me right to the set command description. Guess I should learn the tools a bit better.

massysett• 5 years ago

Contrary to typical GNU practice, the bash info manual states that it is intended as a “ brief introduction” and that the man page is the definitive reference on shell behavior:

https://www.gnu.org/software/bash/manual/html_node/index.htm...

airocker• 5 years ago

REPL will not work well with multitheading/futures in Clojure.

utxaa12• 5 years ago

or the ridiculous namespace system.

jefurii• 5 years ago

Linked to in the article: UNIX as IDE (https://blog.sanctum.geek.nz/series/unix-as-ide/)

dig1• 5 years ago

I use add-lib branch of tools.deps.alpha that allows me to add dependencies dynamically at the REPL and then start using them immediately, just like in the shell.

You could do this with (battle-tested) pomegranate [1] ages ago, which is used by leiningen as the default resolver.

[1] https://github.com/clj-commons/pomegranate

REPL vs CLI: IDE wars

Related Stories