An Exploration of SBCL Internals (2020)

https://simonsafar.com/2020/sbcl/

by v9v • 2 years ago

132 106 2 years ago

fooker7• 2 years ago

It's a bit sad that this ecosystem has fallen out of favor for mainstream software development.

One could have designed OOP like frontends to this fairly easily, and I'm sure there are existing ones. How did Java and C++ take over?

FredPret3• 2 years ago

I’ve tried multiple times to get into SBCL, but every time I run into the problem that I have to learn Emacs first.

I know there are lots of other ways to work with SBCL, but they are either second-class citizens or paid.

ngcc_hk• 2 years ago

Vim plus sbcl has been ok. Follow this I am happy since

https://susam.net/lisp-in-vim.html

Not 2nd class and I can use it mostly on mac; linux and windows (linux side)are ok. All good.

taeric• 2 years ago

Why would you have to learn emacs? You can largely batch compile and run lisp code same as any other language, if that is what you want to do. In many ways, the workflow can be very similar to python, if you don't want to get too advanced in the repl. And, to stress, you don't have to get too advanced to get going.

aidenn02• 2 years ago

You can use emacs just for the REPL/debugger, and another editor for editing files. I did this for a couple of years, after previously using CLISP for its REPL. You can use the menus for things; the only keyboard shortcut you MUST know is CTRL-G (which if you hit enough times will cancel whatever operation you accidentally started).

Alternatively try geany-lisp[1], it was created specifically for those with emacs phobia. There are probably some sharp corners using it, so feel free to file issues if you run into any. It should work with Geany 1.26-1.38; when posting this, I just noticed that there is now a Geany 2.0; I'm assuming the major version bump broke plugins so it may not work anymore.

1: https://github.com/jasom/geany-lisp

rayiner• 2 years ago

You don't strictly need to learn Emacs, but you should probably just join the cult. I bit the bullet and did it about 20 years ago now, and it's been a good thing. Heck, I've got a non-programming dayjob now, but I've still got a keyboard that puts the control key under my thumb: https://x-bows.com.

potta_coffee• 2 years ago

I'm in the same boat. I just don't care about EMACS, don't like it, and there's little incentive when I'm just playing around. It's a shame because Lisp is really fun and interesting. I just don't want to spend so much time dicking around with an editor just so I can write some code.

aredox• 2 years ago

There is Lem too.

https://lem-project.github.io/

reddit_clone2• 2 years ago

Learning Emacs is not a bad thing you know..

It has a lot of other goodies like : Org, Magit etc which makes it worth while.

cess114• 2 years ago

I agree that it's a hurdle.

Portacle, https://portacle.github.io/ , is a way around config and whatnot, lowering the threshold a little.

whartung2• 2 years ago

I normally dabble with CLISP (notably because of its built in readline), and if I could SOMEHOW get it to build with SSL, I'd play with it more.

This is what I stick at the top of my Lisp files when dabbling.

  (defun l ()
    (load "file.lisp"))

  (defun e ()
    (ext:shell "vi file.lisp"))

And then I just muddle my way through with a (e) and (l) cycle. Since I tend to not work on 10,000 line files -- this works fine (my files are < 5K lines). More than fast enough, and I get to retain any work variables within the image (plus my command history). It's quite effective really. (print...) debugging works because you can run your code at whatever granularity you like (and you can always refactor it to a finer grain if you want). Don't doubt it until you try it. Turn around is very fast.

That said, I discovered that for some reason, firing off something as simple as "vi file.lisp" from SBCL is stupifyingly complicated. I made several 10s google searches and 5s GPT attempts to make that work, but gave up.

So, can't say I can recommend that style of development in SBCL unless someone is willing to chime in with the, apparently, paragraph of code necessary to launch vi on a file.

vindarel3• 2 years ago

The plugin in Pulsar (Atom) is pretty good: https://lispcookbook.github.io/cl-cookbook/editor-support.ht... (Sublime, Vim, Jupyter notebooks, VSCode (OKish), Eclipse (simple)…)

Qem2• 2 years ago

I’ve tried multiple times to get into SBCL, but every time I run into the problem that I have to learn Emacs first.

I have the same problem with Hylang.

hedora2• 2 years ago

Java took over by targeting large teams of inexperienced developers.

C++ took over by adding OO to C with zero cost abstractions, and then evolving into modern C++ (which discourages OO and encourages composition via template instantiation) without leaving programs behind.

Lisp has supported composition forever (it is one of the big advantages of functional programming), and the macro language is similar to templates. I think the big problem is that it is difficult to scale lisp projects with inexperienced developers (loses to java) and is not zero cost (has a GC, is often interpreted; loses to c, c++ and now rust).

pfdietz8• 2 years ago

is often interpreted

Every competitive Common Lisp implementation has compilation. SBCL defaults to not even using an interpreter for eval.

medo-bear4• 2 years ago

I think the big problem is that it is difficult to scale lisp projects with inexperienced developers (loses to java) and is not zero cost (has a GC, is often interpreted; loses to c, c++ and now rust).

i think scalability problem is a bit overstated. the biggest issue with lisp becoming popular is that it doesn't have a large user base and is not a new language like rust. however despite being old, common lisp /still/ has so many 'novel' things in it that in metrics of programmer ergonomics and programming user interface make common lisp a big winner for me.

as regards zero cost, lisp programs can be made arbitrarily close to being zero cost, but here be deamons.

ilrwbwrkhv2• 2 years ago

Well take over is relative. I built my startup using SBCL and it is very profitable. Couldn't have done it without SBCL.

fooker7• 2 years ago

Couldn't have done it without SBCL

Could you elaborate this part?

p_l4• 2 years ago

Could you share the name (and what it does)f your startup?

linguae3• 2 years ago

Let’s make a trip back to the early and mid 1990s when C++ was a young language and Java was under development (it was released in 1995). At the time the most popular object-oriented language was Smalltalk, Common Lisp was at its peak of commercial popularity (largely in symbolic AI niches), and there were other niche OO languages that were heavily influenced by Smalltalk and Common Lisp (Objective-C in the NeXT world, Apple’s Dylan under development).

However, if I know my history correctly, Smalltalk and Common Lisp implementations were generally not cheap. Objective-C was largely tied to NeXT, which required buying into a niche ecosystem. Dylan was a casualty of Apple’s mid-1990s business struggles.

C++ benefitted from inexpensive implementations from vendors such as Borland and Microsoft, and it also benefitted from its close relationship with C, which already gained a foothold in the 1980s.

Java benefitted not only from Sun’s marketing machine, but also from its free compiler and runtime from Sun. Java overtook Smalltalk, despite the fact that Java lacks Smalltalk’s dynamics.

I think had there been solid cheap or free Smalltalk and Common Lisp implementations around 1992, Java wouldn’t have gained a foothold, though C++ would have still been very appealing to C developers, though perhaps if Objective-C were more broadly available to non-NeXT developers, it would’ve been a formidable competitor to C++ in terms of providing a “C with objects” environment.

medo-bear4• 2 years ago

I think had there been solid cheap or free Smalltalk and Common Lisp implementations around 1992, Java wouldn’t have gained a foothold, though C++ would have still been very appealing to C developers, though perhaps if Objective-C were more broadly available to non-NeXT developers, it would’ve been a formidable competitor to C++ in terms of providing a “C with objects” environment.

100% agree. i think it is also interesting that in comparison to before lisp community of today is so supportive of free software, standing more on the left wing of the open source community.

fooker7• 2 years ago

The economics of this is interesting to hear.

GNU Common Lisp (https://www.gnu.org/software/gcl/) seems to have existed in various forms from the mid 1980s, but it was probably outclassed by the contemporary big tech companies.

wglb4• 2 years ago

Well, there is this: https://www.norvig.com/java-lisp.html

coliveira2• 2 years ago

Java claim to fame was to be close enough to C++ but portable in binary form. This made it possible for millions of people to learn it quickly. Lisp is too distant from C++ to have had any change.

akira2501• 2 years ago

for mainstream software development.

LISP feels like both an "assembly language" and a "higher level language" at the same time. It's excellent for solving unusual problems, but it carries a lot of baggage when aiming for "usual" problems.

coliveira2• 2 years ago

The big problem for Lisp is that it was too slow during the 80s and 90s. At that time, C and C++ took the market share and maintained it since then. New languages had to conform to C, since Lisp has its own independent way of doing things.

anothername12• 2 years ago

A lot of people write Lisp interpreters, post it to HN and call it a day, but I think compilation is the far more interesting task for a Lisp implementation.

SBCL is great to poke around in and the decompiler’s great. Also check out ABCL, ECL for what compilation to JVM bytecode and C looks like.

mtreis86• 2 years ago

The scheme compiler in Paradigms of Artificial Intelligence Programming by Peter Norvig is a great introduction to compiling Lisp. I was particularly impressed by it only taking a change in one line to go from regular recursion to handling tail call optimization.

fooker7• 2 years ago

This is a bit Haskell oriented, but the "Implementation of Functional Programming Languages" by SPJ is one of my all time favorite books.

https://simon.peytonjones.org/slpj-book-1987/

These compilers were pulling heroic data structure optimization stunts in 1985, that no modern production compiler for major compiled languages can replicate in 2024.

thewakalix• 2 years ago

Why? Is it mostly because of Haskell's purity (side-effect-freeness), so optimizations can change expressions more freely and safely?

nxobject• 2 years ago

Huh – I didn't know he'd left MSR for Epic Games.

amock• 2 years ago

  remember that we're on x86 here, which is a big-endian architecture

While I wish this were true, x86 is little-endian.

greenyoda• 2 years ago

The author correctly states that x86 has "lower bits at the beginning, bigger bits on the end", but, as you said, that's the definition of little-endian.

ssafar• 2 years ago

Apparently, I even made up an explanation for why "big-endian == start with the least significant bit" totally makes sense... couldn't convince the Universe this way either though.

Thanks for catching it, fixed!

lukego• 2 years ago

SBCL 25th anniversary workshop is in Vienna thu-fri next week. See you there!

koito17• 2 years ago

I wish other Common Lisp implementations, particularly Clozure CL, documented their internals as well as SBCL does. Apple Silicon support has been a long-standing feature request, but nobody can really work on it besides rme himeself due to the sheer amount of "institutional knowledge" (and time!) it takes to understand CCL's compiler backend then add support for a brand new architecture. SBCL, on the other hand, was able to provide support fairly quick, possibly due to more maintainers, but I also think a big role that played in it was the fact lots of corners of the implementation are documented and it's fairly easily to get a hold of people actively working on the compiler (doug, stassats, et al)

vindarel3• 2 years ago

Fresh news from 2 weeks ago for CCL:

I recently decided to leave my current full-time job, and I’ll be working on CCL again. […] I’ll be able to work about half-time on an ARM64 port. Please write me privately if you want to talk about supporting that ARM64 work.
I've worked on CCL for quite a few years, and I did the 32-bit x86 port, so I have experience in this area. I’m not as good a hacker as Gary Byers, but then again, few people are.

rme

https://lists.clozure.com/pipermail/openmcl-devel/2024-April...

waynesonfire• 2 years ago

My issue prevent me from investing in Lisp is that Erlang gives me a similar REPL which I believe, though I am not a Lisp developer, is a huge value proposition of Lisp. I utilize REPL-based development using Erlang.

The macro narrative is worse in Erlang; but Lisp isn't perfect either in other ways.

The point is, I net out staying on the BEAM. What I like about Lisp is that I think it's a tool that will serve you for the rest of your career, even if you don't use it for production. That's a pretty cool proposition and maybe one day I'll get there. Emacs is a similar tool.

vindarel3• 2 years ago

Last time I played with Elixir (I'll assume it's similar enough to Erlang), I found the REPL / interactive experience not as rich as CL's (which I must say, reassured my beliefs), even though the command-line tools are great:

- can't install libraries from the REPL while working on your program

- can't "compile this function" with a keystroke (I did find "send region/line/buffer to REPL" which is different, or "reload module")…

- … and get type errors and warnings

- can't get an interactive debugger on an error, go to the bogus line (while keeping the debugger), fix the function, compile it, switch back to the debugger, resume execution from any point in the stack. I only found out IEx.pry which is useful and all, but looks like Python's ipdb break point.

- lack of little useful editor commands, at least in the Emacs modes I tried like "write this function in the REPL", "eval and print the last expression". I struggled to have the "go to definition" feature but that depends on the IDE I suppose. The Emacs modes are not in good shape :S

- no cross-references? doubting. And maybe other small things only about the REPL, didn't find my notes.

differences of runtimes are also at play here, but I still find it interesting to compare the REPL experience, I'll welcome more comparisons.

cess114• 2 years ago

BEAM/OTP is rather different from CL:s image-based REPL experience.

You can get relatively close by putting your application into processes and keep them in a collection or registry, then when you've found an issue and changed the code in your editor you can run recompile and relaunch affected processes. You'd lose a subtree in the application, and hence in-memory storage tightly coupled to those processes, but I find it rather easy to do it this way. Often you'll put in-memory data in separate caching processes and that reduces friction in restarting processes with application logic even more.

The 'let it crash'-foundation of the BEAM system isn't exactly tailored towards trapping and recovering from errors but if you really want to you could probably implement something like that with try/rescue but I suspect it would turn out quite obtuse compared to the CL mechanics.

7373737373• 2 years ago

Any explanation for the name?

xenophonf• 2 years ago

Steel Bank Common Lisp (SBCL) is a fork of Carnegie Mellon University Common Lisp (CMUCL).

University founders Andrew Carnegie, Andrew Mellon, and Richard Mellon are best known for their work in the steel industry and banking.

Akronymus• 2 years ago

This describes fairly well what we've seen so far. The "lowtag" is those 4 bytes at the end of Lisp objects; also, we're clearly on a 64-bit architecture here, with N_WORD_BITS being 64, 8 of them reserved for the lowtag. It also details various values for the lowtag, the important one being...

Shouldn't that say bits? As 8 bytes would be the whole thing ayways.

An Exploration of SBCL Internals (2020)

Related Stories