OpenLDK: A Java JIT compiler and runtime in Common Lisp

by varjag • 1 year ago

212 61 1 year ago

atgreen6• 1 year ago

Hello -- I wrote this! It's still very much a work in progress.

whalesalad2• 1 year ago

A pragmatist will point to this and say, why not Clojure? Can you share more about what keeps you using SBCL instead of Clojure?

jhbadger• 1 year ago

Or for that matter, Armed Bear Common Lisp (real CL implemented in the JVM rather than Clojure which is more like Scheme mixed with its own ideas). Still, it's a neat trick, running Java classes on a Lisp that isn't in the JVM itself.

remexre• 1 year ago

I thought Clojure already had JVM interop?

eadmund• 1 year ago

Well, for one thing Clojure is not Lisp. It’s a neat language (and is informed by experiences with Lisp), but it’s a different thing.

atgreen6• 1 year ago

I'm sure they would, but I have the freedom to not be practical! The journey is the destination.

Common Lisp is fun to code in, and the tooling is top-tier.

koito17• 1 year ago

why not Clojure?

Common Lisp implementations like SBCL have static typing, performance, instant (and effortless) compilation to native machine code, and so on. No need to deal with Leiningen or copy-pasting a deps.edn / build.clj template across projects. If I want to experiment with SIMD intrinsics with the comfort of a REPL, I can't do that with Clojure, but I can do that with SBCL and a built-in library (SB-SIMD).

Note: I have used Clojure professionally and still use it for some personal projects, but I try to be aware of the trade-offs when deciding to use Clojure / Common Lisp / any other programming language.

thunkingdeep• 1 year ago

I am a hobbyist Scheme programmer, and I can chime in and say that Clojure stack traces are among the worst I’ve ever seen in my life. Chez errors aren’t the best in the world, but anything beats those JVM ones, imo.

hencq2• 1 year ago

Very cool! I'm curious whether you're using it (or planning to use it) yourself for anything serious or if it's just to see if it could be done. Are you planning to support classes beyond Java 8 at some point?

atgreen6• 1 year ago

I'll definitely use it when it becomes usable. And, yes, it needs to support newer versions of Java. Crawl.. walk.. run. But you are right.. part of my motivation was just to see if it was possible. Many years ago, I worked on gcj, which was similar in concept, except that it mapped Java classes to C++ classes (as implemented by g++). I've been on a Common Lisp kick for the past few years, so it seemed like a fun challenge. The fact that the OpenJDK class libraries are free software now makes this more practical.

skapadia• 1 year ago

@atgreen - Would love to hear what key lessons / takeaways you've learned while working on this project?

wbl• 1 year ago

Are you planning to do a Futamura projection?

atgreen6• 1 year ago

No. It's a simple transpiler for now.

rootnod32• 1 year ago

Finally I can run Clojure in Common Lisp :D

Joke aside, that is an amazing project.

blu3h4t• 1 year ago

Does this have something to do with recursion? I guess what I’m thinking is bout is inception, I guess there already exist some lib called inception, wonder what it would do, a compiler? :D Java to cl compiler I guess :D

anonzzzies3• 1 year ago

Nice work. I will have a read of the code over the weekend.

behnamoh2• 1 year ago

Speaking of Lisp, I'm developing a programming language that is similar to other Lisps, but I noticed that we can sprinkle a lot of macros in the core library to reduce the number of parentheses that we use in the language.

example: we could have a `case` that works as follows and adheres to Scheme/Lisp style (using parentheses to clearly specify blocks):

    (case name
        (is_string? (print name))
        (#t         (print "error - name must be a string"))
    )

OR we could also have a "convention" and treat test-conseq pairs implicitly, and save a few parentheses:

    (case name
        is_string?    (print name)
        #t            (print "error ...")
    )

what do you think about this? obviously we can implement this as a macro, but I'm wondering why this style hasn't caught on in the Lisp community.

Notice that I'm not saying we should use indentation—that part is just cosmetics. in the code block above, we simply parse case as an expression with a scrutinee followed by an even number of expressions.

Alternatively, one might use a "do" notation to avoid using (do/begin/prog ...) blocks and use a couple more parentheses:

    (for my_list i do
        (logic)
        (more logic)
        (yet more logic)
    )

again, we simply look for a "do" keyword (can even say it should be ":do") and run every expression after it sequentially.

hencq2• 1 year ago

A bunch of lisps do that. Clojure for instance or arc use conventions like that to reduce the number of parentheses.

See e.g. cond in clojure: https://clojuredocs.org/clojure.core/cond

diggan• 1 year ago

Clojure for instance or arc use conventions like that to reduce the number of parentheses

Clojure also reduces parentheses by introducing more characters for various data-structures (good and bad, many would say), like [] for vectors, which are for example used for arguments when defining functions

    (def add-five [a]
      (+ a 5))

shawn_w• 1 year ago

Your for example is basically Common Lisp's `loop`

    (loop for i in my-list
      do (logic)
         (more logic)
         (yet more logic))

kgeist• 1 year ago

In an old programming language project of mine, I wanted to use something similar to S-expressions for most of the grammar because they are very universal and flexible, but I didn’t like having too many parentheses, so I extended the syntax by allowing semicolons as an alternative.

So, in pseudocode:

  (print (+ 1 2))
  (exit 1)

becomes:

  print (1 + 2); // "+" is the 2nd place because it's a method call on "1"
  exit 1;

Both variants are valid; semicolons are just syntactic sugar to reduce the number of parentheses. In the AST, they represent the exact same expressions.

For function bodies, I also introduced an extension to make lambdas stand out more from the rest of the code by using curly braces:

  (0 to 10) loop ^(i) {
    print i; // Internally transformed to (print i)
  }

  // clarification:
  // 1. The "to" method of the 0 object creates a Range object, which has a "loop" method that accepts a lambda.
  // 2. The ^ symbol is just a shorthand for "lambda".

As you can see, this wasn't a Lisp dialect (and I'm no expert at Lisps), but I liked the idea of having a universal expression structure in the AST, however I eventually ended up extending it with two additional modifications that made it more readable (at least for me)

tmtvl• 1 year ago

Imagine if we had a Lisp with proper types, we could do something like...

  (typecase name
    (String (print name))
    (otherwise (print "error: name must be a string!")))

And if that Lisp had some kind of Object System, we may even go further and have stuff like...

  (defmethod namecall ((name String))
    (print name))

  (defmethod namecall ((name Number))
    (print "I am not a number, I am a free man!"))

codr72• 1 year ago

Wild!

behnamoh2• 1 year ago

clojure's multi-arity addresses this to some extent but pattern matching like we know it in Haskell is very weak in Lisps...

gjm11• 1 year ago

In case anyone is missing the snark here, all of tmtvl's code above is in fact perfectly valid Common Lisp.

bradrn• 1 year ago

Actually, this reminds me very strongly of Rebol [0] and related languages (e.g. Red [1]). They work as you describe at the end: whenever a function is encountered, the interpreter inspects the following arguments, without any need for parentheses. Since these languages are homoiconic like Lisp, it’s possible to use functions as if they were macros (achieving it at runtime using a rather strange ‘binding’ system).

[0] https://en.wikipedia.org/wiki/Rebol

[1] https://www.red-lang.org/

to4534o5345• 1 year ago

It's just a bunch of macros to this, no ?

There are a lot more hairy/pressing issues given how old the CL spec is (the type-system esp. comes to mind).

anonzzzies3• 1 year ago

You can do a lot with creating your dsl/tooling and something like Coalton really. More than enough imho.

attila-lendvai• 1 year ago

if you think that there are too many parens in lisp then you should fix your editor.

the parens are just a textual representation of a tree. once you have a proper editor, then the eliminated parens actually become an obstacle.

anonzzzies3• 1 year ago

Just install an editor plugin (parinfer I like now) to fix that easily. It's a pleasure. It's not hard to make your own really; it's, like you say, only cosmetic. You can format your code for reading in the same way with a trivial plugin; it is a really nice thing that you can very rapidly make something nicer tailored for you while still benefitting from the performance / debugging / etc in sbcl. For my dayjob, I work a lot with Typescript parsing and rewriting and, while the ts (and ts-morph and so on) are pretty nice, it's really a quite horrible experience vs what we have in lisp/scheme.

svilen_dobrev• 1 year ago

pff. Now run a hardware Lisp machine under it..

https://en.wikipedia.org/wiki/Lisp_machine

cduzz• 1 year ago

Have you seen:

https://winworldpc.com/product/ibm-visualage-for-java/10

QUOTE: VisualAge for Java is based on an extended Smalltalk virtual machine which executes both Smalltalk and Java byte codes. Java natives were actually implemented in Smalltalk. Later versions were implemented on regular Java.

Obviously not the same....

atgreen6• 1 year ago

Yes, I'm familiar with that! It was created by the OTI team in Canada, which was acquired by IBM. I think some of them are still around, working on IBM's OpenJ9 java implementation. They reach out to me now and then on libffi-related issues.

hondadriver• 1 year ago

Reminds me of Greenspun's tenth rule of programming (also counts for Java, but this most probably predates it):

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

mjburgess2• 1 year ago

Ritchie's Revenge: And this half is still faster than Common Lisp.

pjmlp• 1 year ago

Only when adding UB exploits done by optimizing compiler backends.

I am old enough to remeber the days any junior Assembly programmer could easily outperform any machine code generated by the C compilers of the day.

masijo• 1 year ago

SBCL (the most popular Common Lisp implementation) is pretty darn fast, being comparable to Java, Go or OCaml.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/

wavemode• 1 year ago

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

...including Common Lisp.

kreelman• 1 year ago

Haha... It would be strange to run this inside https://abcl.org/...

A Java runtime... running on a Java runtime... Hmm, maybe not. But could be possible.

password4321• 1 year ago

A Java Virtual Machine and Bytecode-to-IL Converter for .NET

breadchris• 1 year ago

this is like the opposite of clojure right? lol amazing

draven• 1 year ago

More like the opposite of ABCL (Armed Bear CL.)

tempodox• 1 year ago

This will achieve perfect cult status if you use a Common Lisp that was implemented in JavaScript.

OpenLDK: A Java JIT compiler and runtime in Common Lisp

Related Stories