Web Embeddable Common Lisp

https://turtleware.eu/static/paste/wecl-test-gl/main.html

by todsacerdoti • 1 year ago

151 43 1 year ago

tempodox• 1 year ago

Are we consing yet?

  > #'cons
  #<compiled-function CONS 0x43cee0>

That's a definite yes :)

jackdaniel6• 1 year ago

This is just a test page. Today I've shared an information about an accepted grant proposal:

https://functional.cafe/@jackdaniel/114742776265318353

The work will go towards improving browser integration and porting to WASI.

remexre• 1 year ago

Will this include some form of SLIME support?

jackdaniel6• 1 year ago

I didn't plan explicitly for SLIME, but loading swank shouldn't be much of a problem. The missing piece would be a bridge between a websocket and tcp I think.

HexDecOctBin• 1 year ago

Is WASM a good target for REPL-driven programming? I haven't studied it in detail, but I do remember that it was Harvard architecture. Would that mean that code can't be updated at runtime in real implementations?

jackdaniel6• 1 year ago

ECL features native code and bytecodes vm for targets without incremental compilation support - both native and bytecode can be freely mixed at runtime.

That means that you may interactively use repl to call functions that were compiled ahead of time and to add new functions even on uncooperative targets.

After you've finished prototyping you may compile the finished library to native ahead of time.

DonHopkins3• 1 year ago

It is, actually! I didn’t realize that until I read the handwritten WASM s-expression source code of WAForth.

Languages like Lisp or Forth that run on WASM — or even handwritten s-expression-style WASM code like in the WAForth kernel — can dynamically compile and assemble new WASM bytecode and data. They can then call back to the JavaScript host to create and link entirely new WASM modules on the fly, and immediately call into them from the current module or others. A single WASM context can juggle many dynamic modules, large and small, all generated and linked at runtime!

So you can compile new code in Forth or Lisp and immediately link and call it using indirect cross-module calls. It’s not the fastest way to call into WASM, but a metacompiler — like Mitch Bradley’s ForthMacs / OpenFirmware — could recompile the whole system into one efficient, monolithic WASM module that uses direct calls.

Here's the author's blog post about WAForth:

A Dynamic Forth Compiler for WebAssembly

https://el-tramo.be/blog/waforth/

https://news.ycombinator.com/item?id=34374057

DonHopkins on Jan 13, 2023 | parent | context | favorite | on: What the hell is Forth? (2019)

This is a great article! I love his description of Forth as "a weird backwards lisp with no parentheses".

https://blog.information-superhighway.net/what-the-hell-is-f...

Reading the source code of "WAForth" (Forth for WebAssembly) really helped me learn about how WebAssembly works deep down, from the ground up.

It demonstrates the first step of what the article says about bootstrapping a Forth system, and it has some beautiful hand written WebAssembly code implementing the primitives and even the compiler and JavaScript interop plumbing. We discussed the possibility of developing a metacompiler in the reddit discussion.

I posted this stuff about WAForth and a link to a reddit discussion with its author in the hn discussion of "Ten influential programming languages (2020)":

https://news.ycombinator.com/item?id=34056735

Yes, I agree FORTH should be probably be on the list, at least if the list was a few languages longer, for as influential as it's been.

WAForth for WebAssembly is beautiful and modern!

https://github.com/remko/waforth

It's a lovingly crafted and hand written in well commented WebAssembly code, using Racket as a WebAssembly macro pre-processor.

I learned so much about WebAssembly by reading this and the supporting JavaScript plumbing.

The amazing thing is that the FORTH compiler dynamically compiles FORTH words into WebAssembly byte codes, and creates lots of tiny little WebAssembly modules dynamically that can call each other, by calling back to JavaScript to dynamically create and link modules, which it links together in the same memory and symbol address space on the fly! A real eye opener to me that it was possible to do that kind of stuff with dynamically generated WebAssembly code! It has many exciting and useful applications in other languages than FORTH, too.

Lots more discussion and links in the reddit article.

But here's the beef, jump right in:

https://github.com/remko/waforth/blob/master/src/waforth.wat

Reddit /r/Forth discussion of WAForth:

https://www.reddit.com/r/Forth/comments/zmb4eb/waforth_wasmb...

remko:

Author here

If you can't be bothered to install VS Code, you can have a look at a standalone version of the example notebook (in a 26kB self-contained page).

And if you're planning to go to FOSDEM 2023, come say hi: I'll be giving a talk there on WebAssembly and Forth in the Declarative and Minimalistic Computing devroom.

DonHopkins:

I really love your tour-de-force design and implementation of WAForth, and I have learned a lot about WebAssembly by reading it. Never before have I seen such beautiful meticulously hand written and commented WebAssembly code.

Especially the compiler and runtime plumbing you've implemented that dynamically assembles bytecode and creates WebAssembly modules for every FORTH word definition, by calling back to JavaScript code that pulls the binary bytecode of compiled FORTH words out of memory and creates a new module with it pointing to the same function table and memory.

WebAssembly is a well designed open standard that's taking over the world in a good way, and it also runs efficiently not just in most browsers and mobile smartphones and pads, but also on the desktop, servers, cloud edge nodes, and embedded devices. And those are perfect target environments for FORTH!

What you've done with FORTH and WebAssembly is original, brilliant, audacious, and eye-opening!

I'd read the WebAssembly spec before, and used and studied the Unity3D WebAssembly runtime and compiler to integrate Unity3D with JavaScript, and I also studied the AssemblyScript subset of TypeScript targeting WebAssembly and its runtime, and also Aaron Turner's awesome wasmboy WebAssembly GameBoy emulator .

I first saw your project a few years ago and linked to it in this Hacker News discussion about Thoughts on Forth Programming because I thought it was cool, but it's come a long way in three years, and I'm glad I finally took the time to read some of your code, which was well worth the investment of time.

Until reading your code, I didn't grasp that it was possible to integrate WebAssembly with JavaScript like that, and use it to dynamically generate code the way you have!

Also, the way you used Racket as a macro assembler for WebAssembly was a practical and beautiful solution to the difficult problem of writing maintainable WebAssembly code by hand.

Even for people not planning on using FORTH, WAForth is an enlightening and useful example for learning about WebAssembly and its runtime, and a solid proof of concept that it's possible to dynamically generate and run WebAssembly code on the fly, and integrate a whole bunch of tiny little WebAssembly modules together.

Playing with and reading through your well commented code has really helped me understand WebAssembly and TypeScript and the surface between them at a much deeper level. Thank you for implementing and sharing it, and continuing to improve it too!

remco:

Wow, thanks a lot, I really appreciate that! It makes me very happy that I was able to get someone to learn something about WebAssembly by reading the source code, which is exactly what I was going for.

[More links and discussion of WAForth, WebAssembly, and Forth Metacompilers:]

https://www.reddit.com/r/Forth/comments/zmb4eb/waforth_wasmb...

Here's the author's blog post about WAForth: A Dynamic Forth Compiler for WebAssembly

https://el-tramo.be/blog/waforth/

https://news.ycombinator.com/item?id=34374057

DonHopkins on Jan 13, 2023 | parent | context | favorite | on: What the hell is Forth? (2019)

This is a great article! I love his description of Forth as "a weird backwards lisp with no parentheses".

Reading the source code of "WAForth" (Forth for WebAssembly) really helped me learn about how WebAssembly works deep down, from the ground up.

I posted this stuff about WAForth and a link to a reddit discussion with its author in the hn discussion of "Ten influential programming languages (2020)":

https://news.ycombinator.com/item?id=34056735

Yes, I agree FORTH should be probably be on the list, at least if the list was a few languages longer, for as influential as it's been.

WAForth for WebAssembly is beautiful and modern!

https://github.com/remko/waforth

It's a lovingly crafted and hand written in well commented WebAssembly code, using Racket as a WebAssembly macro pre-processor.

I learned so much about WebAssembly by reading this and the supporting JavaScript plumbing.

Lots more discussion and links in the reddit article.

But here's the beef, jump right in:

https://github.com/remko/waforth/blob/master/src/waforth.wat

Reddit /r/Forth discussion of WAForth:

https://www.reddit.com/r/Forth/comments/zmb4eb/waforth_wasmb...

remko:

Author here

If you can't be bothered to install VS Code, you can have a look at a standalone version of the example notebook (in a 26kB self-contained page).

And if you're planning to go to FOSDEM 2023, come say hi: I'll be giving a talk there on WebAssembly and Forth in the Declarative and Minimalistic Computing devroom.

DonHopkins:

What you've done with FORTH and WebAssembly is original, brilliant, audacious, and eye-opening!

Until reading your code, I didn't grasp that it was possible to integrate WebAssembly with JavaScript like that, and use it to dynamically generate code the way you have!

Also, the way you used Racket as a macro assembler for WebAssembly was a practical and beautiful solution to the difficult problem of writing maintainable WebAssembly code by hand.

remco:

[More links and discussion of WAForth, WebAssembly, and Forth Metacompilers:]

https://www.reddit.com/r/Forth/comments/zmb4eb/waforth_wasmb...

fsckboy• 1 year ago

Yes, I agree FORTH should be probably be on the list, at least if the list was a few languages longer, for as influential as it's been.

Forth has not been influential, at all. If it was your first or second encounter with computers, it might have been influential on you, which is a great thing for you, but it was not influential on the industry.

the late 70's-early 80's was the beginning of the microprocessor era. This opened the door to computers for people who would be interested in computers but had no access prior. But the "leading" people in computer science at that time were the people who did already have access to computers. This created a bifurcation between the people who were, say, pursuing lisp since 1960 because of what they had learned about lambda calculus before, and people pursuing Forth who had never heard of Lisp, but had heard of Pascal but didn't realize that UCSD Pascal was a p-code stack machine just like Forth was; if they had realized it there might have been overlap, there wasn't. Programmable HP calculators were already more advanced since the HP-65

(please don't nitpick me, if you remember that time, you know what i'm talking about.) The much revered Dr Dobbs and Byte magazine were not on the reading list of people who were at MIT or Carnegie or Stanford. If you graduated at that time (hint hint) you had this choice of working at a place like Microsoft (DOShit in the boonies) or Apple (8-bit 6502 nonsense on an OS nobody remembers) or at DEC or HP or Xerox or BBN, companies pursuing excellence based on the already-history of CS.

because microprocessor "ate the world", the early history of microprocessors seems "important": it wasn't, except for what it became

when Apple and Microsoft borrowed ideas from Xerox PARC to create MacOs and Microsoft Windows, they were borrowing from programmable microcode architectures which are still more interesting than amd64

runlaszlorun• 1 year ago

I'd use the analogy of current dynamic language VMs being written C. You're not modifying structs at runtime or creating C functions as a user of the dynamic language. But the creators of the VM are providing as thin a layer as possible on top of that to give the runtime behavior it's dynamism.

There's an often repeated line that WebAssembly isn't really "assembly" and it's not "web". There's a lot of truth to that. I dove in hoping for a cross platform, web enabled assembly. It looks a lot like an assembly but I find it's relatively few restrictions like lack of a true jump command rippling out more than one would have expected. It's also sortuva stack machine and sortuva register machine at the same time.

It does share a lot with the internal VMs in many places like the JVM, Python's VM, .net, etc.

DonHopkins3• 1 year ago

"I'm a little verklempt. Talk amongst yourselves. I'll give you a topic: WebAssembly is neither Web nor Assembly. Discuss." -Your Host, Linda Richman

https://www.youtube.com/watch?v=oiJkANps0Qw

ethan_smith• 1 year ago

Modern WASM implementations use JIT compilation with dynamic code generation capabilities that effectively bypass the Harvard architecture limitations, allowing for runtime code updates through module instantiation and function replacement.

potholereseller• 1 year ago

Apparent source code, though last update is January 1, 2025: https://fossil.turtleware.eu/wecl/timeline

jinlisp6• 1 year ago

When executing (time (loop for i below (expt 10 8) count t)) it takes a long time, sbcl takes less than a second on this. So not useful when speed is required. More: (time (loop for i below 1000000 count t) takes 7 seconds on my computer, so counting to (exp 10 8) would take 700 seconds, and sbcl do that in 0.1 seconds, so in this example the ratio is 7000 times slower than sbcl.

jackdaniel6• 1 year ago

Bytecodes compiler used in this build from repl is one-pass with very little optimizations, so it is not surprising. Natively compiled code is much faster.

jinlisp6• 1 year ago

Should it be possible to implement an optimization compiler? so that compiling the code in the web page produces results similar to ECL or sbcl?

mishoo4• 1 year ago

In SLip: https://lisperator.net/s/slip/

    SL-USER> (time (loop for i below (expt 10 8) count t))
    Evaluation time: 19376ms
    100000000

    SL-USER> (time (loop for i below 1000000 count t))
    Evaluation time: 272ms
    1000000

    SL-USER> (defun fib (n) (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
    #'FIB
    SL-USER> (time (fib 32))
    Evaluation time: 1601ms
    2178309

These timings are in Firefox; Chromium is almost 2x faster. Also, it doesn't completely freeze the browser in the mean time (we have green threads).

I used to think it's quite slow, but somehow it's much faster than any other posted here. The only Lisp in browser I know to beat SLip (by a large margin) is JSCL, which compiles to JS.

jinlisp6• 1 year ago

Both slip and jscl takes the same time in my computer: 63 seconds to run the loop.

jinlisp6• 1 year ago

I was going to ask if maxima (a symbolic computation system) can be implemented in the browser, but it was answered four months ago [1]

[1] https://news.ycombinator.com/item?id=42853528

mixedmath• 1 year ago

Do you use maxima? Would you say it's still worth using and learning?

jinlisp6• 1 year ago

It depends of what you need, but for example for calculus is a nice program. There is also sympy and Wolfram Mathematica. For symbolic computation I think that Mathematica is the strongest then maxima and then sympy, but sympy is based on python and I think it will get stronger. If you need numerical computation then there is octave or matlab or julia.

johnisgood• 1 year ago

Yes, I think it is worth learning. GNU units is worth learning, too. You know these math riddles? You can solve it using GNU units alone. :D

jinlisp6• 1 year ago

This is another comment of the author about maxima in the browser from four months ago: I'm experimenting with WASI and the GC extension for WASM, but that's months from today if we speak about complete port (given my time capacity at the moment). Don't know if the gc extension is used in this example.

bevr13374• 1 year ago

Why was LISP not one of the first languages able to compile to WASM? I've been curious ever since the awesome WASM project started tracking different projects. They use emojis for progress and LISP has been a hatching egg.

no_wizard2• 1 year ago

It sorta has been. As a language design it is already native via the WAT[0] which compiles to web assembly.

[0] https://webassemblyman.com/wat_webassembly_text_format.html

bevr13374• 1 year ago

Do you know why only sorta? My understanding is that only toy implementations exist right now but nothing production ready. I'm poking around Google and GitHub and not finding any new information.

Implementing Lisp onto WebAssembly is so obvious that you may wonder why somebody did not have this idea long ago.

I did find this on the Femto GitHub and got a chuckle. Yes, dear author, that is exactly the piece of information I want explained. Too funny.

dmkolobov• 1 year ago

If I had to guess:

1. lack of native GC, you had to roll your own by providing a runtime.

2. lack of tail-call elimination in V1 of the spec. This essentially forces you to trampoline everything.

stassats• 1 year ago

lack of tail-call elimination

But lisp doesn't need TCO.

jackdaniel6• 1 year ago

Btw, eval (ed "wecl.lisp") to see some interesting function definitions, like canvas or webgl access drafts.

deosjr• 1 year ago

I recently shared a project I did using Hoot, by Spritely institute. It's Guile Scheme compiling to WASM, and it works really well! See https://spritely.institute/hoot/

Latest on my project, in case you want to try it out: https://deosjr.github.io/dynamicland/whiskers.html

feeley• 1 year ago

The Gambit Scheme REPL that comes with a tutorial, supports threads in the browser and has a JS FFI: https://try.gambitscheme.org

Gambit in emacs in the browser: https://feeley.github.io/gambit-in-emacs-in-the-browser/

mishoo4• 1 year ago

Hey, nice to see Ymacs is still used somewhere else than on my website :)

I don't know if you've been tracking it lately, but there was a big overhaul last year [1]. The version in your second link doesn't seem to work in current Firefox. If you'd like to update and need any help, please feel free to get in touch. (also, there's still some of your code in Ymacs :) although I kinda broke filename completion...)

[1] https://lisperator.net/blog/ymacs-reloaded/

Web Embeddable Common Lisp

Related Stories