ChrysaLisp – A 64-bit Lisp OS with GUI

by bespoke_engnr • 9 years ago

306 121 9 years ago

I just saw this come up in an arpilisp discussion thread (https://github.com/marcpaq/arpilisp).

ChrysaLisp is an Assembler/C-Script/Lisp 64 bit OS. MIMD, multi CPU, multi threaded, multi core, multi user, and will build from source in ~2 seconds on most OS X or x64 Linux desktops.

Maybe this will let me live vicariously through someone who has actually built the project I always dream of creating in my free time.

dogprez2• 9 years ago

Maybe this will let me live vicariously through someone who has actually built the project I always dream of creating in my free time.

You too!?

bespoke_engnr2• 9 years ago

Heh, maybe you'll get a kick out of http://nand2tetris.org/ (if you haven't come across it already). I finally finished part 1 of the course (implementing the Hack machine) and it was one of the best and most challenging learning experiences I've ever had. My plan is to do part 2 (VM, high level language, OS) with Lisp instead of the language they implement. That's been the plan for the last 6 months, anyway...

tejaswiy• 9 years ago

I started on this path, got through the boot-up, interrupts etc. and to a simple shell. Once that was done, instead of targeting something simpler like malloc(), I wanted to get a VGA driver working. VGA drivers are notoriously hard and I really wanted my OS to have a UI so after a couple of weeks of studying and getting nowhere, I gave up on the whole project.

I think the one major thing that caused me to give up on the project was whenever it became time to refactor or to think about the big picture (for ex. where do the drivers live? what about a libc implementation - can you just get glibc to work on your OS? Or do you have to rewrite the whole of libc?), I found that I was just copying whatever linux had already done. The feeling that I wasn't really doing any original work and am making a shittier version of an existing system demotivated me for some reason even though this was just supposed to be a learning exercise. So some advice for anyone attempting this - be prepared to go out of your way to rewrite significant amounts of helper code that has nothing to do with actually building your OS.

kryptiskt• 9 years ago

For a libc on a weird OS I would recommend newlib (https://sourceware.org/newlib/), which is a libc for embedded systems and so has a well-defined way to implement the basic stuff that it needs to run.

d08ble• 9 years ago

Very interting, my dream too.

I made similar OS. ACPU OS run on mobiles/desktop/browsers/bare-metal, with multiuser p2p team development livecoding, time travel debugger, multiple libraries, startup in 30 seconds on iPad in development mode with synchronization, compile sources & symbol navigations, etc. Powered by ACPUL programming language.

Here is OS API:

https://github.com/d08ble/acpul-demo/tree/master/sys

Simple multiplayer DOTA game prototype:

https://github.com/d08ble/acpul-demo/tree/master/dota

Some demos:

https://www.youtube.com/watch?v=pk58kWIAqMM

https://www.youtube.com/watch?v=UsQklNVUDwM

https://www.youtube.com/watch?v=ubNfWarTawI

pjmlp5• 9 years ago

This is quite cool.

xmichael99• 9 years ago

For me, calling something an OS requires it to have it's own kernel ... maybe thats just me, but I think this is the 2nd project this month which has called itself an OS when in reality it is just a GUI.

tyingq5• 9 years ago

Looks like it might evolve that way...

"Could move to bare metal eventually but it's useful for now to run hosted while experimenting"

And this: https://github.com/vygr/ChrysaLisp/blob/master/sys/kernel.vp

phoe-krk• 9 years ago

It already has a kernel, except it uses the host OS's system calls instead of its own drivers. Once you have a runnable OS that runs as a separate Unix process in its own address space, it is actually possible to port it over to bare metal by providing all the modules that the OS would normally provide: memory management, graphics, device management and HAL, keyboard/mouse input.

See https://github.com/froggey/Mezzano for something that can run on bare metal.

mypalmike4• 9 years ago

port it over to bare metal by providing all the modules that the OS would normally provide: memory management, graphics, device management and HAL, keyboard/mouse input.

In other words, it can become an OS if you merely add to it the exact things that define an OS.

Santosh83• 9 years ago

Operating systems manage IPC/resources and facilitate I/O. This one does the former but as yet depends on another underlying kernel for the latter. It's just a matter of writing the drivers/platform specific code and it can stand on its own. So not that much of a stretch to call it an OS, IMO.

pjmlp5• 9 years ago

Yep, for example MS-DOS programming using only BIOS calls.

nickpsecurity• 9 years ago

I agree. These things aren't OS's. They're platform emulators that run using the low-level plumbing of actual OS's. Going from that to bare metal has been hard enough for hobbyists that many barely got anything done on it or didn't try. We should only call them OS's when they run on bare metal or something like VirtualBox.

disconnected• 9 years ago

I think the discussion of what constitutes an OS is mostly academic.

That said, to me, an operating system is basically any environment that allows me to run applications.

Usually, that implies a kernel, since you can't do much without one, but in some circumstances, you might not have a "kernel" in the traditional sense.

For example, LXC containers don't have their own kernel, but they provide an environment - binaries, libraries, etc - where I can run applications. I can rightly call that an OS, I think.

Edit: clarity

ZenoArrow• 9 years ago

"I think the discussion of what constitutes an OS is mostly academic."

An operating system is the software used to interface hardware and applications. If you miss out on control of hardware it's not an OS. At most, solutions like unikernels are an edge case, but anything with less direct control over hardware than unikernels are not. If the term 'OS' is too loose it becomes meaningless.

DSMan195276• 9 years ago

I think it's worth noting the distinction between OS and kernel is a bit messy. I think that you could argue that this together with a kernel creates an OS (In the same way that Linux + GNU creates an OS). You don't have to write the kernel yourself to create an OS, just have a kernel. It is kinda false advertising though - it's like saying you wrote an "OS with CoreUtils" when you just wrote a set of CoreUtils. It's still really good, but not what people are thinking of when you that.

That said, I agree with you that it kinda stinks this doesn't have its own kernel - it really is just a GUI without it. It surprises me that so many people are blowing it off as no big deal. Writing the kernel is not a simple detail, we're talking about a lot of functionality that is missing that takes a lot of work to get right.

tyingq5• 9 years ago

Does this roughly offer the same benefits that the old lisp machines provided?

I've never used one, but have heard people rave about how productive they were on them.

dsgfhasgjklas• 9 years ago

Doesn't look like there's anything like CLIM support, unfortunately.

lisper9• 9 years ago

No.

I expect this will be a fairly controversial comment, so I want to preface this by saying that I'm a big Lisp fan (just look at my handle). Lisp is my favorite programming language. I've been using it for nearly forty years. My first Lisp was P-Lisp on an Apple II in 1980. And I worked on Symbolics Lisp machines in the 1990s. They were very cool, but there's a reason they failed: general-purpose computing is infrastructure, and the economics of infrastructure are such that having a single standard is the most economical solution, even if that standard is sub-optimal. For better or worse, the standard for general-purpose computing is the C machine.

Because it's general-purpose you certainly can run Lisp on a C machine (just as you could run C on a Lisp machine). You can even do this at the system level. But Lisp will always be at a disadvantage because the hardware is optimized for C. Because of this, C will always win at the system level because at that level performance matters.

But that in and of itself is not the determining factor. The determining factor is the infrastructure that has grown up around the C machine in the last few decades. There is an enormous amount of work that has gone into building compilers, network stacks, data interchange formats, libraries, etc. etc. and they are all optimized for C. For Lisp to be competitive at the system level, nearly all of this infrastructure would have to be re-created, and that is not going to happen. Even with the enormous productivity advantages that Lisp has over C (and they really are enormous) this is not enough to overcome the economic advantages that C has by virtue of being the entrenched standard.

The way Lisp can still win in today's world is not by trying to replace C on the system level, but by "embracing and extending" C at the application level. I use Clozure Common Lisp. It has an Objective-C bridge, so I can call ObjC functions as if they were Lisp functions. There is no reason for me to know or care that these functions are actually written in C (except insofar as I have to be a little bit careful about memory management when I call C functions from Lisp) and so using Lisp in this way still gives me a huge lever that is economically viable even in today's world. I have web servers in production running in CCL on Linux, and it's a huge win. I can spin up a new web app on AWS in just a few minutes from a standing start. It's a Lisp machine, but at the application level, not the system level. My kernel (Linux) and web front end (nginx) are written in C, but that doesn't impact me at all because they are written by someone else. I just treat them as black boxes.

I don't want to denigrate ChrysaLisp in any way. It's tremendously cool. But cool is not enough to win in the real world.

[UPDATE] ChrysaLisp is actually doing the Right Thing with respect to its GUI by using a C-library (SDL). But it's trying to re-invent the compiler wheel (and the language design wheel) so that it can run on bare metal and "grow up to be a real Lisp machine" some day, and I think that aspect of the project is a fool's errand. There are already many Lisps that can run on bare metal (ECL was specifically designed for that). None of them have succeeding in displacing C, and I believe none ever will because the economic hurdles are insurmountable.

stcredzero• 9 years ago

It's tremendously cool. But cool is not enough to win in the real world.

Cool is synergistic with other qualities. On top of that, everything is contextual. Does anyone know of any particularly cool ClojureScript electron apps?

zokier• 9 years ago

But cool is not enough to win in the real world

But is winning in the real world really important? OPs question was after all "Does this roughly offer the same benefits that the old lisp machines provided", and as LISP machines did not really win in the real world the first time around, I don't think that is really prerequisite for bringing the benefits of a LISP machine.

gumby2• 9 years ago

Yes, but...

I basically agree with everything you wrote, but for two subsidiary points.

1 - it's a phenomenal development platform. I have on multiple occasions built large systems later deployed, for the reasons you discussed, via C ports. It didn't nullify the work done in Lisp: we learned a lot, and quickly; were able to do quick experiments, etc. But ultimately we needed to join the wider world.

2 - there is / has been a sea change in the software deployment model; it includes web browsers, even more rapid and more flexible development etc. Clojure is a good example and consequence of this change. It's quite possible that the circumstances that led to Lisps decline (though was it really ever "mainstream?") may swing the tide the other way. E.g. "Rapid development and realtime deployment of lisp-based microservices" to agglomerate some popular buzzwords.

FWIW: I was a developer from MACLISP to MIT CADRS, various Symbolics machines (at one point I had two 36xx machines on my desk with color monitors attached) and various D machines. Right now I'm doing my development in C++ though.

crististm• 9 years ago

How far do you think is the next local maximum from the current C one? I'm asking in terms of power of displacement of the status quo. Do you think it involves Lisp in one way or another?

Annatar5• 9 years ago

I have web servers in production running in CCL

WANT!!!

Please write an article about it and post a link to it here.

jlg23• 9 years ago

But it's trying to re-invent the compiler wheel (and the language design wheel) so that it can run on bare metal and "grow up to be a real Lisp machine" some day, and I think that aspect of the project is a fool's errand.

I'm not so sure about the actual intentions behind the development beyond "experimenting".

shalabhc• 9 years ago

The way Lisp can still win in today's world is not by trying to replace C on the system level, but by "embracing and extending" C at the application level.

Interestingly, another current top post on HN is HPAT - A compiler-based framework for big data in Python (https://news.ycombinator.com/item?id=15466829). This replaces C level languages with something like Python for a specific domain. That could be somewhere lisp wins as well?

mark_l_watson• 9 years ago

I also ran P-Lisp on my AppleII. Fun times.

A few years later I got a Xerox 1108 Lisp Machine at work, a cheaper alternative to your Symbolics, but very nice.

bitwize2• 9 years ago

The reason why Lisp machines failed is because, with compiler and CPU architecture improvements, "C machines" started trouncing Lisp machines in terms of performance, even at running Lisp code. Symbolics's last product was Genera running on a "C machine" architecture. This doesn't obviate your statements about library support, but I don't believe the dream of "Lisp from bare metal on up" is worth abandoning entirely.

abrax31412• 9 years ago

I think that a better idea was (is?) to create a LispM on cloud services, and just use it through the browser, like this:

   https://en.wikipedia.org/wiki/BioBIKE

"...[the authors] wanted to create a web-based, multi-user Lisp Machine,..."

jstewartmobile4• 9 years ago

I'd be more optimistic. Tools are better today. Machines are faster. Re-inventing the wheel isn't nearly as costly or involved as it had been. Look at how much has already been re-done and re-tooled in JavaScript. The same process is being replayed in Rust, Go, and Clojure as we speak.

As for the C machines, Moore's law already has a foot in the grave. Ten years ago, we would have just assembly-optimized any inner-loop bottlenecks and assumed that in a few more months Intel would get us the rest of the way there. Now I'm seeing more and more being pushed off into GPUs and custom ASICs. With clock speeds topped-out, the mainframe approach of pushing more into the hardware grows increasingly attractive.

That, and so much is written in other garbage-collected languages today that LISP-friendly hardware could benefit Python/C#/Java/PHP programs as much as it would LISP ones.

flavio817• 9 years ago

Does this roughly offer the same benefits that the old lisp machines provided?

This project is cool, but, in short, no.

Please take a look at this paper, which answers the questions of which benefits could be obtained with a Lisp OS. This goes regardless of if such OS has or does not has a nice IDE.

http://metamodular.com/lispos.pdf

This is a really good paper, recommended lecture.

rbanffy• 9 years ago

Could look more like a Lisp Machine...

steelbird• 9 years ago

Quick start to get the GUI going (not so obvious from the README):

$ clone https://github.com/vygr/ChrysaLisp.git ChrysaLisp

$ cd ChrysaLisp

$ sudo apt-get install libsdl2-ttf-dev

$ make -j

$ ./run.sh

To quit:

$ ./stop.sh

scottyelich6• 9 years ago

and for OSX?

antimass5• 9 years ago

see parent reply

antimass5• 9 years ago

For a recent macOS version, do:

0) Make sure you have a recent Xcode installed

1) Install the libsdl2 and libsdl2_ttf frameworks under /Library/Frameworks

   libsdl2: https://www.libsdl.org/release/SDL2-2.0.6.dmg
   sdl2+ttf: https://www.libsdl.org/projects/SDL_ttf/release/SDL2_ttf-2.0.14.dmg

2) git clone https://github.com/vygr/ChrysaLisp.git ChrysaLisp

3) cd ChrysaLisp

4) make (takes about 2s!!)

5) ./run.sh (shows full working gui. Amazing!)

schpaencoder3• 9 years ago

Getting SDL2 to be recognized under MacOS is no joy

gavanwoolery3• 9 years ago

One can debate whether or not this is a "true" operating system, but I'd like to coin a (new?) term: "virtual operating system"...not to be confused with emulation or virtual machines, but rather an OS that runs on top of a host OS. One obvious use for such a thing is reskinning an underlying operating system in a non-trivial way, i.e. completely changing the UI paradigms. Another use is allowing programmatic access to virtually everything.

craftyguy• 9 years ago

So a userspace application with ring0 access, or are you describing paravirtualization or containers?

gavanwoolery3• 9 years ago

It could technically fit any of those cases depending on the implementation, but the simplest path is probably building an application that disguises itself as an operating system. It could even be sandboxed to some degree and still probably suit most needs.

mikepurvis• 9 years ago

Robot operating system (ROS) is a bit like this. It sits on top of Linux or OS X but offers a message passing framework and some naming and type conventions for passing around data common in robotics, like transform information, laser scans, odometry, etc.

Sir_Cmpwn• 9 years ago

I would just call this a shell or a runtime environment. It's not an OS and I don't think we should co-opt that term. We don't call nurses virtual surgeons or construction workers virtual architects.

gavanwoolery3• 9 years ago

I agree there is probably a better term, but the differentiation would be that you are abstracted one level up from a native shell, and thus could probably more easily write a cross-platform "virtual OS" that could run on top of Windows, OSX, etc - and install it with the ease of running an application, as opposed to fiddling with more sensitive things.

notduncansmith• 9 years ago

I think is called "Emacs" ?

Jokes aside, you could probably call this a "desktop environment" and communicate most of the meaning. What remains is either a "virtual machine"[1], "runtime system", or something along those lines.

[1] https://en.wikipedia.org/wiki/Virtual_machine

Annatar5• 9 years ago

Here is proof that Rust is unnecessary. Lisp comes from the past to save the future from itself.

jstewartmobile4• 9 years ago

We can start a LISP evangelism strikeforce and bomb the Rust threads the way the Rust guys bomb the C ones.

flavio817• 9 years ago

Where do I sign?

rurban2• 9 years ago

You will be heavily downvoted by the Rust zealots, without any argumentation, even if your arguments are in their official docs. I try from time to time because their discrepancy of marketing talk to delivered features is even worse than with Perl.

Comparing to this lisp os its a bit unfair though. This is an async distributed actor model os with various practical shortcuts. Refcounted with manually breaking up cycles. The lisp is very rough, but extremely fast. No linked lists (i.e. cons cells), just vectors. (Very good idea btw, same as in potion). A better macro assembler with message passing OO (smalltalk, Erlang but better than beam). I still want to see the other TAOS features though: selfrouting network, dynamic binding

Annatar5• 9 years ago

They don’t just crash the C parties, anything or anyone that’s not on or who is not on Rust is fair game as far as they’re concerned. If you aren’t singing along with the Rust evangelism strike force party line and voice any kind of dissent expect severe repercussions justified by “lack of arguments”, even when the arguments presented are their own damning statements or criticise concrete code and syntax.

xenophonf2• 9 years ago

I wonder how difficult it would be to move an existing Lisp to bare metal, like SBCL.

derefr• 9 years ago

Less ambitious: has anyone yet built a Lisp Machine unikernel (i.e. a "Racket on Xen", like "Erlang on Xen")?

xenophonf2• 9 years ago

Would such a thing be a stripped down Linux or NetBSD kernel that ran (e.g.) SBCL instead of init?

scottyelich6• 9 years ago

I'm looking for a bare metal boot OS that has internet capabilities -- lisp or forth based (ONLY). Any pointers would be greatly appreciated.

nicklaf2• 9 years ago

Firmworks has a open source implementation of OpenFirmware which is part of the OpenBIOS project, can be used with Coreboot, and implements TCP/IP with various protocols including HTTP, all in Forth.

http://www.firmworks.com/ofwfeatures.htm

http://GitHub.com/openbios/openfirmware

scottyelich6• 9 years ago

Thank you.

avodonosov3• 9 years ago

A Common Lisp one: https://github.com/froggey/Mezzano

nicklaf2• 9 years ago

Interesting. Looks like there aren't many drivers, though. I only see stuff about running it through QEMU or Virtualbox.

Maybe somebody can try running it from OpenFirmware (see my my sibling post to yours), which has been ported to lots of platforms, and has lots of code one could borrow from in the way of drivers. Of course at the end of the day I guess the point of QEMU / Xen is to use Linux as a device driver.

bluefox• 9 years ago

A bit of a stretch to call this a "Lisp OS" yet. If you look at the project's history, you can see that it was written in nasm, and at some point the author "slapped some parens around" and took steps to make it into an actual Lisp dialect, but it's not there yet, as it lacks many of the features you'd expect from a Lisp. Also see README_LISP.md.

galaxyLogic3• 9 years ago

JavaScript has been called "Lisp in C's clothing". What it lacks is macros basically but you can get a lot of practical work done in JavaScript and it is as mainstream as it gets. So how about a JavaScript OS/Machine/Hardware anybody?

nv-vn2• 9 years ago

JavaScript is really nothing like Lisp. Macros/code is data and simple syntax are basically the defining features of a Lisp. Apart from these, JavaScript's type system is totally different (way weaker, based on objects but nothing like CLOS), JS isn't based on lists/pairs as the primary data structure, JS emphasizes imperative programming, etc. I don't know where this myth came about (probably because rumor that Brendan Eich initially intended to use Scheme instead of JS, although this never happened).

unkown-unknowns• 9 years ago

rumor that Brendan Eich initially intended to use Scheme instead of JS, although this never happened

I wonder what'd have happened had he done so. Would it have grown to the size that JavaScript is at today, or would it have lost to something similar to JavaScript that some other browser would have implemented?

BrendanEich• 9 years ago

Rumor? https://brendaneich.com/2008/04/popularity/

flavio817• 9 years ago

JavaScript has been called "Lisp in C's clothing". What it lacks is macros

No, it lacks a lot of things in comparison, and if it only lacks macros, then it's lacking a lot as well.

Considering, for this case, "Lisp"="Common Lisp", this is what Javascript lacks:

1. Strong typing. CL is very strongly typed. Javascript is very weakly typed. Typescript doesn't fix this 100% because the runtime is still javascript, and thus the running code is weakly typed.

2. Code should be a first class citizen; code can be processed/sliced/diced/handled as easily as data. That is, code is data so most of the functions that apply to data, also apply to code. This doesn't happen in Javascript. Thus in Lisp metaprogramming is very easy and can be applied in many ways, greatly enhancing the power of the language, making it a "programmable programming language."

---------------

I could stop here, because these two above are strong, significant differences enough to tip the balance towards CL. But there's more:

---------------

3. A really good object system. CLOS might be the most advanced OOP system out there.

4. Really good numeric support. Javascript numbers are internally floats. In CL you have a full numeric tower. Complex numbers? check. Arbitrary-length numbers? check.

5. Image-based interactive development

6. A killer exception handling system: The condition-restart system.

7. Speed. CL can execute significantly faster than JS and can approach C/Fortran speeds if needed (by using tricks.)

8. Optional static type checks, and type declarations for increased performance if needed.

9. In javascript, you code so you define what happens at runtime. In CL, you can define when is your code going to be run: "at read time" versus "at compile time" versus "at runtime". So there are three "times" you can leverage!!

There are even more differences, to be honest.

galaxyLogic3• 9 years ago

I can only agree that Common Lisp is in many ways a much better language than JavaScript. But JavaScript is much more popular and it has evolved into a lisp direction, EcmaScript 6 has "arrow functions" which are basically lambdas.

One question, you say "CL is very strongly typed". Then you also say "Optional type checks". Are you saying CL is optionally very strongly typed?

alekratz• 9 years ago

So how about a JavaScript OS/Machine/Hardware anybody?

Please, no.

avodonosov3• 9 years ago

Not exactly a Lisp OS - asm and C beneath. Compare to https://github.com/froggey/Mezzano

colesantiago• 9 years ago

Very cool, interested in Lisp but I would like to know if any HN'ers would provide me some insight into why Lisp is so popular?

Does anyone use it in production? Is it comparable to JS?

DonaldFisk• 9 years ago

Lisp is popular? Where?

It was almost mainstream in the 1980s when there were four Lisp Machine companies (Xerox, Symbolics, Texas Instruments, LMI) and symbolic AI, particularly expert systems, were used. Not so much now.

I still use it. If you know it you can be much more expressive and productive in it than in other mainstream languages, and when there's something you need that's missing from Lisp, you can extend Lisp to include it.

There are reasons it's fallen from favour, some of which lisper has given here. It's a mistake to languages necessarily succeed or fail on their merits.

flavio817• 9 years ago

I would like to know if any HN'ers would provide me some insight into why Lisp is so popular?

In the late early 80s it was rather popular but it required expensive specialized hardware and software. Or you could execute it on more common hardware, but memory requirements were pretty high, which adds to the expense.

It's not like in 2017 where you can have many excellent Lisp implementations for free and they run blazingly fast on your machine, without needing more memory than what you have already.

It also has something to do with the triumph of UNIX over the Lisp machines

rurban2• 9 years ago

So this looks like basically the followup project to the TAOS os which was ways ahead of its time. https://news.ycombinator.com/item?id=9806607

autcrock• 9 years ago

What this reminded me of most is the Connection Machines, which allowed you to configure the interprocessor connection topology. It had specialised C, FORTRAN and Lisp languages. They were also the best looking computers I ever saw!

ChrysaLisp – A 64-bit Lisp OS with GUI

Related Stories