HN user

abrax3141

2,542 karma

Hi. I'm Jeff Shrager. I mainly hack Lisp, but have dabbled in pretty much everything. I’ve been both a classic and modern AI hacker, but these days I’m hacking atmospheric chemistry. I was also an Adjunct Professor in the Symbolic Systems program at Stanford for over 20 years. DM me @ JShrager at gMail.

Posts126
Comments467
View on HN
github.com 1mo ago

Show HN: Hacker's Guides

abrax3141
2pts0
github.com 2mo ago

Recreation of the 1956 IPL-I version of the Logic Theorist theorem prover

abrax3141
26pts1
sites.google.com 2mo ago

Parry Parries Again: Reanimating the Famous Paranoid Chatbot (In a Day)

abrax3141
11pts1
www.mtc.com.au 3mo ago

Eliza a Play by Tom Holloway

abrax3141
23pts14
arxiv.org 4mo ago

Executable Archaeology

abrax3141
2pts0
github.com 4mo ago

Logic Theorist Reanimated in IPL-V

abrax3141
5pts3
timereshared.com 4mo ago

Reconstructing OPL: Joseph Weizenbaum's Online Programming Language

abrax3141
2pts0
docs.google.com 7mo ago

Workshop Annoucement: Retro AI: Archeologies of Artificial Intelligence

abrax3141
1pts1
news.ycombinator.com 10mo ago

Ask HN: GAN'ing Coding GPTs

abrax3141
2pts0
news.ycombinator.com 11mo ago

Patient Lisp Hacker Seeks Same for Long Walks Through IPL-V Code

abrax3141
7pts2
github.com 1y ago

Irony and PathOS

abrax3141
2pts2
www.computer.org 1y ago

ELIZA Reanimated: Restoring the Mother of All Chatbots

abrax3141
106pts30
www.youtube.com 1y ago

Reanimation of the original Logic Theorist, the first AI, in IPL-V [video]

abrax3141
34pts6
news.ycombinator.com 1y ago

Ask HN: Seeking an IPL-V Interpreter

abrax3141
52pts10
sites.google.com 1y ago

Eliza Reanimated the first chatbot restored on the first time-sharing system

abrax3141
2pts0
sites.google.com 1y ago

We're All Bozos on This Bus: Eliza and the Firesign Theater

abrax3141
18pts5
drive.google.com 1y ago

What does LETTER. do in LISTRD in FAP from 196x?

abrax3141
23pts7
sites.google.com 2y ago

Two new (well, old and old) *RUNNABLE* ELIZA adaptations

abrax3141
2pts1
sites.google.com 2y ago

Draft Paper Discovered in Which Joseph Weizenbaum Envisions ELIZA's Applications

abrax3141
125pts24
sites.google.com 2y ago

Real Conversations with the Original Eliza (Discovered in the Garfinkel Archive)

abrax3141
2pts0
sites.google.com 2y ago

Eliza on an ASR33 (Simulation)

abrax3141
2pts0
sites.google.com 2y ago

Kent Pitman's Lisp Eliza from MIT-AI's ITS History Project

abrax3141
66pts15
news.ycombinator.com 2y ago

Ask HN: Functional Fixedness and the Game No ChatBot Can Play

abrax3141
1pts0
whatever.scalzi.com 2y ago

When the yogurt took over (2010)

abrax3141
43pts8
www.youtube.com 3y ago

Training Video for Bell Labs' Holmdel Computing Center (1973) [video]

abrax3141
80pts29
apps.dtic.mil 3y ago

Three Reviews of J. Weizenbaum’s Computer Power and Human Reason (1976) [pdf]

abrax3141
28pts11
markcmarino.com 3y ago

Hallucinate This – An authoritized autobotography

abrax3141
38pts5
markcmarino.com 3y ago

Hallucinate This: An authoritized autobotography

abrax3141
1pts0
apps.dtic.mil 3y ago

Three Reviews of J. Weizenbaum’s Computer Power and Human Reason (1976) [pdf]

abrax3141
20pts1
www.mechanism-design.org 3y ago

Journal of Mechanism and Institution Design

abrax3141
50pts1

Earlier this week, having come across my ArXiv paper about reanimating the Logic Theorist, David Moews (https://djm.cc/dmoews.html) wrote me about an amazing piece of work he's done:

He built an interpreter for the original IPL-I pseudocode of the original Newell and Simon Logic Theorist, straight out of the 1956 RAND report (P-868), and then got it running!

(I'll call the version that David reanimated "LT1" or "LT56", and mine "LT5" or "LT63" because mine was rewritten for IPL-V and published in 1963.)

What makes David's work especially interesting, aside from pushing the RetroAI window back 8 more years (!), is that IPL-I was NEVER ACTUALLY IMPLEMENTED! It was hand-executed by Simon's students (and supposedly his kids!) simulating the imagined IPL-I machine. This actually makes the problem much simpler. (Not at all to diminish David's accomplishment!)

In the 1956 report Newell and Simon describe the process in something close to the cognitive operators they hypothesized underlay human theorem proving. This is essentially LT1: A (somewhat) high-level specification of the Newell and Simon theory of cognitive theorem proving. But because LT1 didn't have to actually run on a real computer, it could depend upon human intelligence and flexibility to handle the complexities of actual implementation that you need to do to make a real computer actually do the whole thing end-to-end. (Or, as in David's case, a pile of Python code, which, of course, Simon and Newell didn't have in the mid 1950s!) As a result, LT1 is a bit over 400 lines whereas LT5, which is what you get when Shaw had to actually nail down the complexities of actual implementation, is nearly 3000 lines!

Anyway, huge congratulations to David; well worth a look if you care about the prehistory of AI, Lisp, or theorem proving. His repo is here: https://github.com/dmoews/logic-theorist. The readme provides a lot of intersting and important detail that I've glossed over.

Lars Brinkhoff brought up PARRY, Kenneth Colby's 1972 paranoid chatbot, and Rupert Lane, who recently reanimated the original ELIZA, was able to (sort of) reproduce the famous RFC439 trans-internet conversation between ELIZA and PARRY. See the blog post for the amazing (and amusing) details! (Turns out that RFC439 used a Lisp ELIZA, not the MAD-SLIP original.)

"Inspired by true events, this world premiere by award-winning playwright Tom Holloway is a gripping psychological thriller about the birth of AI, and the ethical consequences of outsourcing our humanity."

I have successfully reanimated Simon, Newell, and Shaw’s Logic Theorist, the world's first AI[note], on an IPL-V emulator written in Lisp! This dribble shows it proving 11 complete theorems from the original Principia (or, at least from Simon and Newell's original inputs, which claim to have been from the Principia. I haven't checked.)

(It does break at the end, and I haven't started to track that down -- it may actually be my own runaway limiters that tripped it.)

[note] Or at worst the second -- depending on whether you count Arthur Samuel's first checkers player. But certainly LT is the first cognitive model, and first explicitly symbol processing AI.

Irony and PathOS 1 year ago

To "pre-teach" my son CS, I created a (highly) simplified code-to-bytecode-to-machine stack for a language called "irony", and (extremely simple) OS, called "pathos". (Hacker ethos: Hundreds of lines of code just to cache out a bad pun! :-)

The files "compiler.py" and "virtual_hardware.py" are the primary irony modules. Irony supports recursion, including mutual recursion, but nothing fancy. There's no lexer, so you need to put spaces between tokens. (Being a native Lisper, I hate lexers!)

compiler.py takes Irony source to byte code, and has a byte-code interpreter. virtual_hardware.py take the same byte code, but emulates actual hardware.

These are designed to go with an exercise where the student add FOR LOOP capability. Given the well-designed base, this is extremely simple, but requires understanding each step of the stack. ("compiler_for" and "virtual_hardware_for" are the solutions; The idea, of course, is not to give these to the student!)

(You'd think that it ought be the other way round: Start with for loops and then ask the student to add recursion, but it's much harder to add recursion than loops, so I put recursion into the base, and then ask the student to add for loops.)

The files "pathos.py" (which imports "irony.py") is a simple OS whose goal is to create, edit, compile, and run programs written in Irony. ("irony.py" is a combined version of compiler.py and virtual_hardware.py, but leaves out the byte code interpreter.) The "comp" and "exec" commands in pathos compile (to byte code) and then execute the reuslting byte code (basically assembly) on the emulated hardware. (See "pathos_demo.log".)

Everything here was written in about 5 total hours using a combination of chatbots. This wasn't as easy as I'd hoped. As many folks who use LLMs to assist in coding discover, they are bad at keeping track of even marginally complex or large projects, and don't deal well with conceptually twisty programming concepts. They were basically incapable of implementing recursion correctly, or generalizing to the desired level, and they kept losing their place in the series of steps. Eventually they would just seemingly lose track entirely and be unable to fix what turned out to be trivial errors, at which point I had to clear the decks entirely, reload the latest versions of the code base we were working on, and then re-explain the project and what to do next. In the end I had to give them nearly step-by-step guidance to get it right, and the way I wanted it to be, to be understandable and teachable. That said, to their credit, once I had the compiler and hardware emulator the understandable and teachable way I wanted it, I could feed those to the LLM and it was able to understand the code and make reasonable changes. For example, the entire FOR LOOP extension was done completely by Claude. I get that "pros" don't do it this way, but it's slightly fun trying to talk a chatbot into doing one's bidding, and being emacs-based, I don't have a magical code assistant built into my code editor. (I haven't even looked -- has someone already done that?)

Operating systems being conceptually simpler than programming languages, PATHOS was easier for the LLMs, and was almost entirely written by Claude, although based on several paragraph of detailed spec. And Claude was able to plug Irony into PATHOS (that is, create comp and exec commands) first try!

However, I then asked it to create a help command that simply listed all the other commands, which was nearly a trivial task, and it failed over and over, until I did one of those resets described above, and then it worked. (The attention model is just the wrong model of working and short term memory! Mark my words! [Yes, I do get my own meta-joke.])

I also asked the LLMs to make presentations. Those are here essentially "as is" and you'll see that although these are a reasonable start, they definitely aren't complete teaching materials.

Clarification: These papers are different enough that I don’t feel like I double dipped by posting both in HN, also the new pub is ... well, new. (Also, thank you for reminding me that I need to update the arXiv entry since it’s not been published!)

SHRDLU was written in Lisp, albeit a pre-CL list. So it was quite easy to bring up by writing macros that make CL work like older Lisps. We did the same thing with Bernie Cosell's ELIZA:

  https://sites.google.com/view/elizagen-org/eliza-clones
Although IPL is a direct Lisp predecessor, it can't be easily mapped to Lisp. Indeed, Lisp and IPL-V were contemporaneous for about 5 years, but Lisp was so much simpler and more elegant that it rapidly supplanted IPL. As a result, Ed Feigenbaum and I are probably the only living person who know IPL (and I only barely know it as I just learned it in the past few months!), whereas Lisp has been essentially endemic for 60 years! (I'm hoping I'm wrong that Ed and I are the only folks who know IPL. If you know of a native IPL speaker who I could talk to, I'd love you to DM me!)

Excellent point. We have many decently large programs. In fact, we have pretty much the jackpot. Ed Feigenbaum famously wrote EPAM, one of the first simulators of human memory. Not only is the program extensively documented in numerous published papers and reports, and not only is entire the code online (https://purl.stanford.edu/vq775jv4844) but if you look at that code, it's actually a compiler output, so it shows us how pretty much every type of IPL-V instruction compiles!

Re: Why do you need the old code if you implement your own interpreter anyway?

Several reasons. First, we don't need it, we want it -- it's a nice-to-have not a need-to-have.

It's nice to have for many reasons, none of them huge, but together they vie towards having it if we can find it:

1. What we're after is running the old AIs, not having IPL-V. We don't intend to write any new IPL code. The old AIs just happen to be written in IPL.

2. We can run the code without having to reformat it. There are numerous annoying nuances when recreating a very old language, not the least of which is the importance of card columns, which is annoying to "wrap parens around".

3. If we bother to write an emulator anyway, we can test our emulator against the real thing.

4. IPL-V being Lisp's conceptual machine code (one version of it, anyway), it should be easy to do so, so there's that as an engineering experiment.

5. Writing anything in Lisp is fun so it's an excuse to take a break from ... well, from pretty much anything else on my agenda, most of which do not require Lisp programming (although I often manage to squeeze some in anyway. :-)

Two good questions; Two different answers.

First, re "ironically":

Short answer: Lisp and IPL were competitors for the list&symbol-processing community of early AI. IPL invented a lot of what Lisp implemented in nicer syntax -- in effect, Lisp was an HLL for IPL. Lisp (obviously) won and now we're (ironically) emulating IPL in Lisp in order to emulate Lisp's underlying machine in the HLL that sits on top of that machine. (Actually - ironically^2 - SLIP won ... see below.)

Longer answer: This whole corner of language development was full of ironies. SLIP (Weizenbaum's approach to list processing) was a plug-in for Fortran (originally) and shortly thereafter, MAD. Just as Lisp wiped out IPL, it also wiped out SLIP. Ironically, today we do what Weizenbaum envisioned: Write in powerful general HLLs and add in specialized packages for things like list processing. So, in the end, SLIP won!

ELIZA Reanimated 2 years ago

Overlapping. Some are the same, but the earlier team was Lisp hackers instead of MAD-Slip & FAP hackers, which makes sense and is almost a non-overlapping set of individuals IRL :-)

ELIZA Reanimated 2 years ago

This has been picked up by a bunch of tech news sites. On Gizmodo someone wrote: “Hello World. OMG what the fuck happened while I was asleep?!!” Which is funny enough, but of course the next comment in response is: “How does that make you feel?” :-)

You want to be truly amazed, check out Newell’s IPL-V, which is a machine language for a stack machine, developed in the 1950s and used to implement the first AIs. It had every idea n Lisp except the parens.

Jeremy Braddock just (well, in October) published a scholarly yet extremely enjoyable book about The Firesign Theater, the brilliant 1970s-ERA audio improve group. Firesign often rolled technologies into their humor. In particular, the album "We're All Bozos on this Bus" has a long intricate piece that spins in and out of Lisp and ELIZA jokes (among many many others). The Firesign folks had had a demo of ELIZA, probably on a PDP-10. That would almost certainly have been Bernie Cosell's Lisp ELIZA, and we (ElizaGen.org) provided an image of the listing of that ELIZA, which appears as a figure in the book! (I'd tell you which figure and the page, but I loaned my copy of the book out. But if you are a Firesign fan, the book is, IMHO, fantastic throughout!)

Team ELIZA (and friends) are trying to make the original ~1965 MAD-SLIP ELIZA (discovered a few years ago in Jospeh Weizenbaum's MIT archives) run. A number of semi-overlapping groups are involved in this. Some have a 7090 emulator, some have a CTSS emulator on that, and some have various versions of MAD, SLIP, and the underlying FAP (IBM's Fortran Assembler) code that runs SLIP. (Yeah, I know. Go ahead and get the laughter out of your system. I did't name it!)

We (they) have this mostly under control, but for one single function, called "LETTER." that we cannot find the code for, and are having trouble divining the function of.

The code of concern is here:

  https://drive.google.com/file/d/1zAJ2zX9WaeR7Ui8F88ZNFZtI5Q-Ow5sb
If you search for "LETTER." there is a single call to it on card 00386. (MAD functions all end with "."; There are several occurrences of LETTER as labels, but only one function call.) This is in the function(entry) LISTRD. (000096) [This was in the days when functions could have multiple entry points in order to conserve memory.] and is used to load an array called KNOW (indexed by I). LISTRD is an s-expression reader (but into SLIP not LISP -- see below for LISTRD documentation).

Some potentially important factoids: The 7090 had 36 bit words and packed 6, 6-bit BCD characters into each. (You can see this in line 000356 where it appears to be packing 6 close parens (BCD 34k) into a word: CARD(I)=343434343434K (K for octal). Notice cards are (were) 84 characters in width and 84/6 is 14 -- the number 14 is used in multiple places as a loop limiter, so it's apparently scanning across cards (and the minimal comments and some variable names suggest this as well).

[Unfortunately, comments were used quite sparingly in those days because you had to punch them into cards!]

Again, the only function for which we don't have code is LETTER. So, okay, Sherlock Hackers...What exactly does LETTER. do and what exactly is it doing here?

Here's a SLIP manual:

   https://drive.google.com/file/d/1XtF7EM1KhwMPKsp5t6F0gwN-8LsNDPOl
LISTRD is documented on pg. 24 of the above manual.

(Possibly important is that lists had to start in column 1 with an open paren.)

Yeah. That's actually one of the things that makes this story so interesting: It took place exactly at the dawn of the inter(arap)net, and physically close to BBN (which was the arpanet implementor), so there was lots of sharing between MIT and BBN, in particular, McCarthy put a lisp on BBN's PDP-1, and Bernie Cosell's Lisp ELIZA, which he did from the algorithm in Wiezenbaum's CACM paper, was the one that became public, so everyone thought that ELIZA had originally been implemented in Lisp, which was wrong!

Not sure what you consider "production", but I used *Lisp extensively on the CM2 at Xerox PARC in the late 1980s. In fact, I published several papers based on this research.

We've just added two new runnable ELIZA adaptations to the ElizeGen collection. One is Steve Cherry's Apple II P-Lisp ELIZA, c. ~1980, put online by Ron Garret. (This one's been around for a while.) The other (brand new ... well, brand old :-) is one built in HyperCard by Brendan Bellina, and runnable through infintemac.org. I highly recommend trying Brendan's version as it speaks its respones and the natural stupidity of ELIZA, combined with :yoda mode and the hysterical voices absolutely kills!