rahen

Not very far off, the IBM 1130 was very much built around its punch-card reader. I've written a backprop in Fortran IV for the 1130 and while it works, it was tedious.

Add ten more years, and the IBM 801 could have been a CPU architecture good enough to scale all the way to the present day without emulation, unlike the 360.

What kind of virtualization do you use internally? I assume some kind of LXD fork? OrbStack really feels like a single-node Incus host.

The first AI winter was largely triggered by Minsky in a book he published in 1969, which mathematically proved that single-layer perceptrons couldn't solve non-linear problems. Favorite quote: "Our intuitive judgment is that the extension [to multilayer systems] is sterile."

Yet we had the computational power to run backpropagation in the 1960s and small Transformers in the 1970s (I'm the author of both):

https://github.com/dbrll/Xortran (backprop on IBM 1130, 60s)

https://github.com/dbrll/ATTN-11 (Transformer on PDP-11, 70s)

What was missing wasn't the raw processing power, but the ideas and algorithms themselves. Because funding and research were completely discouraged during the AI winter, neural networks research was left dormant and we lost two decades.

In the early days of machine learning (before the first AI winter), networks like this were often implemented and trained in hardware: https://en.wikipedia.org/wiki/ADALINE

That was the first thing that came to mind when I read "the smallest brain you can build". Nowadays, that "small brain" would likely be built on a breadboard using op-amps instead.

The Gamma 3, which competed with the 604, only had about 400 tubes as far as I remember: https://en.wikipedia.org/wiki/Bull_Gamma_3

Both the 604 and the G3 were bit serial to save components.

Scheme has a coherent and minimalist design, but its ecosystem and abstraction facilities feel too sparse for large applications.

When I started building a Lisp-based machine learning framework, Guile seemed like the right choice because it provides GOOPS and generic functions, yet I still ended up with a lot of boilerplate to compensate for the lack of a strong type system.

Scheme feels to me like C is to C++: not ergonomic for large-scale application development. Go is one of those languages that has both minimalism and productivity.

I was thinking of curated replay buffers, which would act like "dreams". To prevent collapse, the offline dataset would mix the new mid-term data with a baseline of anchor data (the original training distribution) so the model doesn't drift.

Also, we wouldn't train on the whole session. A separate critic module, like a reward model, would filter the KV cache to extract the high-value information, like a garbage collector before the LoRA.

That's just an idea though. Right now most research focuses on changing the architecture itself (TITAN, HOPE...) instead.

That's an idea I had a few months ago: after going through a compaction once the KV cache is nearing capacity, accumulate this knowledge into a dataset to fine-tune a LoRA during offline hours.

This would create a three-layer memory system:

- Stable long-term memory (initial base weights)

- Mid-term memory built from the compactions and replay buffers

- Short-term memory (KV cache)

Sleeping would just be a fancy term for consolidating and transferring information from one memory layer to another during offline hours. Maybe that's also what the brain does while sleeping.

Strictly speaking, this is very domain-specific and doesn't enable any performance that Triton couldn't already achieve (eliminating global memory round-trips via epilogue fusion is nothing new). The real takeaway is the design shift for LLM-driven codegen rather than handcrafted kernels.

LLMs are still bad at low-level hardware optimizations, but really good at high-level composition. Designing compiler abstractions with a restricted, composable API so an LLM can easily glue expert-written blocks together is a smart move. I suspect this will eventually become the norm for codegens as we move to agentic development.

There is ongoing research on neural cellular automata, as they seem to be a very efficient way to generate pretraining tokens: https://arxiv.org/html/2603.10055v1

Emacs Lisp is a descendant of PDP-10 MAClisp, which makes it one of the oldest Lisp dialects still actively maintained. Whether it's version 24.5 or 30.2 doesn't make much of a difference semantically.

Previous: https://news.ycombinator.com/item?id=47861270

Obvious slop still makes it to the front page of HN, and sometimes farms GitHub stars.

These posts also usually get all these glowing comments from users who clearly haven't checked the code. It's even worse when authors get busted and claim "Okay, Claude wrote it, but the design is mine" despite clearly not understanding the output themselves.

Unfortunately, that makes high-effort projects less visible. The SNR will probably keep getting worse until slop can be flagged on HN.

Ink is the Electron of text-based apps. I tried OpenCode out of curiosity, it routinely used hundreds of megabytes of memory.

I'll stick with Emacs as my TUI platform of choice, especially for tool-assisted development.

try making a DAW in it...

It would in fact work pretty well for a tracker.

Jank looks promising if you want a typed Lisp. It’s essentially native Clojure without the JVM: https://jank-lang.org/

In case you're into machine learning, I'm also building something similar - a tensor-first, native Clojure-like ML framework.

I wonder how they plan to implement that on decentralized social networks such as Nostr. I assume targeting big centralized networks such as X and Facebook is good enough.

Some subreddits have already taken measures against this: https://www.reddit.com/r/ProgrammingLanguages/comments/1sd66...

Same everywhere: avalanches of AI garbage and intellectual dishonesty. People claiming "I wrote this", then a look at the code shows massive slop and an author with no clue about the topic.

More worrying, this trend is creeping to all domains: "Nearly 75,000 tracks uploaded to Deezer are fully created using AI. That’s 44% of daily uploads, and more than 2 million per month. Back in June, the daily number was around 20,000."

https://www.vice.com/en/article/how-deezer-is-fighting-fraud...

Before WSL, the best ways to run unmodified Linux binaries inside Windows were CoLinux and flinux.

http://www.colinux.org/

https://github.com/wishstudio/flinux

flinux essentially had the architecture of WSL1, while CoLinux was more like WSL2 with a Linux kernel side-loaded.

Cygwin was technically the correct approach: native POSIX binaries on Windows rather than hacking in some foreign Linux plumbing. Since it was merely a lightweight DLL to link to (or a bunch of them), it also kept the cruft low without messing with ring 0.

However, it lacked the convenience of a CLI package manager back then, and I remember being hooked on CoLinux when I had to work on Windows.

Yes. The author mentions Claude for testing, but it was obviously used for the README and code as well.

This is a giveaway for AI generation, from the docstring to the terrible opcode dispatch (Claude sucks at assembly or low-level optimization): https://github.com/gizmo64k/soulplayer-c64/blob/main/src/cpu...

A human would use a proper dispatch table and wouldn't make excuses for a sloppy implementation ("Python is fast enough").

Besides, the author has an art and design background, which doesn't seem to match the deep knowledge of Transformers or assembly required for such a project.

A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.

I'm also writing a compiler and CS6120 from Cornell has helped me a lot: https://www.cs.cornell.edu/courses/cs6120/2025fa/self-guided...

I don't mean to be 'that guy', but after a quick review, this really feels like low-effort AI slop to me.

There is nothing wrong using AI tools to write code, but nothing here seems to have taken more than a generic 'write me a small LLM in PyTorch' prompt, or any specific human understanding.

The bar for what constitutes an engineering feat on HN seems to have shifted significantly.

Out of curiosity, what were those wonderful things you were hearing about the 11/34 back then?

Yes. The Cray supercomputers from the 80s were crazy good matmul machines in particular. The quad-CPU Cray X-MP (1984) could sustain 800 MFLOPS to 1 GFLOPS, and with a 1 GB SSD, had enough computer power and bandwidth to train a 7-10M-parameter language model in about six months, and infer at 18-25 tok/sec.

A mid-90s Cray T3E could have handled GPT-2 124M, 24 years before OpenAI.

I also had a punch-card computer from 1965 learn XOR with backpropagation.

The hardware was never the bottleneck, the ideas were.

I also have a working design for a small Transformer on the original Game Boy. It has around 4000 parameters fitting in the 8 KB cartridge SRAM, where the "saved game" is the trained model. A TI-82 with its 32 KB of RAM would be even more comfortable.

I encouter two main failure modes. First, the bipolar PROMs degrade at the atomic level, the metal ions in the fuses tend to migrate or 'regrow' over decades, causing bit rot. Second, the backplanes suffer from mechanical fatigue. After forty years of thermal expansion and structural flexing, especially when inserting boards, the traces and solder joints develop stress cracks. Both are a pain to repair.

https://retrocmp.com/articles/trying-to-fix-a-dec-pdp-1134-b...

The WASM GUI is probably the easiest way to see the Transformer in action on this machine: https://dbrll.github.io/ll-34/

There's also the original Tetris from 1984 to play.

That thing is a Tamagochi though, it constantly needs attention, pardon the pun. I did most of the development and tuning on ll-34 for that reason.

Thanks for reposting! I'm the author of ATTN-11. Happy to answer any questions about the fixed-point arithmetic, the PDP-11 hardware, or the training process.