mattmight

Original article author here.

One of the interesting things about the lambda calculus is its universality: by itself, it's a complete foundation for computation.

Here's a different old post of mine showing how to build the rest of the programming language, all in a miniscule subset of Python that is the pure lambda calculus:

https://matt.might.net/articles/python-church-y-combinator/

You can even extract recursion out of the Y combinator or the more primitive U combinator -- out of nothing but lambdas!

So, it's lambdas all the way down.

Another interesting thing about the lambda calculus is that it wasn't intended to be a programming language. When Alonzo Church created it, there were no computers to program.

Alonzo Church was trying to solve problems in the foundations of mathematics.

But, untyped lambda calculus has a "bug" that makes it problematic for mathematics -- the self application that enables recursion is a problem if you're a logician who cares about soundness, but it's fantastic if you're a programmer.

I don't think of functional languages as obfuscating. I think of them as terse and expressive. They let me most directly encode the model in my head as running code.

Good point -- JavaScript's syntax grew up since I wrote that article!

Thank you for the kind words! I keep hoping I can find the time to do more writing again. And, yes, I started that blog in 2008.

Wondering the same, and in somewhat different terms.

And as models shrink in size yet go up in intelligence and performance, I'm finding ever more life in older hardware.

When I got my M1 Max in 2021, GPT-3 was about 1.5 years old and it was SOTA.

Yet, that machine is now able to run models that crush with gpt4, and even compete with o1 (SOTA from about 1.5 years ago.)

The idea that I could run something like that locally would have seemed absurd in 2021.

Yet, if somehow I'd had those local models in 2021 on the exact same hardware, I would have had, by far, the most powerful AI on the planet -- and that would have remained true for the next several years.

I'm also noticing that the ever-improving smaller models I can run on this machine are crossing the "good enough" threshold for ever more tasks by the month.

I just don't need a frontier model for every task.

I have an M4 Max 128 GB RAM now, but I still find plenty of tasks to delegate to the M1 Max machine.

I don't know how far this can go in the limit in terms of packing more intelligence into smaller models, but older hardware, if maintained well, seems like it's going to increase the value it can deliver in terms of "intelligence per watt-hour."

Blog: https://matt.might.net/articles/

Main site: https://matt.might.net/

Nothing to feel bad about. Thank you for sharing that too.

My son’s life changed my own in profound ways, and even though he died four years ago, he is still changing my life in profound ways. I am always grateful for the reminder and to reconnect with the purpose that his life gave to mine.

That post also reminds me that while he was alive, I did the best I could for him under my abilities, and that’s all any parent can do in the end.

If you want to know more about his life, I wrote on it here: https://bertrand.might.net/

Happy to hear that! Thank you for sharing it with others. :)

So happy it helped! Thank you for the kind words!

Good for you!

Original author of the guide here. Wonderful to see these little illustrations still making the rounds. I first published them in 2010!

To those in the comments who mentioned you are just starting your own PhD: Good luck to you! And, I hope you, like I once did, find a problem that you can fall in love with for a few years.

To those just finished: Congratulations! Don’t forget to keep pushing!

To those many years out: You have to keep pushing too, but there can be tremendous value in starting all over again by pushing in a different direction. You have no idea what you may find between the tips of two fields.

"Think of it as an unholy combination of bash and Lisp" reminded me of Scheme Shell: https://scsh.net/

My Ph.D. advisor Olin Shivers created this. It's more pulling bash into Lisp than Lisp into bash.

It doesn't seem to be maintained anymore, but some of its ideas live on in Racket.

Same! I use ollama a lot, but when I need to do real engineering with language models, I end up having to go back to llama.cpp because I need grammar-constrained generation to get most models to behave reasonably. They just don't follow instructions well enough without it.

Glad to see you're still doing the right thing, JT!

(Article author here.)

This was the reference I used for TeX programming: https://pgfplots.sourceforge.net/TeX-programming-notes.pdf

I had a similar thought after finishing:if you wanted to really learn a new language in depth, you could solve all 25 days using that language.

Day 7 is still impressive!

Thanks!

I have a part two planned to comment more about the languages / days themselves.

This post was just getting too long to include it here.

Woops -- I'll take those down. Thanks for pointing that out.

I’ve never written any serious software using C++.

But, I did once learn how to do template meta-programming:

https://matt.might.net/articles/c++-template-meta-programmin...

Thank you for the kind words!

I missed blogging too, so I set a New Year’s resolution in January to write at least two blog posts this year. It had been over seven years since my last!

I still hadn’t written one by November, but now I’ve got three for the year.

I’m optimistic I can keep it up now.

Feel free to reach out. Happy to help.

Also, I've put much of what I've learned online:

https://bertrand.might.net/articles/algorithm-for-precision-...

I've been in your shoes, but for my son, who ended up being the first case ever discovered of his particular genetic disorder.

I'm happy to help.

I written down an Algorithm for Precision Medicine that abstracted the journey all the way from diagnosis to treatment:

https://bertrand.might.net/articles/algorithm-for-precision-...

My day job is now to help patients like your partner all day every day at the Precision Medicine Institute at UAB.

Feel free to reach out to us, and we'll be happy to craft research strategy and provide technical tips.

miniKanren definitely exploits a lot of the syntactic dexterity of Scheme/Racket/Lisp to do some really cool things -- or at least to express them more succinctly.

Yes, it is always possible, in fact, since logical arguments can always be rendered as proof trees.

Thanks for the kind words!

It's definitely a deliberate leap back to the old logical/symbolic era of AI.

Not mentioned in the article is that logic makes it much easier to emit "proofs" that end users like physicians can understand.

I'm the researcher mentioned in the article. We used Racket in part because before we transitioned into academic medicine, Will Byrd and I were in academic computer science with a focus on programming languages. Will was building up miniKanren to do lots of cool stuff in program synthesis and program analysis. When we jumped over, it felt natural to bring the power of relational reasoning to knowledge graphs over biomedical knowledge, and so mediKanren was born.

Thank you for the kind words -- and also for reminding me about the Godel hashing work!! That was one of the last things I did before the hard pivot into academic medicine.

Thank you :)

The Precision Medicine Institute that I now run produces mediKanren: https://github.com/webyrd/mediKanren

It's an open source logical reasoning engine (read: 1960's AI) for drug repurposing that we deploy routinely to help patients.

There is always a need for better relationalization of biological data sets that feed such tools too.

For example, SemMedDB is really showing its age for NLP of the scientific literature and yet it is still astonishingly useful for helping patients even as is.

Thank you for looking at the draft! Suggestions and corrections always welcome if you have any.

Thank you for the very kind words and friendship. I certainly hope Bertrand's legacy of science in the service of patients continues to grow with time.