HN user

mtdewcmu

1,439 karma
Posts9
Comments1,259
View on HN

I think that the common meaning of AI has changed since this was written. This book was written at least 14 years ago, long before anyone had heard of an LLM. Matt Mahoney incorporated neural networks in his compressors. Afaik they weren't pretrained. They were adaptive and made one pass over the plaintext, simultaneously learning and predicting. Decoding worked similarly.

If you go and (re)read what he writes in relation to AI, which I just did, it's about exclusion. He excludes "Universal Compression" as impossible, Kolmogorov compression as uncomputable, and then he gets to Artificial Intelligence. Artificial Intelligence is an appropriate way to model data, since data is created by humans with human intelligence. And, AI doesn't violate mathematics the way Universal Compression and Kolmogorov solutions do. So therefore, Artificial Intelligence is what's left. That seems to be the argument.

When decompressing, you need to reproduce the output of the LLM exactly as it was during compression, otherwise the decompressor would output gibberish. Can you count on the LLM being that consistent?

Fabrice Bellard did something with neural nets and a transformer model [1] that was very successful.

I suspect that LLMs wouldn't be ideal to use as compressors, because they are large, consume a lot of resources, and are constantly changing. You need the model to produce exactly the same output at encoding and decoding time, or else you get gibberish.

[1] https://bellard.org/nncp/

You could write imperative code in Racket without much difficulty, as long as you don't mind the exclamation points.

I started learning Common Lisp, but ASDF and Quicklisp threw me off. I couldn't tell if you were supposed to choose one or the other or they were used together. This might revive my interest in Common Lisp if I get around to reading it. But in the meantime I drifted off to Racket, which is relatively well documented and has extensive libraries and really unique features.

Black Hole Puzzle 2 years ago

Let's see if I can state this properly. The atoms of the ship will pass right through the event horizon like nothing. To see the ship, though, photons have to travel from the ship to your eyes. As the ship goes deeper into the black hole's gravity, the photons will "appear to" be getting slowed down by the black hole's gravity well, each photon more than the last. So, an outside observer would have to wait longer and longer to get the next photon. In fact, he'd have to wait an infinitely long time to get all of them. It's like an optical illusion, except that a real physicist would say it's not an optical illusion; it's time dilation, etc.

Black Hole Puzzle 2 years ago

A single photon can't be seen multiple times, right? So, if photon A goes into Alica's retina, then Bob can't see photon A. If a big, opaque object passes through the event horizon right in front of you, it would absorb or scatter the photons in its path, and you would not see them.

Black Hole Puzzle 2 years ago

I'm going to guess. From Bob's perspective, Alice's ship would still be able to block light. So he wouldn't be able to see what was ahead of him through the back of Alice's ship; Alice's ship would occlude his view.

If speed is an issue then you want it near the most commonly accessed data.

Yes. You expect the seek time to dominate performance.

The reason that the swap was faster when placed at the beginning is likely because the filesystem is mostly empty and so the allocated portion is at the beginning of the partition.

If the filesystem was near capacity and the files are distributed throughout, then you would expect the performance of the swap at the end and the swap at the beginning to start to converge.

The pilots aren't inventing anything on each flight. If they fly the airplane exactly the same way every time, that's a good thing. Writing new programs is not like that.