HN user
I think so, specifically lossy compression though.
A modern version of the book would include an extra section in the 'Lossy compression' chapter - 'Text' (alongside Images/Video/Audio) that would discuss LLM's.
LLM's seem to be the weird interesting outcome of applying lossy (de)compression concepts to text instead of the audio/image/video domains where they have traditionally been used.
Strange that they are feeding raw audio in. Even in humans, there is a hardware transform to the frequency domain (the cochlea) before data is fed to the brain, effectively doing this part in the LLM seems inefficient.
My favorite is 'spongy', an underwater journey through a Menger-sponge fractal in 128 bytes:
https://www.pouet.net/prod.php?which=53871
https://www.youtube.com/watch?v=36BPql6Nl_U
I'm more impressed by the eerie beauty of it than the technical achievement, even if the code was orders of magnitude larger it would still be wonderful.
For a more laid-back chill 'rain' demo, have a look at "Rainy 32b" 32-byte x86 demo:
I had to read that twice, I first thought you meant 4 GB of _storage_...
Worrying about only having 4GB of RAM.... that's how I know I'm getting old I guess. Kids these days don't know they're born.
I've always wondered if it was perhaps the inspiration for the novel Neuromancer (2 AI's in different continents plotting to combine with each other to form a global super-intelligence)
We got both, so twice as good.
Ah, 'demand destruction', which in sounds positive in theory but in practice will likely mean poverty, famine and 'population reduction'.
You are assuming that a battery is full at 4.2V and empty at 0V. In practice a Lithium Ion battery is empty at 3V. You are also assuming the relationship between capacity and voltage is linear - it is not.
In the developed world we have 'energy slaves' instead; around 100 per person in the USA in the 2020's.
https://en.wikipedia.org/wiki/Energy_slave
This is arguably the reason why the Overton window has shifted towards the rejection of human slavery over the last century or so, with the growth of fossil fuel use.
Human slavery will thus likely swing back into fashion again in the future as oil, coal and natural gas run out.
That will happen on the day there is enough to go around.
Unfortunately I fear that time has long gone.
As long as the population is growing and/or the (fossil) energy is falling there will never be enough to go around.
I keep reading this, is it meant to be a joke? What is being won?
People are losing their minds at the prospect of oil availability dropping just 20% for a month or two with the closing of the Strait of Hormuz - even just this could collapse the global economy.
So yea, no way is oil stopping or even dipping slightly any time soon.
Just to mention the original was cited in the most recent Veritasium video:
"The Internet Was Weeks Away From Disaster and No One Knew"
https://www.youtube.com/watch?v=aoag03mSuXQ
(at about the 9:50 mark)
Economies die from lack of produce though.
I wonder if you could use the same technique (RAM models as ROM) for something like Whisper Speech-to-text, where the models are much smaller (around a Gigabyte) for a super-efficient single-chip speech recognition solution with tons of context knowledge.
I think this lack of 'G' (generality, or modality) is the problem. A human visualizes this kind of problem (a little video plays in my head of taking a car to a car wash). LLM's don't do this, they 'think' only in text, not visually.
A proper AGI would have have to have knowledge in video, image, audio and text domains to work properly.
Has overtaken Saudi Arabia as nation with largest proven oil reserves.
Although it is 'heavy' oil, the 'brown coal' of liquid fossil reserves (i.e. low quality).
The fact that such a fuss is being made about low-grade oil is a concern in itself.
At least HDMI is a 'low frequency' connector, often only ever plugged in once, as opposed to USB (or refueling a car)
I believe this is what "Cargo-Cult Programming" is.
Using 'copy' as a clipboard script tells me OP never lived through the DOS era I guess... Used to drive me mad switching between 'cp' in UNIX and 'copy' in DOS. (Same with the whole slash vs backslash mess.)
I believe this is why all modern digital watches use a 32768.0Hz crystal resonator, it's a power-of-2 frequency above the 20Khz top end of the range of human audio perception, to avoid the whole 'tinnitus on your wrist' thing.
The old 1980s computers also had no equivalent of flash storage though, the RAM had to store the program being run as well as act as scratch-space.
I believe youtube still uses 40 mel-scale vectors as feature data, whisper uses 80 (which provides finer spectral detail but is computationally more intensive to process naturally, but modern hardware allows for that)
that's not true, consumerism is only growing, people are not giving up anything in that regard.
The planet is getting trashed and 'the children' are doomed.
Individually We try and help, driving less, recycling and so on, but it kinda gets diluted by a billion Chinese moving into the middle-class and burning coal like there's no tomorrow.
Reminds me of the Pentium FDIV bug from 1994.... a half-billion-dollar 'ouch' for Intel were many hard lessons were learned.
Surely math libraries and optimizations have been a solved problem for the last 20+ years.
I would disagree. The dark ages were hundreds of years ago, the electric grid is much less than a century old. Plenty of countries have unreliable supply and rolling blackouts and have adapted to it or have just never became accustomed to the luxury of 24/7 electricity on demand. Being without juice is not the end of the world.
Bit ironic when the path to ketosis is fasting (i.e. not using products).
and yet an article about a 'safe' language has code examples full of 'unsafe'... wut.