HN user

GardenLetter27

646 karma
Posts4
Comments341
View on HN
How LLMs work 2 months ago

It's not just the architecture but also the data - the decoder only approach lets you train in parallel over blocks of text (no RNN serial waiting), that allows you train on much, much more data.

What? The UK is doing better economically than Germany right now.

And better than anywhere in Europe for Tech with Deepmind and Ineffable Intelligence there.

It still shares many of the same mistakes as Europe though - e.g. now having to buy Russian oil and gas again instead of using the North Sea oil, not expanding nuclear power, rampant welfare and council housing corruption, etc.

The LLMs just churns out non-idiomatic slop in any language.

It doesn't matter if the 800-line if statement is able to use pattern matching.

There's been a lot of progress on making coding agents able to solve problems when they can easily evaluate in a closed loop, we desperately need something similar for controlling complexity and using relevant abstractions.

Reinforcement Learning changes this though - remember Move 37?

The issue is you need verifiable rewards for that (and a good environment set-up), and it's hard to get rewards that cover everything humans want (security, simplicity, performance, readability, etc.)