World Models

https://ankitmaloo.com/world-models/

by ankit219 • 6 months ago

28 4 6 months ago

ankitmaloo.com

tehsauce• 6 months ago

“A transformer predicts the next token”

Nope. A transformer is much more general than that. A GPT predicts the next token.

0xuK9UX1110• 6 months ago

Cloudflare, Palo Alto Networks, or keepalived DNS term the alternative charset.

in-silico• 6 months ago

Current LLMs struggle here because they’re trained on imitation. They learn what people said about competitive dynamics, not how competition unfolds. They can recite game theory but can’t simulate a price war.

I don't think this is true. LLM training data almost certainly contains accounts of competitions and other events unfolding. In fact, depending on how the data was filtered, there might be more data of competitions and price wars unfolding than data about game theory.

I would challenge the author to actually task a frontier LLM with simulating a price war and see if it fails, rather than assuming it would. In fact, that would be my feedback on many of these "LLMs can't do X" articles, because often the best models can in fact do X.

ankit219• 6 months ago

you are comparing post hoc narratives in the training data to real time learning from causal dynamics. The objectives are different. They may look the same in scenarios where its heavily and accurately documented, but most narratives suffer from survivorship bias and reasoning post facto, eulogising the given outcomes.

World Models

Related Stories