HNGlance

Top Best New Show

Baba Is Solved by Fable 5 and GPT-5.6 Sol, but at what cost?

https://quesma.com/blog/baba-is-bench/

by stared • 5 days ago

3 4 5 days ago

QU

Related Stories

Loading related stories...

Comments

4

sscaryterry• 5 days ago

I'd love to see Kimi K3 here :)

stared2• 5 days ago

It is on the way! Sadly, this Kimi K3 API rate limits are devastating, at least on the OpenRouter. So far it solved all The Intro levels (as expected) - I am much more curious how it works with The Lake.

dash2• 5 days ago

Wouldn’t solutions be in the training data? The game is quite old.

stared2• 5 days ago

See the "caveats" section.

It was also our initial assumption, but to our surprise there were no signs of models "knowing" the solutions (we investigated all trajectories). Compare and contrast with SWE-Bench Verified, for which models suddenly generate solutions, or at least, have substantial hints, https://openai.com/index/why-we-no-longer-evaluate-swe-bench....

Source preview

quesma.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms