Show HN: Llama-8B Teaches Itself Baby Steps to Deep Research Using RL

https://github.com/dCaples/AutoDidact
by diegocaples • 1 year ago
39 3 1 year ago

I've been tinkering with getting Llama-8B to bootstrap its own research skills through self-play. The model generates questions about documents, searches for answers, and then learns from its own successes/failures through RL (hacked up Unsloth's GRPO code). Started with just 23% accuracy on Apollo 13 mission report questions and hit 53% after less than an hour of training. Everything runs locally using open-source models. It's cool to see the model go from completely botching search queries to iteratively researching to get the right answer.

Related Stories

Loading related stories...

Source preview

github.com