Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings

by MMMercy2 • 3 years ago

50 7 3 years ago

HoshinoAI2• 3 years ago

It's glad to see the old technique is used for new models.

You may also learn from 7.33 dota update which uses a new ranking algorithm called Glicko.

zhisbug• 3 years ago

could you provide any reference? Is it a variant of ELO?

HoshinoAI2• 3 years ago

Valve listed some reason for making the change.

bradknowles• 3 years ago

Elo != ELO.

One is a rating system named after the creator, Arpad Elo. See https://en.wikipedia.org/wiki/Elo_rating_system

weichiang• 3 years ago

Surprised to learn StableLM is worse than plain LLaMA. link to their leaderboard: leaderboard.lmsys.org

circuit10• 3 years ago

I’ve heard that it’s really bad for it’s size

freediver• 3 years ago

This is a very good idea.

Related Stories