Book: The Emerging Science of Machine Learning Benchmarks

https://mlbenchmarks.org/00-preface.html

by jxmorris12 • 4 months ago

138 12 4 months ago

mlbenchmarks.org

lazrgatr• 4 months ago

A little rule I live by is that if Moritz Hardt writes it, I will read it

TrainedMonkey• 4 months ago

Why is that?

kaycey20222• 4 months ago

You honestly don't know of Moritz Hardt?

trostaft• 4 months ago

If I'm recall correctly, this was also a keynote at MDS24? That was also a great talk, Hardt is an excellent speaker.

loveparade• 4 months ago

Very cool book. I think a reason why ML has seen so much progress despite benchmark overfitting/abuse is that results are "regularized" by real world applications and the Lindy effect. Methods, or research, that abuse benchmarks aren't adopted by follow-up research so they tend not to survive. And they aren't adopted because people try them but then find out that they don't generalize to other/newer benchmarks. So the system works not because of specific benchmarks, but because of how the community as a whole deals with benchmarks.

salberts• 4 months ago

Read the preface.

1. It sounds like this book can be summarized in a practical blog post or a series of posts

2. Is using the term crisis so many times really necessary?

erdemo• 4 months ago

https://mlbenchmarks.org/

This is the actual link to reach the book. There is no navigation link back to the index on the shared link.

NeutralForest• 4 months ago

Thanks, I had to do the same.

pakapica• 4 months ago

added to my reading list :)

Book: The Emerging Science of Machine Learning Benchmarks

Related Stories