Re:Book: The Emerging Science of Machine Learning Benchmarks
by fxwin on Thursday March 19, 2026 @08:07AM
(#47437946)
Why so snarky? I also didn't know who he was:
I'm a director at the Max Planck Institute for Intelligent Systems. Prior to joining the institute, I was Associate Professor for Electrical Engineering and Computer Sciences at the University of California, Berkeley. My research contributes to the scientific foundations of machine learning and algorithmic decision making with a focus on social questions.[0]
Also simply knowing of him doesn't answer the question.
Re:Book: The Emerging Science of Machine Learning Benchmarks
by loveparade on Wednesday March 18, 2026 @08:53PM
(#47433380)
Very cool book. I think a reason why ML has seen so much progress despite benchmark overfitting/abuse is that results are "regularized" by real world applications and the Lindy effect. Methods, or research, that abuse benchmarks aren't adopted by follow-up research so they tend not to survive. And they aren't adopted because people try them but then find out that they don't generalize to other/newer benchmarks. So the system works not because of specific benchmarks, but because of how the community as a whole deals with benchmarks.
I'm a director at the Max Planck Institute for Intelligent Systems. Prior to joining the institute, I was Associate Professor for Electrical Engineering and Computer Sciences at the University of California, Berkeley. My research contributes to the scientific foundations of machine learning and algorithmic decision making with a focus on social questions.[0]
Also simply knowing of him doesn't answer the question.
[0] https://mrtz.org/
1. It sounds like this book can be summarized in a practical blog post or a series of posts
2. Is using the term crisis so many times really necessary?
This is the actual link to reach the book. There is no navigation link back to the index on the shared link.