Thanks for the suggestions! This is definitely something we'll be looking at.

We're currently working on providing a more extensive interface to show users a variety of performance metrics of the models they're running. Having efficiency metrics would be a great addition.

I think additionally an important facet of these tests would be providing clarity on the details of the tests to make them reproducible. I find that sometimes reported stats don't quite translate to real-world experiences. It can feel like results are presented using the workloads that look best on a system, so a standardized/reproducible approach would be best.

We're always keen to chat to as many users/experts/academics/enthusiasts as possible. Please feel free to reach me at [email protected] and we can set up a time to meet!