I have been interested in this and to my surprise I was able to find just a single project that does something similar: https://github.com/master-of-zen/Av1an

It has the ability to do trial compression (w/ scene splitting) and evaluate quality loss up to a desired factor.