Traditional ML is no stranger to measuring accuracy in terms of agreement with human evaluators.

A customer churn model or revenue forecast did have hard objective data (ground truth) to compare against - isn't it?

you seem to think all classical ML models were supervised, which isn't true. and we have metrics for unsupervised approaches as well