As a sort-of counterpoint I work at a non-amazon FANG where I am much more involved in model training and evaluation. I still have never needed to define a model myself, but it's much more complicated and far more ownership than someone throwing me a binary. I think that speaks much more to what's going on in Amazon *grins*.

It's about the same tier as BI dashboards. Huge breadths of data you don't produce but are nevertheless accountable for. All your stakeholders can easily stream-of-consciousness rattle off dozens of new metric/feature ideas or questions about the feature/metric values they're seeing over the course of 45 seconds, each one taking you tens of hours eyeballs deep in SQL to even begin to answer.

A job we would be glad to have replace by an AI for sure

Next up: why did the model say that? Which features contributed? Wouldn't it be cool if it also said... ? This other model responded slightly differently, can you take a look?

That is certainly coming soon.