You’ve probably refactored the ORM Data layer 3 times in that time period as ORM producers have a hard time figuring out the interface which they wish to provide you

Agreed. I'd say my point still stands because duplicated database calls affect performance and scalability and this can grind the project to a halt at a certain point. Also this refactor is much more dangerous/probably buggy.

On the other hand refactoring your data logic is just business as usual, and you will probably do it in both cases anyway.

caching is a very tricky area with tons of pitfalls. In my opinion, the ORM should not be caching. Let the clients (or any other layer) cache/clear based on their needs.