Traffic control systems, computer hardware design...these all incorporate the same combinations of depth and breadth of thinking. I think it's wise to keep an open mind and ask whether it's really necessary to exclude a given model or entire set of them.
Goodhart's law is an adage often stated as "When a measure becomes a target, it ceases to be a good measure"
Who exactly are we criticizing for doing this? Seems a bit like a straw man argument tbh. I'm not even sure how that relates to the more general concept raised above, of a single model of reality being reliable.
This is true even if what’s being observed is by a simulated entity, the observation exists outside of it’s internal models and is indirectly related to things in the underlying reality. Ie, physical states of various bits of physical RAM.