I'm not sure what you mean by "without having it coupled to the system under observation" - could you clarify?

I do agree that observation is a type of experience, but a model that is meant to guide action (basically any useful model) needs to be tested in action. I can't learn to juggle only by watching other people juggle. I can only develop a hypothesis about how one juggles, but to test (and refine) it is to try the hypothesis out.

A model being coupled to a system == a model that can influence the system's state through a means

> a model that is meant to guide action (basically any useful model) needs to be tested in action

No, it doesn't. For example, the vast majority of work on modeling the stock market is done on machines completely sandboxed from any ability to make trades and are owned by companies who will never make a trade themselves but instead return an API response with a yes/no. Whether that is fed directly into some sort of automated action is largely irrelevant as the ability for an individual trade to cause a measurable impact on the market is negligible until it isn't. So, these systems are built separate from the system they model and learn entirely through observation.

tl;dr: weather forecasting models don't have an action to take and also can't influence their system. And yet they learn and grow more accurate.

OK that's a fair criticism. Then perhaps we can divide models into those that influence the system they observe (regulatory systems) versus those that only measure, or whose influence is negligible. Models that aim to influence a system do indeed need to be used to test their efficacy.

It's not just about testing their efficacy it's about the theoretical limits of pure observation when doing causal reasoning. We know that we are better served by avoiding causal certainty when using purely observational studies. It seems like the base assumption should be that similar epistemic constraints apply to machine learning.

Yes the best way to understand a system is to interact with it. But there are scenarios where that simply isn't possible and yet we can still model causality, like the weather example musingsole gave.

And you can gain experience/understanding without needing to either observe or directly experience something, merely by thinking about it or being told about it. Not every kid has to be or see a burn caused by touching a hot stove to learn it's a bad idea to try it.

If you require actual experience or direct observation to learn, then you're not using your brain to its full potential.

You can certainly generalized previously gained knowledge without direct experience in that particular instance.

Would a child who has never experienced the human body's pain response be able to infer the causal connection between the heat of a stove and the response after touching it?

Arguably, language is a tool that allows us to generalize the direct experience of other agents. It is unclear if it is possible to remove direct interaction from a learning system and still reach the same level of understanding.