News for hackers. Stuff that matters.

A comment on Too much efficiency makes everything worse. Parent comment

Re:Too much efficiency makes everything worse by jaschasd on Friday November 11, 2022 @09:50AM (#33561481)
A note that the datapoints you train on are part of the training objective. If you are using different data at test time than you use at training time, then you are measuring the wrong thing during training, the same as if you used a different loss function at training time.

Also -- as you say, feedback loops and non-stationarity make everything more complex, and are ubiquitous in the real world! But in machine learning we also see overfitting phenomena in systems with feedback loops -- e.g. in reinforcement learning or robotics, where the system changes depending on the agent's behavior.

(blog author here)

[ Parent | on HN ]
Re:Too much efficiency makes everything worse by c7b on Friday November 11, 2022 @12:24PM (#33563683)
Cool that you're responding here. Well, regarding robotics, I'm sure there's all sorts of problems when it comes to training models for them, but I'm not sure that Goodhart's law is one of them, unless you can give a concrete example. It's really geared towards social problems. Sure, some natural systems may also exhibit the kind of adaptive response that leads to breakdown of structural relattions (eg the cancer cells mentioned before may evolve to avoid detection by the AI), but that happens on completely different timescales.
[ Parent | on HN ]