This has an odd implication that the metric you care about will be static.

Consider your thermostat. What do you optimize for daily with it? Is it the same in winter as summer? Why not?

To that end, why do we think there is an "optimum" setting?

In the case of a thermostat you could define a goal temperature for a given time range (e.g. 18C during the day, 10C at night) and then your loss is, say, the squared difference from goal and current temperature.

But I agree that in many cases it is difficult to define a goal.

My point was supposed to be that your goal will vary across time.

Week of vacation at home? Probably optimizing for comfort. Standard work week where you aren't home all day? Cost will be big.

Of course, getting costs in there could be a little tricky. Not impossible, just likely a ton of heuristics. And there is some level of stress where maintaining a temp might be cheaper than quickly getting to it, depending on how far you will vary based on being off. Which is just a long way to say it is easier to keep the house warm on warm days. :)

If you drive a lot, this would be akin to trying to find the optimum spot to hold the gas pedal. Sounds like something you could look for, but odds are high that it has to vary based on circumstances.

So, in the end, we aren't looking for a static parameter, but a system of dynamic parameters to keep in tune.

The usual way a thermostat works is the user specifies goal temperatures in time ranges and the thermostat tries to achieve them. E.g. our thermostat allows, IIRC, 4 different segments per day so we have morning (warmish), daytime (minimal heating), evening (warmish), night (minimal). The optimization problem them becomes one of achieving these goals.

To account for the dynamics / circumstances you can augment the input the thermostat receives from just the current temperature to also include information about the rate of change of temperature (velocity and acceleration). This is the idea behind PID controllers and it allows things like easing off the heating if the house is warming quickly so the thermostat doesn't overshoot the goal.

You're right that having an optimization system turns the problem from one of solving the problem to one of defining a metric that we directly care about.

For battery optimization use cases, e.g. screen brightness, one metric that works is minimising the amount of times a user goes and changes the settings (with reinforcement learning), and I think that could definitely be a starting point for thermostats.

Obviously, you also care about other things in a thermostat, e.g. energy usage (eg when you're not home), and maybe the amount of changes a user needs to make is not sufficiently tight supervision for a thermostat, but it's a starting point that does take personalization into account.

Optimization against a kinda-good-enough loss function is better than no optimization at all, which is what usually happens with the giant pile of random program parameters that every program relies on.

Can be. Can also be just as wrong.

In particular, my challenge is that in most programs the "loss" function that is of interest to the user is actually not visible. Why I picked the thermostat example. It doesn't know if you are there or not. Or what the cost of electricity is right now. Or what the average temperature outside will be in 2 hours.

It can be made aware of all of these things, but that will itself be a large part of the complexity. Such that "just optimize this loss function" is not wrong. Just, I challenge how much it helps.

I think it can definitely help in the thermostat example. This was literally the original value proposition of the Nest Thermostat, and they went to some length to demonstrate that they were effective, using both internal and third-party studies. The claim is an average of $130 in energy savings per customer per year... Which translates directly into a positive effect for carbon emissions.

https://storage.googleapis.com/nest-public-downloads/press/d...

So, if you throw a whole lot of effort at it today, it can work well! I just want this sort of thing to be more automatic.

Exactly, I feel there is stuff there. I just don't feel it is as simple as a loss function. It generally is more capability and more data.

The nest is odd, to me, as I live without an air conditioner. So, at best, it can keep things off when we are not at home. But, you can't just turn on when we are home, as at that point it is too late to hit and maintain comfort before bed.

So, we are back to adding more to my house just to support the data and control this is looking to bolster. Which is almost certainly a net negative, all told. :(