Before COVID were were shopping to upgrade, so we're not going to put any more upgrade money into this house, but if we were going to stay I'd absolutely make changes in this.
Re: the downstairs, unfortunately it warms up quickly on a hot day, and the structure of the house is such that the downstairs room's heat doesn't really have an easy path up.
The other factor is that, in a NORMAL, non-plague year, I'm alone in the house during the work day. If had perfect control of the house climate system, I'd let the upstairs drift to 80 or 82 while keeping my office at 76 or 78, and not start trying to cool the rest of the house until after hours when my wife gets home. Same with heating: just heat down here during the day, and warm the rest of the house in the evening.