LDOs introduce their own inefficiency into the process so it's usually not worth the trouble.
Fun hack! Do you have a line on cheap LiFPO4 or is it just to get a longer run time between swaps?
I prefer not to use disposable batteries for environmental reasons. But with NiMH, a light designed for 3V will be dim, and an LCD will have low contrast. In that sense, the effective run time is zero, so NiMH is a non-starter.
I found some AAA-size LiFePO₄ cells, spacers, and a charger on AliExpress a few years ago.
I use LiFePO₄ to run my rear bike light and a wireless thermometer, both 2xAAA devices. The total capacity is reduced because half the space is wasted, but that's not a big deal if you're only recharging a few times a year.
I haven't tried this with remote controls, since the batteries last basically forever in those.
Another option is to use 3.7 volt lithium ion with an LDO regulator, but I haven't tried that yet.