Daily metrics can vary wildly without affecting overall result.
My solution, and mine alone after trying oh so many different diets and strategies, went through regular aerobic exercise, at a rate of 3-4 hours per week. In my case it was swimming, because I like it; cycling or running would have had the same effect, I believe.
It worked. I lost almost 30kg over about 2.5 years. Why did it work? I don't really know. I can tell you that basal expenditure certainly has gone up, as I had to get into shape to swim 4-5km a week. Of course, swimming a few km introduces some expenditure of its own. However, I had already tried gym and other sports, which should have the same effects, with far worse weight control outcomes.
So, for me, regular aerobic sessions of longer than 45min did the trick. I don't know why, so I can't promise it will work for anyone else, but you might as well try it for a few months. Don't expect huge, rapid, losses in weight, just a very consistent downward trend.
At current trajectory I'll be at normal weight by summer. Two years ago and for many years before I was balancing on the edge between overweight and obese.
I *still* had to really starve myself to lose fat. My aerobic fitness was excellent, as was my flexibility. My core strength was awesome. Fat was another matter.
I'm not the only one like that.
I know people for whom cutting down eating works, and people for whom exercise works. I'm not one of them.
As a footnote, I couldn't lose 30kg without seriously injuring myself. I'm overweight, but not deeply obese. Losing 15-20kg would put me at an ideal fat level.
Yes, this is has been observed and studied. From this page: https://physiqonomics.com/constrained-energy-model/ the relavent source materials are:
https://pubmed.ncbi.nlm.nih.gov/26832439/ https://pubmed.ncbi.nlm.nih.gov/34453886/ https://pubmed.ncbi.nlm.nih.gov/34519717/
So, yes, it seems that the body "compensates" and controls expendeture as you've suggested.
Speaking in general, there are a lot of details to consider for CICO. When people count calories to below their TDEE, their estimate of TDEE is just that, an estimate, not their actual TDEE.
When calories are counted, how accurate are calories being counted? FDA regulations in the US call for the nutritional information on packaging to be +/- 20%, and when audited, the information is often found to be outside that ranges.
Every gram of glycogen stored in your muscles, the body also stores 4 grams of water. The cycle of exercise, fasting, caloric restriction, and refeeding and glycogen storage are going to cause large swings in weight that can make it hard to pull out trends until several weeks/months have gone by, but frequently people are micro-focused on daily weight flucuations and draw their conclusions about diet and weightloss from what is effectively changes in glycogen storage/water weight.
My hypothesis is that the equation that weight loss is related to calorie intake minus expenditure, but I think the conclusions one draws from it fail to take into account how much the body can control expenditure.
My body seems to naturally drift up in fat weight, very slowly -- probably around a pound per year -- no matter how much I eat. If I cut calories a lot, I can lose fat. After that, my fat level drifts back up to the old target weight pretty quickly, and then up slowly again. I'm pretty sure if I plotted my fat from the time I became an adult, it would be a pretty straight line, with short-term dips and recoveries for diets.
Losing weight sustainably would mean resetting that target somehow. I don't know how. Changing my diet and exercise levels don't seem to do much for fat levels (but do more for my general fitness and health).