For example, one could drink one cup of olive oil daily and get ~2000 calories into your body. You would probably start suffering immediately on that type of "diet" because 1. your body isn't going to run on pure fat forever, and 2. your digestive system isn't going to process that amount of olive oil effectively.
Therefore it's not helpful to just say "Calories in, calories out" without actually looking at what you're eating.
What level of optimization are you getting by looking at macro nutrients and digestive absorption rates? More than 100 calories per day? I don't think it's worth the effort.
I think a big problem is that whenever dieting is brought up, people throw out the basics and try to be fancy, talking about insulin and macro nutrients and fat burning and digestive absorption rates. Why? It's unnecessary to complicate it that much and turns people away.
Then you have people starting Keto and being confused why they aren't losing weight, because they were told that ketosis is more important than keeping track of how many calories you're eating.
Similarly, when I'm bulking to gain muscle mass I just eat more calories than I usually feel comfortable eating. However that doesn't mean eating a ton of chips or soda or "empty" calories from alcohol, it means tons of actual food, whether it be chicken or pad thai or chocolate milk. IMO, it's much more important to be watching what you're eating when you're putting on weight.
Personally I don't like the idea of keto because it seems really hard on your liver and kidneys, and that doesn't seem worth it at all from a health tradeoff perspective.
It’s way more complex than the simple equation of calories in = calories out.
It may be complex, but if you're only changing the calories burned by 100 calories a day, then it hardly changes anything. At some point it matters, and at some point it doesn't.
So you can (and people do) use grams in/grams out to determine metabolic weight, but short of a lot of very specialized equipment, it's much easier for someone to have a rough measure of calorie burn at rest by choosing a standard value (or measuring it by the aforementioned metabolic test), and then approximating the amount of calories going in.
The psychology/physiology of weight loss is a complicated subject, but the thermodynamics isn't. It's very easy to just say "diet", but the hard part is the motivation, so while everyone is absolutely correct that counting calories is the "optimal" method of weight loss, it's not the strategy that optimizes results across the population.
More accurately: a calorie deficit is not "optimal" for weight loss so much as it is a necessary and sufficient condition for achieving weight loss. The problems are that accurate calorie accounting is harder than merely reading nutrition labels, and calorie counting on its own does not provide guidance on which foods to reduce or replace in your diet to achieve weight loss while staying otherwise healthy and sated.