In weight lifting you have this concept of "hard gainers," who are generally men who want to gain muscle mass / weight, but can't seem to, "no matter how much they eat." A common refrain from them is something like "I ate a whole pizza on my own, and still nothing!"
Well, the moment they start actually counting their calories, it's the same story you just told. Sure, they ate a whole pizza on Saturday, but also nothing much else that day, and they ended up short of their caloric needs, and then every other day that week was even less, minus the occasional splurge that also didn't bring their average up high enough.
This goes for people who "can't" lose weight too -- when they start counting calories they are surprised to find that they eat a shitload of calories they didn't know about, most of which are empty of nutrition.
One way to reframe the question of calories is to start considering calories per week, instead of per day. You start to get the picture either way--whether you're trying to gain or lose--when you're thousands off your goal by day 3 and realize what a big adjustment you'll have to make during the next 4 days to make your count right.
However I don’t think it’s purely genetic/intrinsic. At least personally I have seen that after a period of dieting my natural hunger reduced to about as much as I had been eating.
Anyway, I agree that these restrictive diets work in general by making it easier/necessary to count calories and eliminate some junk food. Really, eliminating dense addictive foods and counting calories is probably all the vast majority of people need to do to lose weight
That said, you're not likely to find the vegan equivalent of Twinkies at your local bodega, and that is worth something. But I'm guessing that a bigger factor is that choosing to go vegan or low-carb or whatever is tantamount to deciding to join a food culture that places a much higher value on making better food choices, and there's nothing quite like peer pressure for changing one's behavior.
I like the ingredient listings that say "cane juice".
But, as far as its culinary purpose goes, I suspect it ends to end up in processed foods because it's an additive you can use to sweeten the food while still being able to prominently claim "no sugar added" on the front of the box. I think legally you do have to cross your fingers behind your back while you do it, but it's not enforced.
I wouldn't be surprised if it's a trick that even works well on people who like to hang out on the internet talking about how fructose is the devil.
I get that the research is often done from a medical or public health perspective, not a biological one, so compliance is an important factor to consider, but when I talk to people, I find the conclusion they draw from reading these results in the media is not that most people in these studies fail to follow the diet. The conclusion they draw is that a lot of people will not lose weight if they follow the diet successfully. This happens over and over again every time a new study comes out, and the belief a lot of people internalize from it is that we really have no fucking clue what the connection is between what people eat and whether they're overweight, and we don't know what if any change in how they eat will result in them losing weight.
That leaves them vulnerable to unscrupulous and/or unwitting opportunists hawking diet systems backed by complicated theories about toxins or hormones or genetic types.
Simply writing down your calories consumption - with no limitations whatsoever - helps to lose weight.
I remember watching a video of two women, one skinny, one overweight, who were good friends. They said that the skinny one always eats way more food, but doesn't gain any weight.
They had a camera crew follow them both around and it was true that the skinny woman ate larger meals. The difference was all the time in between. The skinny woman didn't eat anything, while the overweight woman was snacking regularly.
The total calorie counts for the day were much larger for the overweight woman.