Yeah, the oversimplification sounds great, except it is a homeostatic mechanism, people extract and expend calories at different rates, and appetite responds differently to different foods. Energy in vs. energy out matters less if certain types of food aren't sating hunger effectively. First you say it "is that simple" then go on to say "except for..." and list exclusions.
Whenever I feel unsympathetic, I imagine trying to "breath less" for several years.
It is only that simple if you assume that all macronutrients are metabolized in exactly the same way which is not the case (example: fat does not stimulate insulin secretion, carbohydrates [for the most part] do, insulin is the primary regulator of fat storage). Treating the human body like a black box like that is vastly oversimplifying the biochemistry of the digestive process to our detriment.
See http://www.bodyrecomposition.com/fat-loss/is-a-calorie-a-cal...