Basically what you allude to is the metabolic rate. Your body has a metabolic rate, and if for whatever reason this gets out of whack - eating unhealthy, gaining a lot of weight, it becomes very hard to come back to baseline. Hence the yo-you effect of most diets.