While "Good Calories, Bad Calories" (great book, hate the title) is generally known for being a "low-carb book", actually the bulk of the book is an in-depth examination of exactly how we got to the point where "low-fat diet = good" is not just a dominant hypothesis but borderline religion. If you are truly interested in an answer to that question, this is it in book form.

The short answer is, indeed, politics, but I actually blame scientific politics rather than traditional government politics. The latter enabled the former to really spread its influence, but had the strongest personalities in the dietary movement agreed that fat is harmless, the government would never have ended up rubber-stamping the low-fat dietary dogma. The longer answer is, as sliverstorm said, the look-under-the-lamppost problem; we got a couple of cheap measurement tools early and then extrapolated the hell out of the small, small, oh-so-small view of the complex system we had.

Incidentally, we do that all the time and continue to do that. Almost every article I see online about how X is bad for you or how Y is good for kids can be traced back to a study that boils down to a point sample in the n-dimensional space of like; such things are often only marginally better than nothing, all of our best scientific controls notwithstanding. The next great revolution in science as a whole is going to have to be figuring out some way to get a holistic view of more things, we're drowning in data points that often add up to nothing.

Actually, I just bought "Good Calories, Bad Calories" about an hour prior to finding this!

It's not all politics. What I've read so far says it happened because scientists figured out how to measure blood cholesterol, and learned that fats increased it. That was (iirc) the core of the 'health disaster'.

According to "Sugar: The Bitter Truth", a YouTube video previously featured on HN (http://www.youtube.com/watch?v=dBnniua6-oM), low-fat diets were mainly a result of a key and influential study that failed to eliminate confounding: the diet it looked at did increase cholesterol and was high-fat, but it was also high in grains/sugar.

In an organic chemistry class in '85, a (admittedly fairly bright and clued in) professor described to us that while people were worrying about saturated fats, research was starting to show that partially hydrogenated oils (aka "trans fats") might well pose a much larger health risk.

It only took 20 years to work its way into the public/popular media. (In the meantime, a lot of margarine was sold -- and a lot of angioplasties and bypasses.)

Running was supposed to destroy one's joints. Now, we learn that when it is not overdone (too much in too short a period of time or with too little prior conditioning), it actually strengthens joints.

I've become increasingly skeptical of new health effect claims, particularly when they aren't backed up by detailed, specific (to the aspect of health under consideration), and longitudinally significant evidence.

(Oh, and remember when vitamin D was considered one of the less significant vitamins (once you had enough in your diet to prevent rickets)?)

[Meta: Sigh, I have indeed become that "grumpy old man".]

[Meta: Sigh, I have indeed become that "grumpy old man".]

I don't know. Assuming you went through the normal college course straight out of high school I'm ~12 years younger than you but I've been coming around to the same conclusions. Personally I think between the internet and some other things it's just that we're getting the data now that says that we need to tune our trust of science down a bit. I still think it's the best way of learning there is, it just isn't as powerful in practice as we'd like.

I remember thinking in the 1990s that as I read about the history of various sciences, the only fields where the dominant consensus didn't spend long periods of time being dead wrong were fields too young to have a dominant consensus. (Perhaps not in those words at the time, but along those lines.) I recall seeing the casual assumption that all that stuff was in the past, and wondered what major sciences were actively wrong. It turns out that in fact they weren't all exactly right and indeed I think that some entire major fields were wrong at the time.

(Bringing up the interesting question of who is wrong today? Back then I could mouth the words but hadn't really internalized the idea, but now I seriously wonder, who is wrong today in a way that I don't even suspect? The odds of it being "nobody" are basically zero.)

"we need to tune our trust of science down a bit"

Science != popular reporting/understanding of science.

But the point about some of our understanding being wrong is right on the money.

I am not talking about popular reporting of science. I am talking about science.

Recently a paper went by on how something like 90% of medical studies are not able to be replicated in follow-up studies. If you really understand science qua science (as opposed to science-as-religion-substitute), you should understand this isn't a minor interesting tidbit to be discarded, this is a dagger to the heart of the field. Other fields seem to be having similar problems.

I reiterate, science is still the best we've got and I don't anticipate a replacement. But the evidence strongly suggests that the quality of the real science being done is a lot lower than it may appear at first glance, or what some people would have you believe.

Coincidentally, this just came up: http://news.ycombinator.com/item?id=1199230 It is almost exactly what I mean. Unfortunately, if the results are sufficiently bad there's actually no salvaging the science; meta-analysis can improve things but still requires good data to start with.

Science itself isn't at fault, because science isn't really a thing. In this case, it was the institution of medical science that we trusted erroneously.

Actually, it's really amazing how many dangerously wrong ideas medicine has come up with over the years.