While anti-fun attitudes probably play some role, I think it mostly comes down to a pro-productivity mindset. Caffeine is generally understood to enhance productivity, and for that reason may as well be a revered god in the American pantheon. Alcohol is borderline; it's a known quantity that generally seems to leave productivity intact except in cases of alcoholism (alcoholism is consequently seen as shameful). Alcohol bans have been attempted, and are still intact today in remote regions of Alaska where the productivity impairment alcohol induces is seen as a particularly concerning matter. Cannabis remains maligned because it's seen as making people lazy and content, not because it might be fun. And amphetamines are legalized in circumstances where doctors say it will cure productivity impairing disorders.
Furthermore anti-fun is not the sole religious motivation in play. Puritanism / American protestantism is also associated with the glorification of productivity that I describe above.
We could have some survey based study that tries to ask respondent questions whose answer could maybe begin to give us a little understanding of how much each of these angle might be at play in a given population. That would already avoid a lot of this "I think" "You think" debate.