I still believe it was worth fighting, but I agree that we shouldn’t delude ourselves into believing that we didn’t have other motivations.
It was only the bombing of Pearl Harbor that gave Roosevelt the political power to openly support intervention in the European theater, which before he could only passively push for. If he had announced his intention to get boots on the ground before the 1940 election, we would have had a different president in 1941.
Diplomatic relations between the US and Japan were already essentially ended before the attack on Pearl Harbor. With the financial freeze already in place, particularly the refusal to sell Japan oil, Japan could not continue their wars in Southeast Asia. The Hull Note signalled a final refusal to terms which Japan could accept, and with that, war between the US and Japan was inevitable[1].
Japan's aggression in Southeast Asia could be seen as aggression against US allies, but it has to be seen within the context of colonialism. Japanese aggression in Southeast Asia was primary directed at colonies which were themselves the result of Dutch and French aggression. It's harder to see the US as being the "good guys" when you realize that until Pearl Harbor, US involvement in WW2 was that of an imperialist power meddling in the colonialist activities of another imperialist power[2].
Even after Pearl Harbor, the atrocities committed by the Germans and Japanese were not a big part of the public justification for war. If you look at American propaganda from during the war, all of it appeals to fear of the Germans and Japanese rolling through Europe and Asia and then finally invading America. It was only after the war, when the true extent of Axis atrocities started to become apparent, that these atrocities were used as an ex-post-facto justification of the war.
Hindsight makes it fairly obvious (to me, at least) that WW2 was a justifiable war. But I don't think, given the knowledge the average American had during the war, that we could have said the same thing then.
[1] https://en.wikipedia.org/wiki/Hull_note
[2] The war which was then called the Second Sino-Japanese was was an exception, in that Chiang Kai-shek's nationalist Chinese government wasn't a colony, but Roosevelt and Chiang Kai-shek were allies of the "enemy of my enemy is my friend" variety--it is unlikely that Japanese aggression against the nationalist Chinese would have motivated the financial sanctions the US placed on Japan in the absence of Japanese aggression against European colonies. There's little evidence that American attitude toward Chiang Kai-shek's government became any less imperialist, even after the war: Roosevelt offered Chiang Kai-shek's government control of French Indochina, but this was still a colonialist move, in addition to being a token gesture given none of the allies had the resources to reclaim the colony (and indeed, neither did the nationist Chinese: Chiang Kai-shek declined Roosevelt's offer).
If Germany and Japan had started the war with their first strikes in the same way, but had not committed war crimes on a colossal scale, it might be remembered rather differently. Few people talk up WW1, it gets remembered as a pointless slaughter. Or all the various post-WW2 conflicts the West chose to get involved in.