The US tax comes from the Union in the Civil War. Fearing their wealthy base would flee to Europe to dodge Civil War tax hikes, they passed the citizenship based tax to ensure the war could continue to be funded.

Of course once a government adds a tax it's very hard to eliminate it, so here I am still paying a Civil War tax, an American living in Switzerland.