That sounds like a lot, but considering that this is money that has accumulated over the course of decades (say 30 years), and only a portion of it would be taxable (say 20%), that would give an annual revenue stream of $133 billion per year. If that all went to the US, that would be about 7 times more than is collected by the estate tax, which is less than 1% of government revenues, but still only about half of what is collected through corporate taxes.

(That $20tn estimate is actually global, though, but no breakdown by country is given)

My guess is that it would noticably benefit a country to stop tax havens, but not to the point of any drastic difference.

[1] http://www.bbc.com/news/business-18944097