Right, you can still do it but there can be some awkward consequences.

This is highly unlikely, but for instance if you're eligible for a draft, and both countries draft you, one of them is not going to be happy with your response.

There are probably some more likely scenarios where your obligations to one of your nations are in conflict with the other.

It also has security clearance implications. Also if he moves back to Finland but remains a US citizen won't the IRS still be able to go after him for tax?

Yes, United States citizens are liable for taxation on their worldwide income (unless there is a tax treaty between the United States and the other country in question).

Only over some threshold (iirc, though it has been some time, around 80k USD equivalent). So maybe yes for someone with a comparatively high income, but not so much a worry for a reasonably large segment of the population. Naturally if this happens to apply to anyone reading I would urge consulting a professional for current advice. ;)

The US has tax treaties with most developed countries, including Finland.

You still need to file tax returns, even if you are under the limit to prove that you owe no tax.

Read that more carefully: "The Supreme Court chose to leave in place the requirement that new citizens must renounce their old citizenship during US naturalization." Presumably, Linus was naturalized, rather than having a claim to United States citizenship from birth.

I did read it more carefully. The next sentence says essentially says that that has no effect (in the vast majority of cases), because other countries don't recognize it as renouncing citizenship. It's essentially ceremonial.

Be that as it may, dual citizenship is definitely allowed, and not at all uncommon in the U.S., especially among European immigrants from what I've seen.