They didn't evade taxes, they avoided them.

There's a big difference. The latter is legal, the former is not.

Apple did not do anything wrong from what I can see: they were given a good deal, they took it. It turns out that deal was illegal so they have to pay some money to rectify the error.

Doesn't seem very fair but governments are pretty unforgiving when it comes to taxes overall.