I am not sure about Sweden, but I know Denmark and it's like you say. But in fact the problem is not too much spending, it is too much spending in relation to the income generated by taxes.
When you think about it, the idea a government debt is ludicrous. When a government needs money, it should just "print" it. To avoid too much inflation, it just needs to destroy money through taxes. In other words, governments should be able to borrow from their central banks at 0% interest rate. Oh, and private banks should not be allowed to create money at all, be it through fractional reserve banking, credit default swaps, or tax havens. Money is too important to be left in the hands of such a tiny special interest group.
Now how do the system actually works? Private banks create the money, lend it to everyone else including governments (because governments relinquished the right to create money around the 70s), and force everybody to pay back at an interest. In the end, there is more debt than money, so the system is bound to either fail or grow exponentially. This is nuts.
Germany has an interest rate of 2%, Greece 18%. Germany has debt of 83% of GDP, Greece has 180%.
If Greece still had the drachma, then they could devalue it and work out their own problems. However, they don't. It is now the problem of every Eurozone country as their economies are linked to Greece through the Euro.
And, given the nature of Europe, if the Eurozone has severe problems then so does everyone else.
That's why it's a European crisis and not a Greece/Portugal/etc spending problem - because of the Euro.
Anyway, of course if you produce more money, you still have a spending problem. Only way to solve this in the long term is either increasing taxes or lowering the expenses. You choose.
There is, rightly, a German trauma towards devaluation -- and that's why they agreed to be part of the Eurozone as long as the Euro was managed like the Mark.
Sweden, for example, does not share the same problems even though it has probably the largest welfare system and highest tax rate as a percentage of GDP in Europe.