Norway doesn't have nuclear either, they can easily regulate using pretty much only hydro. Most of the time this works for Sweden too, but when it doesn't it's nice to have an alternative. Even if it's spewing a ton of carbon into the air. The oil for Karlshamnsverket mostly comes from Russia.
At any rate, I don't think its a very "nice" to subsidize the fossil fuel industry in order to have the alternative to burn oil. There are 8760 hours during a year, so for most part of that we simply are giving away tax money in order to keep the plant ready in the case of high demand. That is a very high price pay for the pleasure of spewing carbon into the air. If we instead reserved a higher portion of the capacity from hydro, and invested in more peak production by modernizing the generators (and at the same time invested in more environmental friendly design!), hydro and nuclear could manage what Norway is doing. A energy grid free of fossil fuels. That would be nice to have and something I would prefer over an oil plant.
Spewing carbon into the air is still a smaller price to pay than not having electricity when you need it in my opinion. While hydro tends to be quite reliable and predictable I wouldn't want to bet everything on it, just like wind and solar you are still dependent on the weather albeit to a much lesser extent. At least a part of your reserve power needs to be independent of the weather. On a global scale it's just a drop in the ocean.
Given the amount of hydro Sweden have, it is a bit of a black mark on the country that an oil plant actually get to run as much as it does. Its worth noting that Norway doesn't seem to use a lot of oil based peaker plants. I wonder if the oil that get burned in Sweden actually comes from Norway.