Is this really what happens? My home country Sweden does the same thing, burning garbage, if that was really the case I doubt it would be allowed.

It depends on the temperature they burn at. You need pretty high heats to combust dioxins, for example, and from some very light skimming of the USA's standards I think you also need various filters and scrubbers for heavy metals (especially mercury which will boil away at these temperatures).

This stuff can be done, but it's expensive. I also don't know how close to 100% it gets even if everything is done properly. And of course to fully remediate this all of the equipment has to be operating properly.