Depends what you mean by "major environmental problem". Landfills aren't going to singlehandedly cause anthropogenic climate change, but they contribute significantly to it, and a poorly sealed landfill leaching its contents into the local water table can cause some pretty drastic problems with the local environment

How are they worse than any alternative? If I dump my food waste in the garden, is that emitting less GHG than letting it decompose in a landfill?

Is it that it comes out as methane from landfills but less harmful CO2 from a little pile? That should be solved by flaring the methane.

Maybe this makes me sounds like a jerk, but:

All material that can end up in landfills is already on Earth.