That depends (almost) entirely on whether the land is public or private, and whether it is local municipal, state or federally owned land, and whether that land has special protections or not.

For example, planting things (or disturbing existing things) on private land or state or national parks is generally illegal and enforced (assuming you get caught, of course).

Planting anything on a controlled list (such that it is considered invasive or otherwise dangerous) is also generally prohibited and enforced.

If it's not one of those things, then it's generally going to be ignored as harmless. If something were to grow to a height such that it obstructs important views (i.e. of a pedestrian crosswalk), or poses a risk of falling on something (a poorly managed tree leaning over a road) then the city or state would likely tear it out. By that point, though, it's not too likely that you'll get the blame for planting it in the first place if you're acting as an individual.

All of this, of course, is just speculation based on assumptions about the potential hypothetical circumstances. IANAL and you would certainly want to think carefully about what you do to property that isn't your own.

Buddy, it's called "Guerilla Gardening"....

I'm not from the U.S but the answer is of course yes, and they would vary wildly per council/municipality

There are many species able to create damages by several millions of dollars (Not, I'm not joking).

Genetic contamination of endangered native plants with hybrids that are almost identical (or had being genetically modified), would be also a huge mistake

Sure? In practice, you're most likely to get in trouble for any initial trespassing.