An example would be Lake Pillsbury in Northern California which the dam has not done hydroelectric for decades but the fight to prevent demolishing the dam has had plenty of local support.
What's more, the dam installation changed the local water table level, and residential wells drilled since the 1940's only had to go down so far. If the dam is removed, all these wells will dry up, and the state is beginning to make it difficult to drill new wells, and they certainly don't want to shell out to install plumbing to unincorporated communities which don't vote for the majority party anyway.
Another dam nearby held a massive amount of water for one of the bigger cities in the state. This is on the west coast, and if we get the Big One, it'll flood a populated area with hundreds of thousands of people. So they've had to draw down on the water levels and it's basically useless now, but deconstructing the dam would probably be a project requiring federal money, and since it's an earthen dam in a seismic zone there isn't really any retrofitting that you can do to fix it.
Basically, dams will give you some cheap electric and a water supply for a while but after 50-70 years they're a huge liability.
https://practical.engineering/blog/2019/3/16/drowning-machin...
That this hazard is tied to a structure with no utility is a specific scenario. But in another where the structure does serve a purpose then restricting one recreational activity in its vicinity may be an acceptable cost. Par exemple, would you be against fencing a substation or prohibiting kite/drone flying near high voltage lines?
That said, any structure spanning the width of a channel alters the flow in ways that are hazardous to swimmers and boats.
I used to hate the "but what about historical site upstream?" argument often used against dams. The prioritisation of a few tombs and piles of stone over what would improve the lives of thousands to tens of millions living now. It's kinda ironic seeing the same asinine argument being used from the other side...
That said, reducing dams to just hydroelectric generation is probably going to skew the narrative to the wrong direction.