In the USA and many other places we, society generally holds that, below a certain age, people are still mentally "children" and unable to evaluate certain life decisions. Therefore, if someone above that age convinces someone below that age to enter into an act that the younger party is not able to understand, then the younger party was effectively coerced, even if from their perspective the activity was consensual.

There are a few activities that fall into this category: purchasing drugs and alcohol, driving motor vehicles, enlisting in the military, and sex.

So yes, having "consensual" sex with a minor is rightly considered coercion and therefore rape, unless you disagree with the line of reasoning stated above. Yes, it's not violent like some rape, but it is still definitionally rape. And it absolutely can and does have negative effects on the victim, see here (https://psychology.stackexchange.com/a/21989) for a pretty thorough dive into the subject and a huge pile of academic sources.

And that's not all. In at least one instance, Stallman specifically hypothesized that the girls who Minsky had a sexual encounter with on Epstein's island "presented themselves as entirely willing". We don't know what exactly happened between Minksy and Virginia Giuffre, and it is indeed likely that Epstein's sex slaves were instructed to present themselves as willing. But with his poor word choice and prior history of publicly not understanding the concept of statutory rape, it was a bad look for him, and all it took was one Vice journalist taking him out of context to bring the shitstorm down on his head.