I'm a bit of a "rich lycra-clad yuppy" myself, and occasionally ride with other such cyclists. I've literally never heard anyone express horror at the idea of their hobby becoming more inclusive.

Mostly e-bike riders are just ignored. Like they don't affect us in any way and we barely even notice that they exist.