The objection I've heard from friends and family is less about any desire to protect the gendered expression of words in Spanish, and more about the fact that LatinX is explicitly and obtusely violating language rules and is effectively unpronounceable. It uses English-specific visual and pronunciation expectations and applies them to another language.

There is already an established way to make this term non-gendered, either drop the gender suffix entirely Latin, or make it explicit by using Latine. Both are which are existing words in Spanish with an established meaning. Rather than use this, activists in the US decided to demand 640 million people change their language to appease those who don't even understand their language.