I strongly prefer aisle seats. I once had a girl ask me to switch seats with her so that she could sit next to her boyfriend. Since she had a middle seat, I politely declined, explaining that I would only switch with another aisle seat. She proceeded to throw a fit, calling me an asshole, etc, etc. I paid extra (how have we even come to this?) to be able to pick my seat, I'm not going to give it up for free.
Don't let people bully you.
If the local authority (flight attendants) won't back you up, you are at the discretion of whoever is hogging your seat. Any display of power over the other usually amounts to potentially being able to resort to violence if necessary to back up your power. The person in the seat know full well that you cannot do that without being forced to leave the plane, so they have effectively nothing to fear from you.
Haven't been on a Ryanair flight for a while but this certainly didn't seem to be the case when I was flying with them regularly. Was this something that's changed very recently or maybe it's only certain routes this happens on?
The staff know that getting everyone back into the correct seats will take a massive effort, so don't bother.