Worse is when boarding and you're in the front row of econ and people will their bags and wheeled luggage smack into your face and knees. Just the other day I was hit in the face pretty good by some guys backpack. Confident people just don't care about anyone else on a flight. That's not even counting the people that can't wait to remove their carry-ons from above and they're way too heavy and fall down.
It's not like I'm taking up a third of the aisle with my shoulder, it's only a couple of inches, but my opinion of the general public is notably worse after a long-haul flight.