I can't really comment on all you have said. But you and a few others replying to you have echo'd the sentiment that "you wish you were invited even though you wouldn't have gone". People pick up on that. It really only takes one instance of turning down an invitation to go out and people will get the impression / idea that you don't want to hang out with them and cease inviting you. Especially if you don't make an attempt to reciprocate and invite them to do something.

Just 2 cents from someone who has experienced similar feelings.