If my previous manager had made a joke like that, I wouldn't have thought twice about it. We had a great relationship inside and outside of work and my job was very secure. If my current manager did, it would make me nervous -- i'm new to the team and I don't know him that well. It's all about context.