Yes, I believe that there's an element of implicit bias to it. It's one of those murky things that gets bundled up with "culture fit" as spurious reasons to reject candidates who don't look and act like you do. Hence the "shibboleth" comment.

It's a complicated question. What's innocent, non-coercive fun for a small group of responsible people can get wildly out of control with one bad hire. And we don't do interviews for "angry drunk"....