After work, on my own time? If I feel like going, I'll go. Haven't ever detected a problem with that over decades of employment. I might make an excuse, I might just say "nah, I was planning on playing an instrument/watching movie/diddling myself", or I might offer no excuse at all.

During work hours and I'm paid for it? I go unless there's an actual reason that I can't. I'm being paid for it, and if an employer is going out of their way, least I can do is show up.