Unfortunately it can also matter if your boss knows. Or the company you’re interviewing with. It shouldn’t matter (and would be illegal if it could be proven), but it matters.