Remote working isn't for everyone, and seemingly not for you. I don't think that makes it a problem per se. If you are not allowed back into the office you have a dispute with you employer.