In addition to the risk of disease, arrest, and robbery, something like that could also jeopardize your chances if you later want to work in a job requiring a clearance, background investigation, or polygraph.

Can you try to find some new activities or clubs there to participate in? Not being fake or untrue to yourself but small steps that push you outside your comfort zone? Dance, theater, athletics?