It was trained on content from the internet. I'd be massively surprised if it somehow wasn't toxic. Humanity (or a small portion of it) is full of assholes. As much as that sucks, shouldn't the embeddings reflect the reality of the training content? If you want fluffy bunnies and flowers and happy people holding hands, shouldn't you just train it on that content?