This doesn't need to be the case, though depending on context extra design elements (colour, descriptive text, …) may be needed to make the interpretation truly unambiguous.

Sometimes (advert stalking preferences being the key example) the confusion is very deliberately not dealt with, and furthermore on occasion extra steps are actively taken to ensure the confusion.

I've never seen toggles show "what happens", they're mimicking the real-life toggle. Accent color means "I'm active"

unless you're on an email newsletter subscription settings page, in that case beware