Careers are not cast in stone. Some people do change their careers if they realize that there's something else that they'd really rather be doing with their lives.

I've met an artist who became a doctor, a lawyer who became a stockbroker, a software developer who became a full-time professional musician, a professor who started a hedge fund, etc.

On the other hand if you are changing field because you are unhappy with your current situation it is relevant to ascertain that your current field is part/cause of the problem.