Generally, the biggest problem with math, physics and chemistry is that if you miss just one lesson (no matter if you physically miss it, or in worst case miss out understanding a concept for a couple minutes), you can easily end up "left behind" for years. With history, arts, languages, a number of computer science topics or most of biology you can either skip out on a certain subject or catch up easily on your own, but not getting a specific part in math/physics/chemistry and lacking the resources to follow up is setting oneself up for exponential learning gap later on.

The societal problem is that we've not set up schools or universities to deal with this "failure mode". Privileged kids can have after-school teaching or siblings helping out, but most will sooner or later resign and hate these subjects for the rest of their lives - and that is incredibly sad.