Mathematics are plural because it's a group of several disciplines, arithmetic and geometry today (music and astronomy used to be two other math branches in the past). It's actually an adjective that became a noun, so these four were the disciplines mathématiques, becoming simply les mathématiques later on.

At least that's why it's plural in French. I wouldn't be surprised if the English term is plural simply because it comes from French.