I did some Fortran in undergrad, and while it's pleasant to work with, if you have to do anything beyond numerical calculations you are basically screwed.
C++ has the features as a language to be almost Fortran level pain for math heavy. But the libraries tend to fall short. E.g. Eigen kind of does it, but it will just explode with a thousand line compile error when you least expect it. Also it's missing some quite elemental stuff like ND arrays. And getting code to work across even minor Eigen versions is a crapshoot.
Xtensor seems to have some potential, although it was too buggy to use back when I last tried it.
However, due to many fundamental design mistakes in C++ I'm not really expecting any code, including math heavy, to be very plesant or productive in it.
Aside from "why not Fortran" the "why not 2 languages" is because moving your implimentation to a language few of your users know creates a big barrier between users and developers of your code. In Python, ~90% of users don't even know the language that the packages they use are written in which makes it a lot harder for them to become contributors. Using a single language means that as users learn how to use libraries they are also learning how to contribute to them in the future.
Fortran would be less painful than C++, for math-heavy code.