Unfortunately, "just write the performance-sensitive bits in C" is pretty impractical because it only works when you're handing the C routine a relatively small amount of data relative to the amount of work to be done on that data (otherwise the costs of marshaling to C data structures will quickly eat up any gains from processing in C). And unfortunately, it turns out that a whole bunch of code works this way, so the whole bargain of "slow interpreter + easy C extensions" breaks down for a lot of real-world applications, but now we're locked into it.
Processing loops adds a lot of overhead because the interpreter has to make the case jumps after each loop. Figuring out a way to minimize that overhead by using built-in data structures and stdlib library will speed up your code by an order of magnitude.
Don't forget, the built-in types are already running in C.
FFI can pervade an ecosystem making changes (including performance optimizations) to the host language more difficult. It also tends to complicate build and deployment stories. For example, from my Mac I can trivially build a native binary that will Just Work on any Linux system, even one without a libc. Contrast that with Python where I can’t even install popular packages onto most non-Ubuntu Linux distros. And there’s a lot of other things like that which crop up with pervasive FFI.
I’m convinced that the FFI sweet spot is “difficult but possible” and native performance should be good enough 99% of the time.