Rather than bare C something like C++, Rust, or even Haskell would be better. C isn't the fastest, especially not with normal code. C++ templates get a bad rep, but if you want to go fast they are extremely hard to beat.

Also those languages show you don't actually have to give up modern features or even that much convenience in order to get blazing fast speeds.