You can get a lot of performance back just by taking branch predictions and cache misses into your designs.

I did just that in C# last years and got at least two orders of magnitude of performance gain from it. Still, I constantly wished I was working in C instead :)