Plenty of features get in the way of performance. Or are you suggesting we should ditch animation, modern font rendering, file metadata, all permissions checks, most low level security features (ASLR, stack protection etc.), Spectre protection, high resolution displays, USB, frequency stepping, etc. etc. etc.