See also https://en.wikipedia.org/wiki/Amdahl%27s_law
This is why, before setting out to improve performance, you have to define a performance metric, goal, and do some profiling. Otherwise people end up flailing around changing things based on what they've heard has better performance.
E.g. suppose we've got a system with three components A, B and C, so that perf(system) := min { perf(A) , perf(B) , perf(C) } . Currently perf(A) is 0.3, perf(B) is 0.5 and perf(C) is 0.4, so the performance of the system is 0.3 since it is bottlenecked by A. You have an elegant idea to reengineer C completely which should lift perf(C) to the region of 0.9 -- 0.95 but it will take a few months to prototype and sort out the details. You've also got a lamentable idea for a hack to increase perf(A) from 0.3 to 0.5 -- it's a dead end and the hack cannot be extended in future, but can be cobbled together quickly. What's your move?
https://en.wikipedia.org/wiki/Theory_of_constraints