Most modern compilers do data-flow analysis to identify which statements are independent of each other, and they'll rearrange instructions so they can saturate a processor's pipeline. That's what superscalar processors are - they have multiple instruction decode units, so they're executing multiple instructions at once even within a single thread.

A bigger problem is that many algorithms aren't really parallelizable. Throw a single mutable shared variable into the mix - even something as innocuous as a counter, or memory allocation pointer, or a logging buffer - and suddenly you need locking and your threads may end up starving each other. Once of the biggest breakthroughs of MapReduce is that it forces you to write your algorithm as functions that operate on independent data. But as someone who's written a decent amount of MapReduces - the thought process about how you design a MapReduce-based algorithm is very different from (and significantly more difficult than) single-processor programming. You can't just go to an algorithms textbook and copy some pseudocode, you need to actually think through what the algorithm and what it's doing, and then rewrite many state updates as message-sends, merges, and CRDTs. (This, BTW, is why Google interviews focus so heavily on algorithms - in many other jobs "How would you find the shortest path between a trillion graph vertices?" is totally academic, but in big data it comes up all the time.)