I would argue that many of the failures you describe are due to a lack of libraries that can exploit these systems. You can't expect every biologist to be an expert in distributed computing. The HPC and algorithms communities failed to consider the users of these systems and happily tinkered in their academic niches. All the while, what was actually needed were easy to use libraries that allowed non-experts to benefit from the advances made. (Also, the algorithms community has lost interest in distributed memory computing).

The abuse of /scratch and the login nodes sounds like a classical mixture of not knowing, not caring, and limited time. That isn't something that has a technical solution.