I think unemployment hurts the poor the most because bosses fire the least productive people (untrained workers), even when those people spend a bigger portion of their salary on consumption and thereby generate demand for products to sell. A lot of the more "productive" people already have what they want and spend their money on acquiring capital which is kind of pointless if nobody is there to buy your products.