Packet drops are not entirely separable from speed. One reason packets get dropped is congestion, and speed obviously helps with congestion, since it is an important determiner of capacity.

But, indeed, the speed of an uncongested hop is irrelevant, in that it could slower by some factor, yet without it making a difference to the packet loss.