Killing a single mining worker can make a difference very early in the game if they're using a very timing-heavy strategy, but that's really only at the very tip-top levels of play.