He's said it multiple times, and has pointed it at specific people.
It is absolutely not an acceptable thing to say on something as public as the Linux Kernel mailing list. Not to mention that when others see him do that, they believe it gives them free range to do the same thing.
It was never directed at anyone by name, nor did anyone ever go look for who had put in the patches.
And it was after what seemed to have been a frustrating bug hunt related to reports of poor performance in that subsystem.