Click to add is faster when adding to the playlist. It's just the addition of the small plus to the play symbol isn't immediately obvious if your mouse is hovering over it. I like it better than click to drag. But drag is surely best for first encounter.
Edit*
I see what is happening. There are a couple of paths users can take based on if they click or drag items, and I just so happened to take the smoothest one.