I think the bad results occurred when there were too many variations in shadow and color on the faces.

People with clean lines and smooth gradients on their faces came out better.