Even ignoring the effects of the interplanetary medium, Earth isn't a homogeneous object. There's a solid core, a liquid sheath around that, a plastic sheath around that, and then the lithosphere we are intimately familiar with, followed by a few levels of gaseous atmosphere. These layers are not fixed to rotate at the same speed. If you speed up the lower mantle, then the lithosphere needs to slow down to conserve angular momentum, lengthening the day. The effect is undoubtedly small, though: as I said, this is on the order of parts per million.