Also Earth magnetic field is very weak, think of how much force a compass needle has, and that's while you are standing right on the surface - it falls off with distance.
Even ignoring the effects of the interplanetary medium, Earth isn't a homogeneous object. There's a solid core, a liquid sheath around that, a plastic sheath around that, and then the lithosphere we are intimately familiar with, followed by a few levels of gaseous atmosphere. These layers are not fixed to rotate at the same speed. If you speed up the lower mantle, then the lithosphere needs to slow down to conserve angular momentum, lengthening the day. The effect is undoubtedly small, though: as I said, this is on the order of parts per million.
we could be in for some serious shit
But recall that there is conservation of angular momentum, and there is an awful lot of mass spinning at high speed and large radius to Earth's center of inertia. The contribution of the magnetic field to the rotation time of Earth's surface is on the order of ms, or parts per million, as the total term itself is 86.4 ks.
In comparison to the effects of things such as the orbit of the Moon, the magnetic field doesn't have enough energy to meaningfully affect the Earth's rotation.