Bear in mind that soap doesn't just wash off the virus, it destroys it. The virus is held together by lipids, which soap dissolves.

Still, that's an interesting point, maybe it's a good reason face shields are better than goggles.