Shear stress. You want something that won't tear when it is pulled against your skin, but with enough friction to remove viscoplastic residue.

The way to make a cloth that supports such high shear stress without tearing is to use a very long fiber. This precludes the fiber from dissolving in water. Some experimental materials actually dissolve not in water but in alkalines such as fats that are found in most sewer systems. But not all sewer systems have the proper conditions so these materials then do not dissolve.