It wasn’t just doctors, “a gentleman never washes his hands because a gentleman’s hands are never dirty.” Was the common thought.

NB, not all men were considered gentlemen.

See also surgeons cuffs on coats (blazers), so as to avoid getting blood on one’s clothes when operating, which are still called the same thing today.