Use 99% isopropyl alcohol. Add in a teaspoon of table salt -- it won't dissolve in the alcohol and will act as a mild abrasive to help remove stubborn residue. Rinse thoroughly with water afterwards.

If you're really serious about your cleanliness, a desktop ultrasonic cleaner is also amazing for cleaning bowls and downstems. You can get them for a few dozen dollars from Amazon and the like.