I've gone 30 years without using soap (or anything else) on my body with similar results and experience to you. Also no problems with body odour; instead quite the opposite, being complimented on how I smell (by intimate partners).

I still use it to wash my hands however.