I think visualization can be a helpful tool to understand the data. I have seen some DS's get caught up in visualization for visualization's sake which I think can be wasteful.

I definitely think a solid mathematical understanding helps to build quantitative and critical thinking skills which are very key in data science.