Your communications can still be tapped, your location can still be tracked through cell towers, and remote malware can still turn your phone into a persistent bug.

I don't think there's much of an advantage to using a dumb phone. At least with a smartphone you can run apps that encrypt your communications.