The difference is ease, physical currency is much harder to break apart and move around in large sums without eventually going through something the government regulates. If you kept your transactions entirely in bitcoin, I do not see how anyone could control what you do.