It makes it easier to keep your money in a personal safe if you would rather not trust banks with it. It is not illegal or immoral to avoid using banks. It is your right as a free citizen. Governments don't like it because they want to have control over your savings.

Is this paranoia? I don't know, ask the people in Cyprus, Greece and Portugal.