Following that logic, you CAN turn off the echo, it's got what looks like mute and power buttons right on top.
The question for the paranoid is whether it (the Echo) is useful enough while not connected to the Internet. (I think it's too much to expect any device to be particularly useful while off. :-) ) At least some commenters seem to think so—https://news.ycombinator.com/item?id=8568572, for example.
(Also, to be fair to chatmasta (https://news.ycombinator.com/item?id=8568265), I think that s/he was suggesting that the Internet connection / voice-recording function is on whenever the device is, not that the device itself can never be turned off.)
“Edison, an off switch!”
“She’ll get years for that. Off switches are illegal.”
— Max Headroom, season 1, episode 6: “The Blanks”, 1987
It has a mic mute button, but not a power switch.
> Do you have any computers in your house? A TV made in the last few years? A smartphone?
Neither the first nor the last of those is necessarily always on or Internet connected. (For example, I have both of them, and, with brief exceptions, neither is Internet connected while in my house.)