Potential abusers are not only individuals - even if the current government was mostly harmless, there is no guarantee that it will still be as nice fifteen years later : tyranny sometimes rises fast and with no warning. So even with a theoretical perfectly benevolent government, privacy restrictions are necessary to slow down the hypothetical rise of tyranny - be it individual tyranny or government-wide tyranny.