Is there malfeasance and corruption? Certainly. Is the solution to go to 100% open records in a hyper-politicized environment? I'm less sure.
I know I wouldn't want Fox News / MSNBC combing over every email I've ever sent in the course of doing my job.
Everything the Government does affects somebody in a way that few businesses can. A bad process can literally result in death in some cases (although the third sector attempts to help when Government fails).
People already get into Kafkaesque situations on a regular basis due to either the department's employees not understanding its own policies, or not knowing the magic words to use. The only way to protect against that is to allow people to force the Government to give them information. Personally, I've used FOIA responses in the UK to great effect in forcing the NHS to follow their own procedures (not medical ones, admin ones).
And I absolutely agree with your comment, but per my comment below, I think 100% transparency carries its own risks in making government less effective.
If the UN and large bureaucratic organizations have taught us anything it's that position jockeying gets ruthless when promotions are a zero sum game because of no organizational growth.
My point is that I believe the public interest is best served by a balance between "the public knows how this business was conducted" vs "any rival who wants my job can dig through everything I've done to find some dirt of me."
Politicians, functionaries, and executive staff are people too. And I think HN can be a bit schizophrenic decrying "people in government are so ineffectual and unwilling to take risks" while simultaneously designing a purity litmus test where only the most boring, unadventurous apparatchiks are able to keep their jobs.
Was that ever a standard for exemption from FOIA? Much less the intent of anyone writing any part of FOIA? Stupidity should hurt.
If stupidity were cause for dismissal, we'd have a hard time staffing most of the jobs in the world.