However due to the federal structure of germany your experience can vary wildly depending on in which state and city you live. I live in Munich and was recently pleasantly surprised how easy it was to request my postal voting documents for the upcoming elections. Took me only 30 seconds or so :)
I think the city council of munich does quite a good job at innovating their IT infrastructure. Most notably the limux project [1] but in other areas like open data and open government as well [2]. Though there is of course still a lot of space for further improvements.
[1] http://en.wikipedia.org/wiki/LiMux
[2] http://www.it-muenchen-blog.de (german)
Yup, it's a quite emotional rant based on some experiences that were unique to my timing and circumstances and yes, from the non-government big insurances and banks you are mentioning, I've already seen much better.
Maybe I should not have been generalizing here, but regularly using and accessing services as the european Tax ID verification or the ECB exchange rate APIs, I could find many many more horrifying examples of user and developer experience and honestly I haven't seen a good counterexample that would give me hope.
To this date, usability and convenience in Government are a one-way street: As a business owner, I'm forced to file my taxes electronically, however electronic bank statements and bills in electronic document formats are still not regularly accepted by the tax authorities...
2013-05-26, 23:10 CET: The link is not hidden, it is clearly visible and with correct description. Currently this works also on a sunday.
2013-05-27, 19:30 CET: Login / registration works on Chrome / OS X .No forced logout.
2013-05-28: link activation worked fine without problem
and so on. I don't want to say that the OP is wrong, it might be the OP has experienced exactly what he has. But generalizing this is total bs. I have done many bureaucratic steps here in Germany and most of them were really fine (submitting tax forms, applying for tax ID, business registration (Gewerbeanmeldung, Handelsregister), banking account, healthcare insurance, ...).
There are many different places where to submit forms, but usually the local trade organizations (Handelskammern) help you by organizing / bundling things together. Been there, done that.