Don't forget that the people working in said public services also need to open holes in their schedules to access the services that other companies provide.
What kind of hour staggering were you thinking about to try and solve the scheduling problem?
For instance my place has a requirement on being there around 9:30 in the morning even though we should be part of the "Ivory Tower" club you mention since we actually work day in day out with people on different time zones.
Same with public services opening at 9h an ending at 18h, forcing everyone else to open holes in their schedule to get there.