Here's a nice blog about how the UK deals with this.
https://governmenttechnology.blog.gov.uk/2016/12/15/next-ste...
As to infrastructure work, easiest way to help and still profit is to hire one or more developers that explicitly work on a set of FOSS libraries. That way you have that knowledge in-house and a connection into the community. Also, you'll have some highly motivated employees.
If you poke around https://government.github.com/community you'll find a lot of government created projects but checking those organizations/ contributors will often turn up a ton of forks of popular tools. One common thing is improving security defaults or accessibility, which are tedious but mandatory for government.
If you value this, make sure to let your elected representatives know: I'm sure they hear from the major contractors regularly.
They do it for research, which is another thing that would otherwise probably not pay for itself.
I guess if they started doing it a lot of commercial companies would complain to them about the competition.