I've delved into this. For lots of detail on the current situation, see my paper, "Publicly Releasing Open Source Software Developed for the U.S. Government" by David A. Wheeler, https://www.csiac.org/journal-article/publicly-releasing-ope...
In almost all cases, if a US Federal employee writes software as part of his/her official duties, then there is no copyright IN THE UNITED STATES. There are a few exceptions (e.g., US Post Service Employees). Also, copyright doesn't apply in the US - but outside is a different matter (though enforcing said copyright is more complicated, and I would argue the US shouldn't try). More importantly, there may not be a copyright, but that doesn't guarantee release to the public (and often there isn't). You can ask via a FOIA, but that's no guarantee you'll get it.
HOWEVER: Most software developed by government funds is developed, at least in part, by contractors. And that is a COMPLETELY different circumstance. The details, unsurprisingly, depend on the contract.
If you're interested in the DoD, I suggest going to the MIL-OSS mailing list: https://groups.google.com/forum/#!forum/mil-oss
It's possible to release software to the public. One approach is to get pre-approval where "as long as the changes meet this scope, and follow these rules, it's okay to post the changes." But you have to get local buy-in.
In short: In a lot of places in the US federal government it's definitely difficult to do what in the rest of the world is normal.
In the US, state/local/tribal is completely different, and I definitely don't claim to know the laws of other countries. Others can explain that better.
So the good news is that any government contractor overseeing a contract should be able to release it as OSS, if ve wants. Bad news is that it’s effort and risk to do it correctly.
Isn't the point that the contact should have a non-negotiable OSS clause of it's using public funds.
My impression, from my seat in Europe, is that USA government is run by rich capitalists that would as soon eat faeces live on TV as suggest private corporations be required to service the public good?
Okay. Say you're a vendor for student information systems. It handles registration, enrollment, scheduling, attendance, report cards, state and federal reporting, teacher gradebooks, parent portal, etc. You sell your product to one school district with 800 students for an initial fee of $100,000 plus an annual fee of $5 per student and $10,000 for up to X hours of direct support (note: these numbers are not far off the actual costs).
Except now it's open source, and every one of the hundreds of thousands of school districts in the country suddenly has access to your software without charge.
Sure, you can come up with dozens of reasons why that's great for schools, but you're the SIS vendor that just spent a couple million dollars producing the software. Why would you be in a business where the moment you sell your product, you can no longer sell your product? Support? Okay, but you'll immediately be competing with other support vendors, etc.
The relevant scenario is that the school approached the company and paid them to develop the software.
Another potential model is for the government to partially fund development. For instance, if the government wants a piece of software that costs $1,000,000 to develop, a company might offer to develop it for them for $500,000 and the rights to resale it to other customers.
This isn't an intractable problem though. You would just need to make a clear delimination between what components the government is paying for, and which ones is coming out of the company's R&D budget. Due to billing, this is already how many of these projects work, but the publicly funded portion is often far less useful without the privately funded portion; and there often is not a good solution to this (think a government funded plugin to Microsoft Excel, where an Excel license costs $5,000)
Edit: or elastic. Or automattic. Or black duck, or canonical, or cloud bees...
Not quite true. Rich capitalists largely have better things to do than bother with the details of running government.
But it is, to a large extent, run by people under the influence of (whether directly seeking money, or just manipulated by paid propaganda) rich capitalists who would rather engage in public coprophagia than have such requirements imposed.
The ones who do care more than that tend to be absolutely awful.
Clearly, they don't want new ideas.
The mad lad my lead was, said we would need a database for retaining some of the basic data we needed to keep track of. Nothing all that special, just a few tables, and by nature nothing sensitive would be tracked.
This got rejected immediately, and our IT literally cited "no database installations are allowed per security team."
Needless to say 4 months later, after some back and forth, he eventually said "whatever, we'll just store everything to files then" and that's how he came up with the world's slowest inventory manager - I suspect just the way they like it.
The state server we were going to use to store some application metadata had the word "database" in the product name. For that reason alone we got flat out rejected.
I said to them, "So it's OK to store all the stuff we need in memory and on the file system on the servers, just not stored in the state server."
Their response was, "Yes, because it's not in a database."
We finally switched to another product that did not have the word database in the name, and they were completely fine with it.
Some state work got in trouble a few years back because they'd updated guidelines that no applications could be on the same machine as a database server. Enforced separation, for security. Having enough staff and management to distribute that message to people dealing with the apps and servers didn't seem to be a requirement, however. And... when I was given this mandate, I had to ask for another server, then told there weren't any available, and I'd have to wait for several weeks, and I probably should shut down the current service (which had run for 3 years this way, well before the guidelines were in place).
Navigating govt IT stuff is crazy. About half the people I've encountered are between competent and really talented, and the other half don't understand the basics.
FWIW, if the web server can't communicate 'directly' with a database... what process do you use to communicate between them? Some sort of proxy? Or something else?
Digital cameras were at that time strictly prohibited from even being on the premises.
So my friend goes ahead and buys the machine he needs, on the assumption that if he doesn't everybody is out of a job, and he can always disable the camera. The machine arrives, he sets it up, and calls security asking for whatever official process they use for disabling webcams. Epoxying over the lens or something, was what he figured.
At first they are very confused. It's like they can't parse the words he is telling them. It's as if he was trying to tell them there were six legged iguanas with Russian flag patches on their backs running around the server room.
So he finally gets them to acknowledge that he is, in fact, speaking English in sentences that adhere to the rules of grammar, at which point they have to take his question seriously.
So they ask: “did you procure these computers through <our base's> government procurement office.” To which he says, yes, of course, there's no other way to do it.
They think for a minute and reply: “by policy, you cannot purchase a digital camera through the procurement office. Therefore, your computers do not have digital cameras in them.”
They were being intentionally daft. The person who told him that almost certainly did so with a wink and a nod, and saved your protagonist a lot of pain.
my 'database and app server separation' thing - never quite got a straight answer on 'why', other than 'security'. except... adding more systems, with more logins, more access rules, more stuff to update/patch/test leads to... more potential for security failure (in my experience).
This is common enough across a variety of organizations (I've certainly run into the same issue at several large-ish companies) and isn't necessarily unreasonable... if IT also provides a process for getting what you need with their oversight/help (bonus if some part of the IT team actually has skills to do audits rather than cargo culting off a checklist).
So, how are applications supposed to store/retrieve data?
"No sir! It's just a base of data."
<checks spec book> "Very well; carry on then..."
By this argument, any database with a public file spec is a flat-file database.
They got MS Office approved for all PCs, so we ended up building all based on Excel, Word and Access.
There were multiple weeks of meetings with lawyers (including discussions of whether we needed a space act agreement[1]) to figure out how to actually open source the modifications we made on NASA's behalf. Ultimately we ended up having to assign all of the nova copyrights to the government[2] so that they could open source the modifications. I didn't even think that the government could own copyright but apparently it can[3].
[1]: https://www.nasa.gov/partnerships/about.html [2]: https://github.com/openstack/nova/commit/c88d1f033bd600e855c... [3]: https://www.usa.gov/government-works/
Here is an excerpt that talks about licensing:
>Q: Can government employees develop software and release it under an open source license?
>Not under typical open source software licenses based on copyright, but there is an alternative with the same practical effect.
>Software developed by US federal government employees (including military personnel) as part of their official duties is not subject to copyright protection and is considered “public domain” (see 17 USC § 105). Public domain software can be used by anyone for any purpose, and cannot be released under a copyright license (including typical open source software licenses).
>However, software written entirely by federal government employees as part of their official duties can be released as “public domain” software. This is not under a copyright license, it is absence of a license. By some definitions this is technically not an open source license (because no license is needed), but “public domain” software can be legally used, modified, and combined with other software without restriction. Thus, “public domain” software provides recipients all of the rights that open source software must provide.
Please excuse what I suspect is a naïve question: why can't they just throw the code into a zip file, and send out the file the same way they would other documents? Digitally or on a physical thumb drive.
(The FOIA requester could then put the code on Github or similar.)
The issue is that, basically 100% of the time, anything being released to the public has to go through a release process. If you produced it as an employee of one agency (eg the Air Force), but if was paid for by a different agency (eg DARPA), then it also needs to go through a similar release process on the sponsoring agency's side.
Typically, this has to happen for every single “release”. In other words, for the original software, again for every subsequent patch, for the documentation, for every correction or update to the documentation, etc. If you want to give a presentation about it your slides need to be released. If you want to write a paper on it that needs to be released as well.
Evey time through the release process takes two to six weeks, depending.
This works OK if you are releasing documents. Once a document is written, it usually doesn't need hundreds or thousands of updates over it's lifetime. A journal paper gets released once, then it's done. But the same process is what we currently use for software, and software is just a very different thing than a document is.
Imagine what would happen to the release cycle of a small OSS project if it took six weeks for github to get around to posting your update, and if github management rejected about half of them for reasons that were mostly inscrutible to you.
(Of course, if the updates aren't happening at all because of opaque internal procedures, that's a different story.)
I've got tons of code I've written over the years that, if I could just throw it onto github, I would. But since it would take me six weeks of pain, I don't. Some of that code I'm kind of proud of, or was at the time I wrote it, anyway. I wrote an rviz app a decade before ROS took off.
It could happen, but it would take something like the Secretary of Defense ordering it. Or a White House directive. The folks at my level, or at any level I can actually talk to, don't have the authority. The thing is, they aren't making these policies and procedures up just to be dense. There are real, hard and fast laws and department-wide policies that require the system to look the way it does.
(If there were more people filing request, I wonder if the internal procedure would be made less painful out of necessity...)
FOI is meant for information that can legally be published with reasonable effort, but not yet available.
The most effective method is to have continuous development through an open source structure so any failures where someone hard codes sensitive info in source get found quickly.
It is completely true that contractor-written software is legally different and has its own release policies. For one thing, it can be copyrighted. Again, IANAL, but I assume that there would be stipulations written into the contract governing IP.
It's going to take time but we're on the right track. Send me a message if you want to be involved in this culture change.
And:
Keep in mind, Los Alamos is one of the big labs doing nuclear work for the DOE, so obviously a lot of what they do is classified, but of the stuff that isn't they're actually doing a pretty impressive job of open sourcing it.
https://www.hec.usace.army.mil/software/hec-ras/downloads.as...
Back when I did work on hydraulic modeling, HEC-RAS was the gold standard for open channel flow. It is only available as a pre-compiled Windows executable.
I had a ton of parameterized simulations to run and would have loved to have gotten my hands on the source code in order to run it on EC2 Linux spot instances and to mechanize the parameter inputs by cutting the GUI input forms out.
> Setting up a git repository
Git != GitHub
> Every single patch has to be pushed through a public release process
Rebase is your friend.
Apparently in Europe and some other places it's not so easy to disavow all rights to a bit of software. BSD- or MIT-style licenses are much more comforting.
All the science done at that national lab for less money before it was privatized was after privatization private property, and before privatization more science was done for less money.
The reason the costs became higher was that a private company is not tax excempt and the government guarantees a profit for the contract.
In effect this is a no cost private theft of government funded research, and I find it fishy that almost all the national labs were made private in the decades after the act. The only upside seems to be that the private company take any blame for a scandal.
[1] https://www.mofo https://www.mofo.com/resources/news/the-governments-patent-p...
The contractors don't have to be.
It's the same reason why most people stay out of federal jobs - you can't make more than the president. But if you're a consultant working for the gov't, you can get paid $$$$$.
IANAL, but my understanding is that all software written by US government civil servants is in the public domain. It cannot be copyrighted or otherwise licensed (although the government can patent things). This does not necessarily mean that it is available to the public, as noted in the other comments; it may be classified, considered For Official Use Only, or restricted from release by ITAR regulations.
Barring any of those cases, you should be able to get a copy by filing a FOIA request. The problem with FOIA requests is that you have to know specifically what to request, usually by name. Obviously, it can be hard to put your finger on exactly what software you are requesting.
The bigger problem, though, is that much of the US government just isn't technically set up to release software (as opposed to documents). Setting up a git repository, in the DoD, is an utter nightmare. Every single patch has to be pushed through a public release process that can take weeks or months and involves review by as many as ten to fifteen different offices, few if any of which even know anything about software. If you're setting up your own server, getting the server itself approved and provisioned can be an exercise in bureaucratic frustration, and can take literally years to accomplish.
Those of us writing software very often want it to get out there. But the structure of the organization makes that excruciatingly difficult.