US government commits to publish publicly financed software under FOSS licenses

https://k7r.eu/us-government-commits-to-publish-publicly-financed-software-under-free-software-licenses/

by dandelion_lover • 10 years ago

773 104 10 years ago

k7r.eu

kyled• 10 years ago

Floating points.... Floating points everywhere!

komaromy• 10 years ago

Wow, this is great. I hope it prevents another contractor-related debacle like healthcare dot gov.

rtkwe• 10 years ago

There was a lot of things that went into that. There was a huge time crunch on the project and until the bill was signed there was only so much groundwork that could be laid because you could only make passable guesses at the final requirements.

thegreatpeter• 10 years ago

If this actually works, wow

andrewstuart22• 10 years ago

It seems like they've cut a wide swath for (probably) exempting the most interesting of the software that could have been developed in the open.

Applicable exceptions are as follows:
The release of the item is restricted by another statute or regulation, such as the Export Administration Regulations, the International Traffic in Arms Regulation, or the laws and regulations governing classified information;
The release of the item would compromise national security, confidentiality, or individual privacy;
The release of the item would create an identifiable risk to the stability, security, or integrity of the agency’s systems or personnel;
The release of the item would compromise agency mission, programs, or operations; or
The CIO believes it is in the national interest to exempt publicly releasing the work.

[1] https://sourcecode.cio.gov/Implementation/

davb• 10 years ago

I agree with your comment, but I think it will be easier to pass this right now with some wide exemptions and potentially expand it somewhere down the line.

nerdponx• 10 years ago

Sure it's wide, but it also seems pretty reasonable to me

andrewstuart22• 10 years ago

My main disappointment is that I'm not sure it will be reasonably applied, as so many things could be interpreted as having national security implications when in reality they're only used with security-sensitive data.

spilk• 10 years ago

This probably excludes pretty much everything except probably the most boring things - public information websites, etc. Anything within the DoD, for instance, will at minimum be tagged "for official use only", which already exempts it from FOIA requests, so I don't see this really opening much up at all.

JonathonW• 10 years ago

DoD work can, in at least some cases, be open-sourced. We had some work with DARPA where open-sourcing was actually mandated in our contract (at our request), and that code was, for a while at least, in active use in the field. Although I guess research is a little different.

Whether or not it's feasible is entirely dependent on the project, though. Something that's ITAR/EAR (or classified) is never going to be open-sourced.

jonas21• 10 years ago

The important thing is that this changes the default. Instead of the default being closed and needing to justify open-sourcing something, the default will be to open-source things and having to justify keeping something closed. That's huge.

derefr2• 10 years ago

It pushes the incentives around as well, though. Unless there's some sort of marginal cost for each line of code they choose to keep proprietary, they'll inevitably do "the least work necessary" when they realize that some project is going to have components fitting these clauses, by by boxing up the entire effort under one of these clauses.

Ideally, with some sort of force opposing propriety, the incentive would be such that the government would think it in its best interests to granularize their code into small libraries, programs, and network services, so that they could open-source as much as possible, leaving tiny "classified kernels" as wrappers for libraries or plugins for apps.

For an idea of such a force: maybe create a "proriety credit" that a government department would to buy for each 1000 lines of code they refuse to release, whose funding goes to this effort to open-source everything else?

Zigurd• 10 years ago

The first two look reasonable, but after that it goes downhill. "The release of the item would create an identifiable risk to the stability, security, or integrity of the agency’s systems or personnel" means "I don't feel like it and you can't make me."

ipsin• 10 years ago

I'm probably misreading it, but "the integrity of the agency's personnel" makes it sound like "the in-house developers wrote terrible code and people will criticize them for it".

revelation• 10 years ago

So they commit to nothing at all.

petra• 10 years ago

This doesn't include anything developed at DARPA, right?

sitkack2• 10 years ago

Lots of DARPA sponsored projects have open source releases.

twistedpair• 10 years ago

Of course there are many exemptions. Did you expect them to release the F-22 source code or NSA PRISM architecture?

robin_reala• 10 years ago

Operational stuff no, but GCHQ publish their graph database for example: https://github.com/GovernmentCommunicationsHeadquarters/Gaff...

sleepersmith• 10 years ago

Of course not. That's why encryption algorithms are always best kept proprietary. /sarcasm.

afarrell• 10 years ago

That looks like PACER still falls under that.

I wonder if this would make it easier to build a replacement whose interface isn't utterly terrible. Of course, the hard part would be convincing the courts to use it. Based on a sample size of one Clerk of Court, who thinks PACER is absolutely fantastic, this will be hard. To be fair, it is state-of-the-art in terms of access to court documents in the US.

nwenzel• 10 years ago

Could you imagine an API-only version of PACER where developers were free to build search, mobile, other front-ends on of it? Sure, that's basically a high school project using a MongoDB or CouchDB back-end. But legal tech is horribly antiquated and govt-legal tech has a long way to go. It seems like the best move would be API-only so that individuals and companies could build services on top.

BTW - check out www.casetext.com for an idea of what private enterprise can do in terms of making public records useful.

konklone• 10 years ago

Different branch, though. This policy only covers certain executive branch agencies.

spacehome• 10 years ago

I hope this includes voting software.

brudgers• 10 years ago

In general the US Federal government does not operate polls. The various local governments operate elections while abiding Federal and State laws and regulations. In practice, the highest level where software standards are promulgated is state governments and there only some states and with variation among them.

chaostheory• 10 years ago

I could be wrong, but in most cases the US government buys appliances from companies like Diebold and they don't develop or contract software for voting. The best that can be done is regulation to force audits of the appliances like what Nevada does with casino slots. Even that would be a huge step up.

nxzero4• 10 years ago

Might be a reason to long-term require all software used be open source and for that matter make it easy for a user to confirm that the system in use real-time is the system that's open-sourced too.

grondilu• 10 years ago

Unless you have a way to ensure the code actually running on voting machines is the code that was published, there is no point.

gpm• 10 years ago

Yes there is, it significantly raises the bar for maintaining plausible deniability while enabling voting fraud.

Right now if there is a bug that creates incorrect vote totals it would be very hard to prove it was done intentionally. If the software was required to be open source, then just the fact that they were running different software than was published would be strong evidence that it was intentional.

macktheknife• 10 years ago

There are actually foundations out there that do exactly that. If you're interested, OSET Foundation (http://www.osetfoundation.org/#welcome) has developed everything from ballot scanners to entry systems. It's all open source and they've been at it for about 10 years. They patented some of their systems and made the Secretary of commerce the owner of these patents, as far as I'm aware. Legislation is slowly moving towards adoption, given the potential reduction in cost.

stonesixone• 10 years ago

There is an effort happening in San Francisco right now to do just this. See here, for example, for a recent March 24, 2016 hearing on the topic:

https://sfgov.legistar.com/LegislationDetail.aspx?ID=2567089...

"Hearing on San Francisco's efforts to fund, design, and implement an open source voting system, including budget costs and projected roll-out schedule; and requesting the Department of Elections and the Elections Commission to report."

It's currently up for consideration in the budget.

SF resident Brian Behlendorf recently endorsed (along with EFF, Code for San Francisco, California Common Cause, local GitHub CEO Chris Wanstrath, etc), and we're looking for more high-profile supporters to help influence the Mayor's budget decision:

https://twitter.com/cjerdonek/status/716448812181512192

tfigment• 10 years ago

It exempts prior work from this clause though it says it encourages vendors to adhere to it. I have a hard time seeing that anything really interesting will be ever released because it will likely have national security exemptions in it and like taxes many companies will find a way to avoid it.

Karunamon• 10 years ago

Yeah.. just about anything can be declared under the umbrella of "national security" if you reach hard enough.

For a true to life example, look at the NSA.

crishoj• 10 years ago

Looking forward to https://github.com/nationalsecurityagency/PRISM et al

nickpsecurity3• 10 years ago

It's funny you say that because that's the one NSA did voluntarily:

http://www.informationweek.com/applications/nsa-submits-open...

https://accumulo.apache.org/

;)

ismail• 10 years ago

Also: https://nifi.apache.org/

mikey_p• 10 years ago

Don't forget https://github.com/simp

mtgx• 10 years ago

Help us, spy you.

jlgaddis• 10 years ago

Heh, head over to Github and fork nsa/xkeyscore, set up a local instance on your network, and forward all interesting events straight to the NSA using their new API!

ThomPete• 10 years ago

Several companies have been started by using software developed by people working in governments and it's only fair as it's all funded by the taxpayers

Because many governmental projects are politically sanctioned, done properly, this could become a great way to do R&D where the private market doesn't.

castratikron• 10 years ago

Great news.

nxzero4• 10 years ago

(Next step, open source the government policy, procedure, etc.)

Laws like requiring all commits to be submitted by the author. Authors must disclose potiental conflicts of interest, affiliations, etc. Submitting commits on behalf of another entity is treason; meaning if a funder wants a policy written, they post the funds, select another entity, selected entity assigns staff, staff submits policy, etc.;

All public, all real-time.

elcapitan• 10 years ago

    >npm install xkeyscore

    npm ERR! 404 'xkeyscore' is not in the npm registry.
    npm ERR! 404 You should bug the author to publish it

Hm.

JetSpiegel• 10 years ago

It probably depends on left-pad

moon_of_moon• 10 years ago

Why is this limited to software?

Why not also publish publicly financed: * legal agreements * HR contracts * management presentations, best practices, and training material * marketing presentations and training material * creative output such as artwork and music

so that the rest of the world can use them for free..

justinlardinois2• 10 years ago

creative output such as artwork and music

If I'm not mistaken, all intellectual property published by the US government is public domain by law. I may be wrong or you may be referring to something else.

tzs• 10 years ago

Close. It's not publication by the government that is important. It's authorship.

If a work is a work prepared by an officer or employee of the United States Government as part of that person’s official duties, then it is not subject to copyright in the US.

It might be subject to copyright in other countries, though.

If the government pays for a work to be created and published, but it is not a work prepared by an officer or employee of the United States Government as part of that person’s official duties, then it will be copyrighted and the copyright owner will be whoever it would be normally. If there is an agreement between the author and the government to assign the copyright to the government, the copyright will still be valid, and owned by the government.

carussell2• 10 years ago

Freely redistributable artifacts are one thing, but publication "in the preferred form for making modifications" is a pretty important part of FOSS.

jMyles2• 10 years ago

It is completely contrary to the notion of an open and free society that the government is permitted to spend taxed dollars developing anything meant for domestic use in secret, much less it being (more or less) the default policy.

Looking at the exemptions carved out of this rule (assuming this source is accurate), this looks more like it is codifying the secret nature of government operations than it is opening up anything substantial.

Hard to regard this as good news, although it may be another straw on the back of the government finally falling and the emergence of an actual open and peaceful world.

dinkumthinkum• 10 years ago

How exactly is I "contrary to th notion of an open a d free society." I think that's just nonsense. And you think this could only be good if it leads to the government failing and a failed government obviously leads to world peace?

I think some just let irrational hatred make them believe inane fairy tales.

jMyles2• 10 years ago

The basis for transparency is government, at least in the common law tradition, is the understanding that without it, government actors will tend to act in their own interest rather than in that of the public.

Do you think that this is a reasonable belief?

If so, then it logically and obviously follows both that 1) development of products by the government (to the extent that this is even the reasonable purview of information age government at all) need to be conducted in a profoundly public way and that 2) a failure to do this will result in a properly disaffected populace and shortened moral stature on the part of the government.

It is no fairy tale to suggest that secrecy in government is a substantial part of the formula of government failure more generally. The only assertion I'm making that might not yet be in evidence is that, amidst the information age, this aspect of good government is more important and that its effects will be realized far more quickly.

known• 10 years ago

I believe it'll be BSD licence

Dowwie• 10 years ago

And after the first data journalist wave of analysis reveals an average of 10,000 redundant projects worked on by agencies at any given time, open access will go dark again.

sitkack2• 10 years ago

This is excellent as it puts something (sorta in writing). But the government only has a limited ability to retain copyright. If there is a wholly in-house developed program, one _should_ be able to get the source under a FOIA request.

Eduard• 10 years ago

What percentage of US government software is currently developed as FLOSS?

niels_olson• 10 years ago

One of the current issues is that government-written code is pure public domain, so any contractor (SAIC) can FOIA some code the know their DoD handlers want implemented (ViSTA, written by the VA) and charge $4B. That is a tragedy of the commons if I ever saw one.

foxhop• 10 years ago

I want the source code of government cryptography programs.

gryph0n• 10 years ago

You do realize the security lies with the keys/passwords used to encrypt, and not in the program/algorithm itself?

rjurney• 10 years ago

A great start. Now if only the same would be required of all code arising from federally funded research, we could massively accelerate research and development in America.

edoceo• 10 years ago

I want more State government to adopt this policy

lifeisstillgood2• 10 years ago

This has been a UK government stance for a while but the problem is that while the UK gov spends something like 8BN pa on software licenses the amount of new development is fairly low (outside of say the GDS - central cabinet office group pushing all this)

So for example, my local council wants to purchase software to help track customer feedback from dozens of sources (web sites, phone ins etc). Sounds perfect for a web era, message queue based project.

But my council is only interested in meeting its fiduciary duty in buying software that already exists, and has been proven to do the job required. Speculative development is out - even though pretty much everyone of 430+ councils, parishes, towns etc of the U.K. Will want something similar in the next few years.

No one is willing to stump up the cash to start the open source alternative.

It's a tragedy of the not commons.

I think there ought to be solutions in some kind of development bonds or other financing arrangements - but it's not clear how.

tvararu• 10 years ago

This has been a UK government stance for a while

As further confirmation, it's cemented as one of the ten design principles behind GOV.UK: https://www.gov.uk/design-principles#tenth

lifeisstillgood2• 10 years ago

Yeah but still... No way to get to the majority of IT spend - even if that IT spend is wasted on licenses costing more than the cost of the capital build out

verst• 10 years ago

Obligatory mention of 18F (https://18f.gsa.gov) within the General Services Administration who develop federal government projects in the open (and have defaulted to open-sourcing since their inception).

https://18f.gsa.gov/2015/01/16/open-source-for-good-governme...

phkahler• 10 years ago

Licensing problems. For new projects this sounds great, put the code either in the public domain or they pick a license. Which one? MIT/BSD, GPL/LGPL, or something else? The former can generally be incorporated into the later, but what if they pick something else that's incompatible? At least we can see what was done and learn from it I guess. And what if they require a particular license as part of the policy and it prevent government projects from extending existing OSS project? That would be a shame.

The intent seems good, but the details will be everything.

meanduck• 10 years ago

Its only the article stating (and misleading) that softwares be Free Software but in actual the draft never once mention Free Software. So unlikely it would be GPL/LGPL/etc.

Its also suprising to see that this question popped up after 500+ points.

batbomb• 10 years ago

GPL is discouraged in government software (speaking from DOE/NSF experience) but BSD or similar is encouraged. The path of least resistance is private/closed source, followed by open source BSD/MIT/Apache, followed by GPL/LGPL, followed by commercialization.

The reason why there is a bit of resistance with open source as a default is because funding has to be traced.

bpchaps6• 10 years ago

Heh. I submitted a FOIA request two weeks ago for code for a $100M system used for storing parking ticket data. They have 5 days to ask for an extension, but they haven't even responded with that.

Guess I'm going to lawyer up again..

MichaelApproved• 10 years ago

What is the penalty for not responding in time?

bpchaps6• 10 years ago

For Illinois, two big things - 1. They can't reject the FOIA with "unduly burdensome" anymore, which is an incredibly common and frustrating response. 2. They can be sued for not complying, which then becomes a slow drawn out process. The upside is that it's something like 8k if the judge thinks the request was rejected/ignored intentionally or without proper followup.

Example - I did a FOIA that was rejected absurdly unfairly. Going through the attorney general took about 6 months to hear a response back, which Chicago's lawyer then used essentially a mistype to throw that request out. 8 months in total there. Fast forward 6 months later and we have a court date with a lot of small things in between then and now.

AckSyn• 10 years ago

https://firstamendmentcoalition.org/2009/06/non-response-to-...

ruffrey• 10 years ago

Let's start with open sourcing the effing voting machines.

rmason• 10 years ago

I presented on this three years ago to the state government here in Michigan. It makes far more sense for the state's to do this than the Feds.

Why should there be fifty different software programs to register for a state campground? Or deal with the DMV? When the Feds pass a new law some contractor ends up writing the same software fifty different times for each individual state with small differences.

I believe the key is to have a plug in architecture, like WordPress, that allow each state to adopt the program for their individual laws. I think the savings could be tremendous and its entirely possible with multiple states collaborating the software could be continually improved.

I failed to convince them that day but I think the tide is turning.

atonse• 10 years ago

Intuitively makes sense until you realize that there are similarly a lot of line of business apps that are used and implemented again and again across federal agencies.

For example, every classified document has a classification level (Secret, Top Secret, with more details on it). And the functionality to roll up the classification is re-implemented by every contractor every single system. So even within ONE agency, every system's implementation is done from scratch.

It's the perfect kind of thing that could've been an open source library (still in a classified network though), but still extremely valuable. It's something I built and recommended to my superiors ten years ago when I worked for a government contractor, but we didn't get much traction for various reasons. (Cynically, most contractors want the free money to reimplement it, so they aren't going to stop that gravy train.)

So there is a LOT of room even within each single agency, to share code that is common to their business practices.

You'll be hard pressed to find an area in any large organization (public or private) that couldn't benefit from the efficiencies of reuse.

deathanatos• 10 years ago

The custom-build software will also be published to the general public either as public domain, or as Free Software so others can improve and reuse the software.
However, it is encouraged to be retroactively applicable for the existing custom-build software developed by agency employees in the course of their official duties.

As much as I love FOSS, anything produced by the government — including software — is public domain, right? The government is not eligible for copyright — including FOSS licensing. While I understand that the government sometimes contracts stuff out, and those third-parties seem to enjoy a weird exception to the rule, the article seems to be including stuff developed directly by the government for the government. How do they think they have any rights to it at all?

incompatible• 10 years ago

Public domain in the USA, but automatically copyrighted in practically every other country in the world.

US government commits to publish publicly financed software under FOSS licenses

Related Stories