People not liking open source (and it's not Oracle)

http://weblogs.java.net/blog/fabriziogiudici/archive/2010/11/07/people-not-liking-open-source-and-its-not-oracle

by Garbage • 16 years ago

30 24 16 years ago

weblogs.java.net

scott_s5• 16 years ago

Another conclusion is that scientists gain very little by publishing their code. In research terms, they're evaluated on published papers, receiving grants and graduating students. The incentives just aren't there.

cdavid• 16 years ago

The article is just really poor, and mostly shows the author's complete lack of knowledge on how science is done, and even what it is about. One the biggest reason for not opening your code is to avoid given an unfair advantage to other researchers: academic research is extremely competitive, and the risk for free ride is significant.

I am all for opening tools more, but I don't think reproducibility is the reason for it. In any case, reproducibility does not mean everyone using the same software (with the same bugs) - it means independent experiments.

ntoshev• 16 years ago

Are there incentives to have a lower/higher barrier for someone else to reproduce your findings and build on them?

scott_s5• 16 years ago

No, not really.

dkarl4• 16 years ago

If you have no reputation and results that contradict expectations, you definitely want someone to reproduce your work. Otherwise, people may follow a Bayesian argument and decide the mostly likely conclusion is that your work sucks.

dkarl4• 16 years ago

If they are willing to be honest, they can publish the code in the condition it's in. What's the extra work in uploading a tarball or exporting a single version to a site like github or Sourceforge? The paper (and the data, which should be published as well) provides all the context needed.

I suspect the extra work involved in publishing code is because the code is in such bad shape that it would decrease confidence in the work that depends on it. If that's the case, then it's all the more important that researchers establish it as a standard, non-optional part of publishing research.

scott_s5• 16 years ago

Again, there's no incentive. You're arguing that it's not a burden, but you're not presenting any benefit it has to the researchers. If you want researchers to start doing this, then you need to figure out how to incentivize it.

JoelMcCracken2• 16 years ago

Investigating the world through computation is a relatively new phenomenon for science.

Science is discussed and advanced through papers. These papers aren't an exact recording or rendition of what was accomplished, but a discussion of the idea, methods used, and eventually a conclusion. Methods may be algorithms, but it doesn't necessarily mean the actual source code.

The idea is that the actual code that a scientist used isn't terribly valuable. If another scientist wants to verify the conclusions, they should just whip up another bit of code (or, do the same experiment) themselves.

Asking for scientists to contribute source code is not wrong, it just isn't done (to my knowledge) with other modes of scientific exploration.

possibilistic2• 16 years ago

This isn't something new at all! Chemists have been using computers extensively since the 70's, and this can be evidenced by the fact that several physical chemists at my university still use Fortran. Computers have been used in science from their inception. Only now are they becoming indespensible for molecular biologists--but they still have a long way to go in both terms of parallelism and the related tools to be really practical for the intense intermolecular interactions we deal with.

Why don't we publish our code? Do you publish one-off shell scripts or database migration code? The only important code is already open source or is (unfortunately) commercial in nature.

josegonzalez• 16 years ago

I regularly see dotfiles in github repositories.

Maybe there is a barrier to adoption in the form of git. A more polished version of Gist - marketed to scientists and researchers - would increase uptake.

JoelMcCracken2• 16 years ago

It is new wrt the history of science, or more specifically the history of academic journals. I am aware of the history of scientific computation -- that's not the point.

The point is that scientists don't need to publish certain things, one of them being the source code that helped them discover their findings.

Shell scripts and database migrations aren't super interesting, but you would want them if you needed to replicate results. I'm not saying they're interesting as published research, but neither is that random piece of scratch paper you used while trying to understand what is going on in some biological process.

It seems like the problem is that non-scientists don't understand the way science happens, and that often the source code isn't something that you would include in an academic journal.

Should science be done differently? Maybe. But that discussion is different than "why wont they publish their source?"

dododo• 16 years ago

in my experience, it's common in academia for code to "float around". often it's not officially open source or distributed, but you know someone who has the code for some paper, so you can use it. there's even a few textbooks in this state (for years).

i think the real reason for not releasing this code is time---you might be surprised at just how much effort goes into making a paper/talk/article polished. sure, you could just release the hacky code you used to do your calculations, but you'd like so much more---for it to be useful, for it to be a credit to you, for it to also do XYZ. research code is efficient in that it gets exactly what you want for the paper; it's rarely usable by anyone else. there is essentially zero payoff in academia in publishing code, and so there's no time to fulfill these desires.

possibilistic2• 16 years ago

I definitely agree with this. Besides, the code itself is an insignificant part of the process. Any paper will include more than enough evidence to back up its assertions and provide a path for duplicating its results.

araneae• 16 years ago

Exactly. In my experience, if someone wants to look at your code, they will e-mail you and ask for it. Then you e-mail it to them.

Sharing on something like sourceforge just simple isn't done. It's not because scientists are "cheating," it's because it's not part of the culture.

eru2• 16 years ago

There could be some payoff for releasing code: If other people use your code, you could probably get a citation for it.

bbb• 16 years ago

> I can only conclude that many scientists are not confident at all with their theories, or they are purportedly cheating.

That's really not an insightful conclusion. I think it's rather a question of pride and effort: the code probably needs to be cleaned up before release, and the reward for investing that time is very low (career-wise). Researchers are going to publish their source code when it becomes required by most major journals and conferences.

tensor• 16 years ago

I think it's worth pointing out that taking someones code and running it again on the same data does not constitute an independent verification of the original work. Just like scientists don't go using the same lab to verify an experiment, they shouldn't use the same code.

That said, it is often useful to release code for other reasons. For applied areas, releasing the code in the form of a software tool is fairly common. This allows people from the target area, who may not be computer scientists, easy access to the new method.

Code review seems to be nearly non-existent in the scientific community, although it does happen occasionally when one researcher uses another's code base for further experimentation.

bhickey• 16 years ago

I've worked as an academic hacker.

Amongst dedicated biologists, there's no motivation to release your source code. It's the job of a biologist to publish papers so that he or she can pull down the next grant. Anything that doesn't further this goal is going to get zero traction.

Publishing it doesn't help you get into Nature or Science. Sure, Sean Eddy (www.psc.edu/general/software/packages/hmmer/) is going to release his code, but he's a methods guy. The software is the focus of publication. The only way code is going to get published is if the journals refuse to publish results without code.

People not liking open source (and it's not Oracle)

Related Stories