Open Science and climategate: follow CERN's lead

http://www.freesoftwaremagazine.com/columns/open_science_climategate_ipcc_cru_needs_take_leaf_out_cerns_book

by nice1 • 17 years ago

22 5 17 years ago

yungchin2• 17 years ago

As for the "climategate" part of that uhm, rant, I'd say Ars' commentary provided a more balanced, real-world view on matters: http://arstechnica.com/science/news/2009/11/uk-hack-puts-cli... - to me, like to the Ars editor, the CRU emails (sadly) look like your average internal communication.

Of course, the guy is otherwise right: we'd all be better off if research was done completely in the open. The problem that's holding that back is that the research-world still only rewards authorship; there's no mechanism in place to evaluate the "impact" of the linked data someone puts online.

CERN are in a special position: they have little problems protecting their authorship, because few peers would be able to claim that they too have a LHC in their backyards...

vixen99• 17 years ago

Rewards for decent behaviour? What's holding 'that back' is an evident paucity of scientists (in this small area anyway) with integrity and a desire to discover the truth, come what may and irrespective of the views of their paymasters, relating to their speciality.

You ask for a mechanism to 'evaluate "impact" of linked data someone puts online'. I'm intrigued. What on earth do you have in mind? How would it work? On the face of it (I trust I've got this about face) it sounds an appalling idea. A load of value judgements delivered by whom?

yungchin2• 17 years ago

Just to be sure we're talking about the same thing: the article above is advocating that we all publish every detail of our research proceedings online. What I'm saying is, that doing that would jeopardize your position as a researcher: the average labjournal is full of thoughts and ideas for further work. Researchers however don't get rewarded for inspiring peers, they only get rewarded for doing the work.

If we want to encourage people to disclose all their data, we need to get rid of the system where grants, tenured posts, and awards are all based on authorships only.

I'm not sure which idea you think is appalling, as I don't have any idea as to how to do it! One way would be to establish something similar to PageRank: if you publish a database of, say, world-temperature readings, an indicator of its value might be how many online labjournals in the world draw data from it. That's a very raw idea, and obviously not good enough, but calling it appalling before I even came up with it?

cswetenham• 17 years ago

"What is required is the wholesale adoption of the standards applied in the pharmaceutical industry drug trials: double blind trials (aka peer reviewing)"

I think the gentleman who authored this article neglected to look up the terms in question.

http://en.wikipedia.org/wiki/Blind_experiment#Double-blind_t... http://en.wikipedia.org/wiki/Peer_review

lutorm• 17 years ago

Rofl. I gave up on the article after the first paragraph, so I missed that nugget.

Did he explain how you conduct a double blind trial on climate change? That's like asking me (an astrophysicist) to conduct a double blind trial on the Big Bang.

It's not possible to do trials on some systems, either because there is only one or because it's impossible to manipulate it. In those cases, you are unfortunately stuck with collecting data and building (physically reasonable) models where you try to match those data and suggest new data which are predicted by the theory that can be used for testing it.

It is important to do the latter. Just because you can adjust a bunch of parameters in the model to match existing data doesn't give a lot of confidence in your model, because of the feedback loop that causes you to work on the model until it fits the data. What you need to do is then say "given this model, we predict x" and then go and look for x.

From what I know about the climate modeling, it would be nice to see more of the latter. I know there are some things (development of regional patterns of changes) that match the models, but I don't know whether those were pre- or post-dictions.

Open Science and climategate: follow CERN's lead

Related Stories