We don't know how to fix science (2021)

https://www.worksinprogress.co/issue/we-dont-know-how-to-fix-science/

by bookofjoe • 4 years ago

106 112 4 years ago

willis936• 4 years ago

What is the thesis here? Inaction because of insufficient certainty? Isn't that the exact issue that landed us in this mess?

Science requires action in the face of uncertainty.

abetusk4• 4 years ago

The thesis is that we should be experimenting on how to get best results. In other words, we should treat science funding scientifically, testing hypothesis, examining results and course correcting based on the evidence.

To your point about inaction, from the article:

"""

This does not mean we should wait decades before implementing any change. Waiting for strong, crystal-clear evidence to act would be engaging in the same flawed thinking that led to the claim that “there is no evidence that masks work” which we heard last year—the costs of delays or inaction would be high. Demands for open access, “red teaming” science, or running multiple-lab reproducibility studies (be it in the social or life sciences, or elsewhere) shouldn’t get stalled by the lack of RCTs. Where there are strong theoretical considerations, indirect evidence, and broad agreement that a proposal will improve science, without a serious cost if we’re wrong, we should just go ahead in at least some cases, and assess the benefits afterwards.

"""

The last paragraph sums up the thesis:

"""

Those interested in meta-science may disagree about what the best way to reform science is, but all of us can agree that we need more evidence about the proposals being made. We have many interesting, reasonable ideas ready to be tried. It is a glaring irony that the very same institutions that enable practitioners of the scientific method to do their work don’t apply that same method to themselves. It is time to change that.

"""

fiprofessor3• 4 years ago

Do people really find the thing mentioned in the last paragraph that ironic? To be honest, I do not (except, maybe, if the scientists are economists who work in industrial organization, or something). Many natural scientists I know are deeply skeptical of our ability to accrue this kind of subtle knowledge about complex human processes and social organizations through experiments...

thrown_224• 4 years ago

. Waiting for strong, crystal-clear evidence to act would be engaging in the same flawed thinking that led to the claim that “there is no evidence that masks work” which we heard last year—the costs of delays or inaction would be high.

We are getting a real time test to the question of if mask mandates work in the wild: https://www.abc.net.au/news/2022-07-22/nz-jacinda-ardern-cov...

Australia has no mask mandates, NZ does. NZ has a higher death and hospitalization rate than Australia. The answer is that it's health theater on par with shoe removal at airports.

Of course we have the typical bait and switch of pretending that masks as used in an operation theater are in any way the same as the crumpled cloth mask from your pocket.

fuzzfactor3• 4 years ago

If Yoda were a scientist he would say something like,

No proposal

No committee

No eminence

No hesitation

Just do

ModernMech4• 4 years ago

I would love to just do!

But I need money to just do. Approximately a million dollars.

Who is going to give me a million dollars without a proposal as to my plan on how to spend that?

Is one person going to be the decision maker on who gets money? That seems like it would be susceptible to strong biases. A committee could make decisions with less bias.

And how will they discern between proposals? Money isn’t unlimited, and a million dollars is a lot of money — you don’t want to just give that away to anybody. Some prior track record of success on the part of the applicant would help demonstrate their ability to accomplish the proposed research. Maybe they would demonstrate that prowess somehow, through written means perhaps, like a document, published for all to read.

So from first principles we’ve arrived back at the current situation. I don’t think things are perfect, but throwing out the whole system without understanding why it’s there will just lead to more dysfunction.

Yes no one likes writing grants, and yes publishing can be gamed. But until you solve the fact that research costs money, and a few people have control over most of the available money, then we’re (scientists) going to have to spend a lot of time convincing them to give us some of what they have.

OnlyMortal• 4 years ago

Yes we do. Stop the “publish or be damned” daftness and hire people who are actually trained in the disciplines.

That will cost more but that’s a solution.

throwawaymaths• 4 years ago

Read the first paragraph. We don't know if those strategies will work; there are no data.

gonzo41• 4 years ago

Sounds like a great moment to try an experiment.

mjevans• 4 years ago

Q: Science is 'broken' (faked / rosy bias in data and publication, less useful results, lack of public trust, etc). // "How do we ''fix'' Science?"

Hypothesis: TL;DR Goodhart's law: 'the measure became the target and thus is no longer a good measure' -- Publish or die / secure funding or die has lead to risk averse research, a lack of research into new fields, and a lack of reproduction / verification of existing research (checking the known is not 'sexy' / profitable).

Experiment: Proposal: Create a 200 year funding guaranteed tenure tracking inflation funded public domain / commons expansion researcher reward that ensures a life long appointment (and retirement) based on some other excellence requirement. Maybe revisit former initiatives like 'project paperclip' (WWII) for criteria suggestions.

Data Collection / Reports: Conduct the real experiment, for the first of at least 200 years OR until all enrolled applicants retire / expire. Issue interim reports every couple years (4, 10, something) to see if clear results contrary to the typical environment persist.

NegativeLatency• 4 years ago

Maybe make academia not have a reputation for burning people out?

ModernMech4• 4 years ago

Tbh a lot of academics burn themselves out. The pressure is real, don’t get me wrong, but a lot of fellow academics I’ve come across are people who have no clue what a work/life balance is. The pressure is internal as much as it is from “academia”.

fiprofessor3• 4 years ago

I see a lot of appeals to "fund people, not projects", but the main issue I have with it is that I think it strongly biases people to select "super stars" who have impressive credentials / come from high status organizations. Of course, there's already a lot of "rich get richer" effects in science, as the article points out, but I think it would be nice to try to shift away from those things, not increase them. (Besides, isn't the resounding message in academia these days supposed to be that assessing people is full of biases that we ought to avoid? Of course you always assess the people behind a grant application to some extent, but focusing on the project would seem to be better in terms of avoiding these biases.)

The other alternative of "funding lotteries" is more suspect. As the article summarizes, the argument for lotteries is:

Advocates of lotteries make two key critiques: a) the current system forces researchers to spend a lot of time preparing grants; and b) peer reviewers cannot reliably identify “good” grant applications. They claim that a lottery system would reduce the time spent on review (because reviewers would mostly skim the proposals to check for minimal scientific robustness) as well as the time spent on preparing proposals (because there would be less of an incentive to meticulously craft proposals, given that no matter how detailed and well written they are, they are going to be chosen at random).

Of course (a) is true, but as the article suggests, the evidence for (b) is a bit shaky. Moreover, we should have strong priors against (b). To be sure, reviewing is not perfect, far from it! But, scientists must at some level be able to judge the future prospects of work, or else they wouldn't be able to make any fruitful decisions about what research to conduct. So the primary plausible way I see that (b) could perhaps be true for a given pool of applications is if all of the ones in the pool pass some thresh-hold bar for quality that makes it hard to further distinguish between them. It's not clear to me that that quality level would be maintained if we move to a lottery.

antasvara• 4 years ago

I think that a lottery has potential if we use peer review as the bar for quality.

My prior is that peer review is better than random chance. This seems sensible; after all, a human being should be better at determining scientific validity versus random chance. Under that assumption, it makes sense to have peer review to narrow the pool down to a smaller number of grant proposals.

It doesn't seem unreasonable that peer review could get the "best" proposal into a pool of 10-15 applicants around 90% of the time. I think that would reduce the pressure on proposals by a decent amount.

towaway154632• 4 years ago

Isn’t that what’s broken? Everyone is trying to game peer review.

catgary• 4 years ago

I think funding lotteries are rarely proposed as straightforward lotteries, there is usually still an evaluation step that filters some 50% of proposals. It’s based on the observation that there is usually consensus about what is a reasonable grant proposal, and very little consensus about what is a great grant proposal.

fiprofessor3• 4 years ago

Yes, I agree that it's the fine distinctions at the upper end that are hard to make. But if we're going to eliminate 50% of the proposals, is that really going to reduce the amount of time people spend on them? It might reduce reviewer time (which would certainly be welcome!), but I'm not so sure it reduces the submitter's time.

It comes down to whether the large amount of time spent is from people polishing and re-submitting stuff that's good but not "great", or whether it comes from the initial stages of getting something to that good quality to start. In my experience, it's the latter. And if that's typical, then I don't see how the lottery helps.

HardlyCurious2• 4 years ago

What happens when, and I know this will be rare, our ideas of reasonable are wrong? Seems like that ends up being a dead end where the research needed to correct our misunderstanding can't get funded.

cossatot• 4 years ago

Please, someone, correct me if I'm wrong, but I think this is essentially how the European Commission allocates scientific funding--professors basically get big block grants to fund their labs to do whatever. Then there are some big grants that go to institutions, with those like ETH Zurich soaking up a lot of the funds, and then those institutions can distribute to the smaller ones as they see fit. Rubbing up to high-profile professors at ETH or GeoForschungsZentrum feels a lot like petitioning King Frederic of Prussia, rather than anything merit-based.

It's really not clear to me that this is a better system; at least in my field (geosciences) I don't think that the European labs are more innovative or productive than their American counterparts, which is what the 'fund people, not projects' idea is supposed to enable. Furthermore, it completely blocks nonacademic research funding, or much project-specific funding. For example, I work for an applied science nonprofit based in Italy, and we have a lot of ideas that would (in principle) be big hits at NSF, but there is no real avenue to fund ~€100-300k projects on an ad-hoc/project-specific basis.

JPLeRouzic5• 4 years ago

but there is no real avenue to fund ~€100-300k projects on an ad-hoc/project-specific basis.

Long time ago I had a colleague who lamented that his projects were never selected by the R&D funding committee of our employer. I suggested him to multiply by ten his proposed budget and then at the next round his proposal was accepted! This was at a time were there where much more money in the Telecom sector than now.

So please do not hesitate to multiply by 10 your proposed budget. It does not have to be more ambitious on your side, all is needed is to find additional partners who could bring value to your proposal, for example by implementing it.

(I was both work package and project leader for EU FP7 projects)

fuzzfactor3• 4 years ago

In Ricón's Fund People Not Projects III article [0] he relates:

Ioannidis et al. (2014) note there are 15M scientists that published anything in the 1996-2011 period, but only 1% that has published every single year in this period. This smaller 150k-strong group accounts for 40% of all papers and 87% of all papers with >1000 citations.

One percent seems like a good size for a major niche; those who do the bulk of the publishing.

This is important, but if that's the only niche you're looking at you're bound to miss 99 percent of what you want to find.

[0] https://nintil.com/newton-hypothesis

Edit: Maybe the people that need to be funded most are secretaries for some of the other 99 percent.

JPLeRouzic5• 4 years ago

you're looking at you're bound to miss 99 percent of what you want to find.

Or those 1% are real professional that know their stuff while the other are publishing because they are doing some internship, or publishing low quality articles on an obscure domain is a way to evade criticism (how many ALS experts in academia?) or other short commitment to a topic.

I read a lot of literature on ALS (Lou Gehrig's disease) and clearly 99% of publications have no value at all, even when their university claims of "a breakthrough".

As far I know there were breakthroughs only every ~5 years (TDP-43 in 2006, C9orf72 in 2011, FUS 2010) and no similar breakthrough since. Big companies like Biogen, which bet much on various genetic (ASO) therapies, are now removing drugs from their pipeline. We clearly do not understand the field, despite publishing 15,000 articles per year!

mike_hearn2• 4 years ago

The main issue with the people-not-projects idea is a transition problem. New systems are desired because the current system isn't working well enough. How do you identify the superstars? By relying on metrics created by the current system.

In particular, any system that identifies superstars would probably have identified Lesné, the guy who seems to have been forging Alzheimer's research. He had already obtained huge and lavish grants for his lab on the back of this work. Winner-takes-all is a system that would throw accelerants on the fire of fraud, as presumably once a scientist has managed to get a blank cheque cut, nobody is incentivized to scrutinize how that money is being spent too closely. They're a superstar, after all.

autokad• 4 years ago

there are so many major problems with 'fund people not projects'. Science is already a little bit broken. Scientists aren't supposed to dragnet data until they get the results they want, but it still does happen. however, if we were funding people and not projects, this would happen almost every time.

I also think politics in science would become even more of a problem. all of our money towards science would go toward funding research that backs their worldview.

I also think we'd see even more social media like driven science. "I am going to research this because I can write on social media/youtube on this and get lots of click/likes-> money"

mikkergp• 4 years ago

Science does indeed have a lot of problems, but is the problem really that science is broken, or that our expectations of it are.

Ideally, we’d like to measure the benefit provided by a study to society. We might ask: had this piece of research not been funded, would a given invention have been delayed? If so, for how long? And what was the impact of that on societal welfare? We could also try to estimate intermediate metrics of research usefulness, like whether a given basic research paper will end up being used as input to an application. It is an argument for humility, not epistemic nihilism.

The whole promise of basic science research is that it may be valuable in ways that are immeasurable, or certainly uncapitalizable. One major issue with science is that we're always hoping that it will say more than it should. I would take a more mindfulness-y approach to science. You do science because it is a good end, and if you do it for that reason, good things will come. If you chase "good science" you get the problems we have.

sicp-enjoyer3• 4 years ago

If that's true, then it doesn't make sense for the public to invest so heavily in it. Humanities research also is valuable in ways that are immeasurable and is also a good end.

The public got invested because it had a lot of useful outcomes in the 20th century.

hiptobecubic• 4 years ago

The point is that you don't know up front if or when that's going to happen.

donatj• 4 years ago

Capital S “Science” where the goal is continuous funding and reputation growth are in a likely unresolvable tizzy as long as long as humans have confirmation bias and a desire to be right. There’s far too much reason to cheat and to lie with p values, even if unintentionally. This isn’t new, it’s just had a light shown on it.

Applied sciences where the end goal needs to be something that actually works and not just getting published are in much better shape and arguably innately of more value to begin with.

kaba0• 4 years ago

But isn’t the solution to the former more funding? Make studies that check on existing ones worthy (especially if they contradict those!)

tomcatfish• 4 years ago

Many people would like to get things like "cure to disease X" or "reliable way to make people healthy" or "way to transport crops to hungry regions" before things like "green jelly beans linked to acne", and this is what the complaint is about. We can fool around with jelly beans _after_ we have done the more important things.

alexashka• 4 years ago

The same we didn't know if cigarettes were harmful, right?

There is just so much we don't know, every time someone is profiting real nicely.

And so much we do know every time someone is profiting real nicely.

bilsbie• 4 years ago

What if we separate collecting data from analyzing data?

andrewclunn• 4 years ago

This is true in some disciplines, but largely inapplicable in others. A great idea, but I'm not sure it would work outside of physics or chemistry. I mean imagine trying to apply that to medical research. But it's a novel idea, so thanks for sharing.

mmusc• 4 years ago

and the collected data should be always available

analog31• 4 years ago

This sounds like the "waterfall" model of project management. I could see it working in some areas and not others. In fact, a friend of mind wrote a thesis on a re-analysis of an existing data set from one of the big physics experiments, looking for a new effect that he had proposed based on theoretical work.

But I come from a background in small-lab experimental physics, and my spouse from synthetic chemistry. In both cases, a model of planning, followed by execution, followed by analysis, doesn't work, for a number of reasons. Often, an experiment fails, over and over again, until the design and operating conditions are refined to the point where the data begin to make sense. In my experiment, the equipment didn't even exist until I built it. And in experiments such as mine and my spouse's, the researcher (grad students) are also developing and refining their own abilities as they progress. I was my own electrical, mechanical, and software engineer. Sometimes, preliminary results change the direction of the project.

An additional issue is that experiments are rarely documented well enough to hand the data off to another team to analyze, without significant back-and-forth.

What little I understand about the "agile" model, seems more applicable to this kind of science.

JTBooth• 4 years ago

Very different operating modes are required depending on how many people you're trying to coordinate. The allegory to software development is pretty clear - sometimes you're working on a personal project and zero documentation is tolerable, but sometimes your work is going to be translated into 200 languages and distributed to a hundred million users the day after it ships, and it needs ten times as many lines of tests as lines of code.

It seems to me that our model of "collect the data and analyze it yourself" is a sort of "ten engineer startup" scale process. Now that many fields of science have four or five digits of PhDs collaborating between countries, there's an increasing need for specialization, specifically in creating reusable data. It'll make us less efficient on a small scale, but creating any artifact at all that can be reliably used by ten thousand people is much higher leverage than a fully packaged data+conclusion that's even odds to be nothing but noise.

nemlx• 4 years ago

perhaps we should scale back government subsidies of mediocrity and remove the bureaucratic bloat that is ill-informed, ill-equipped and ill-incentivized to determine what is the "right amount" and "right goals" of basic research.

despite popular beliefs to the contrary, private citizen donors and industry does indeed understand the value of basic research, and may even be better positioned to value it properly as a whole, despite the private interests of them on an individual basis.

biomcgary• 4 years ago

Uncritical funding of projects submitted by superstars seems to be one significant problem in funding that overly reinforces the status quo (like other rich get richer phenomena). In some fields, only a small fraction of applications (8-15%) are funded and much of it goes to "superstars". I would propose that the bottom 50% of applications be removed by review and the rest be funded at random. In order to reduce spam applications, it would be necessary to adjust the probability of funding by the total amount of money being applied for the previous 24 month by a given PI.

To encourage cross-disciplinary research, I would award the top 1% of projects non-randomly but only if the PI has not been previously funded by the granting agency.

UIUC_06• 4 years ago

The answer to "how do we fix science?" could only be some committee or bureaucracy, and that will probably work about as well as getting a bunch of HR people together to decide "how do we fix Engineering?"

The author's bio says:

José Luis Ricón is a book reviewer and blogger on various topics including longevity and a roadmap for the future of science at Nintil. You can follow him on Twitter here.

This is certainly ad hominem but is there a reason to think Mr. Ricón has any insights, experience, or publications that make him worth listening to?

prichino2• 4 years ago

You're absolutely right.

I'd argue that to make science what it used to be, one would even have to cut funding.

Scientists used to be aristocrats interested in truth. These days they're just interested in grants.

wrp• 4 years ago

Ha! Just yesterday, I was talking to a colleague in the humanities about the relative pleasures of working in a field where there were practically no grants to be sought.

digitalronin• 4 years ago

Scientists used to be aristocrats interested in truth. These days they're just interested in grants.

I'm not sure if that's actually what you're advocating, but personally I'm very much against science only being the domain of rich people who can afford to do it as a hobby.

pvaldes• 4 years ago

Scientists these days are just interested in grants.

Ladies and gentlemen, to add insult to injury, the greedy scientist myth strikes again.

There is not a better feeling in the world than to work for free for several years, six days a week (plus Sundays of course), with the weak promise of receiving a "turtle grant" someday. Maybe. (Tomorrow perhaps, the turtle is blocked somewhere. You will be paid in two years) and then being called "greedy".

Yes, there are people getting rich with the money for science, It just happens that they aren't the scientists. They are the politicians that keep the grants as their own personal loans. Science can't be fixed because is the guarantor of politicians that only need to go creative and put impossible requirements year after year to be allowed to keep most of the money for themselves.

In 2010 under the president Zapatero, the spanish government de-funded first and then keep for themselves the 25% of the money compromised for science. In 2015 under president Rajoy the government keep the 48% of the funds allocated for science and in 2016 the 62% of the promised money never reached any researcher. The money was used instead for, who knows... a new swimming pool for each minister maybe, or fixing accounting holes in other sectors... I wouldn't discard cocaine parties neither.

https://elpais.com/diario/2011/04/02/sociedad/1301695203_850...

https://elpais.com/elpais/2017/10/04/ciencia/1507133529_8680...

wrnr3• 4 years ago

People I know in academia complain about the same issues, publish or perish and all that. They even opine about formal economic models that predict that more funding/subsidies for science leads perversely to less outcome in total and not just relative to the size of the additional investment. However when you take all this as a reason to not take this industry as serious as they would like they get all offended.

epgui4• 4 years ago

However when you take all this as a reason to not take this industry as serious as they would like they get all offended.

Just because there exists valid criticism of a thing, does not mean all criticism of it are valid: and in my experience, I mainly roll my eyes in exasperation when people criticize science for unfair reasons.

What the public thinks is broken in science is very often not what experts think is broken in science.

debacle• 4 years ago

What is there to "fix?"

The system is working as intended in many ways. People just aren't aware who the meaningful stakeholders are. It's not the PhD, it's not the taxpayer, and it certainly isn't "science."

jrochkind12• 4 years ago

That is the kind of thinking that we need to change. Instead, NIH should have considered selecting a subset of investigators and applying a cap to them, and then compared results a decade into the future with those that were left to accumulate more traditional funding.

If you are one of the researchers who is in the experimental group with lifetime capped funding (or at least a decade of capped funding, until the study is up), your career is going to be really impacted, isn't it?

I can't believe the OP suggested this with a straight face, it's obvious why it wouldn't fly, and why NIH was considering only doing it universally, to at least treat all researchers equally, right?

But yes, I understand the argument for how an experimental approach would help us determine the best funding models... although it's not really clear to me how this experiment works. Even assuming that we know how to measure which group did "better" after 10 years... wait, what are you expecting to see in the comparison between the research-capped group vs not in determining if it worked? I don't actually underestand this experiment.

Anyway, regardless, it's not a mystery why it's not going to happen, right? Experimenting with researchers careers.

next_xibalba2• 4 years ago

If you are one of the researchers who is in the experimental group with lifetime capped funding ... your career is going to be really impacted

If you believe, as I do, that a functional "science" is the most important force in making the future better than the past, would not this be a worthy sacrifice? In terms of its impact on all of humanity, now and forevermore. I would also think that true scientists would at least be the most likely to be sympathetic to this rationale.

unlikelymordant• 4 years ago

Or theyd move overseas where they can get more funding.

jrochkind12• 4 years ago

Do you know many employed academic scientists?

Why don't you ask them if they would be willing to accept a cap on their salary, to be excluded from the most prestigious and well-paying institutions, and to do less research than they otherwise would, for a period of ten years, because they will be participating in an experiment which, if it was designed and executed well, might result in findings that might help improve research funding.

Perhaps very few actually practicing scientists are "true scientists".

lmm2• 4 years ago

If you offered a floor as well as a cap, I suspect a lot of researchers would be willing to take that deal.

mbg721• 4 years ago

One problem is that finance and big business siphon off bright graduates in search of benjamins, and all you have left is academics looking for grants.

q-big2• 4 years ago

In my observation, the graduates who go to finance and big business are a different breed of people than those who would love to stay in science.

I don't think the typical person who deeply loves science becomes happy in finance of big business. On the other hand, the people who go to the latter don't have the deep love for research that is necessary to bear an academic career.

Also, in my experience (I know some examples), the typical hiring processes of finance and big business weed out the bright people who have too much of a love for science.

So, I don't think what you mentioned is a problem.

stonogo• 4 years ago

This article could use some peer review. Not the author's peers, who appear to be "science pundits," but from people with experience in managing research portfolios.

bergenty2• 4 years ago

We have the answer. Peer reviews should include a separate team replicating the results (follow methods, without knowing the results). Those that are replicated go into a higher acceptability tier. Those that do not get published in another tier. If replicating is not possible (for instance in the case of the LHC), the paper should go through 3 independent and simultaneous peer reviews.

The problem is not enough “gatekeeping”. Accepting a publication should be more stringent.

naasking3• 4 years ago

I've often said that grants should include funding for at least two independent reproductions. Also, journals should prioritize publication of negative results on par with positive results.

Trial pre-registration and open science are good first steps though.

ModernMech4• 4 years ago

Okay, sounds reasonable. Except that peer review is a volunteer activity. Who is going to pay for all of these replication studies? Where does that money come from? If we can’t increase funding budgets, which are already paid extremely paltry, you’ve effectively cut research output in half. Maybe that’s a win, maybe not.

This reminds me of the welfare debate. On one hand, we could erect a giant bureaucracy to take a fine tooth comb over every application to make sure only the “worthy” get aid, which creates holes through which needy people fall. Or we can accept that driving fraud to 0 is impossible and get rid of the bureaucracy, then put the savings toward more aid. Sure some unworthy people will get it, but others who are worthy (and otherwise wouldn't have) will, and that’s more important.

bergenty2• 4 years ago

If it’s standard practice that a publication involves replication, it becomes a standard part of funding. Will we have fewer studies? Maybe. Will they be much higher quality? Yes.

This is not like welfare. We are trying to enact a stringent gatekeeper so studies are completely reliable. The analogy doesn’t work here.

8bitsrule• 4 years ago

Maybe the ponderings of a highly-respected thinker of the 20th-century might be useful in this discussion, this PDF includes Bertrand Russell's essay "Science and Values". The essay's found on p. 619. of The Basic Writings of Bertrand Russell, a section of Part XVI, 'The Philosopher and Expositor of Science'.

[https://emilkirkegaard.dk/en/wp-content/uploads/The-Basic-Wr...]

"Science used to be valued as a means of getting to know the world; now, owing to the triumph of technique, it is conceived as showing how to change the world."

That it has.

hikingsimulator• 4 years ago

Something that a lot of people forget is that the pondering of those highly-respected thinkers was due, in large parts, to them hailing from influential and rich families.

Russell? British aristocracy. Von Neumann? Heir to an Hungarian and newly titled magnate. Wittgenstein? Wealthiest family in Austro-Hungaria. De Broglie? French aristocracy. They, of course, were insanely smart, but putting that intelligence to such a wonderful use was enabled by their background.

This is a sad state of affairs for researchers today because, by default of having the pre-existing money to explore, they must become subservient to it: grants and fundings.

hyperpallium2• 4 years ago

private tutors are probably the key: https://wikipedia.org/wiki/Bloom%27s_2_sigma_problem

kqr• 4 years ago

This is basically the difference between the proletariat and the bourgeoisie: one needs to work for someone (even if only for its customers, as a business owner) to survive, and the other can do whatever they want while living off of previously accumulated capital.

Not having to worry about how to get bread on the table opens up the door for all sorts of interesting creative labour!

thrown_224• 4 years ago

Einstein? Clock-makers son.

Newton? Grew up with a reverend as a step father.

Kelvin? Son of a teacher.

Faraday? Son of a blacksmith.

Plank? University professors.

Our obsession with credentials is preventing more people from being scientists today than the classism of the 19th century did.

nurettin2• 4 years ago

Although it may help us locate underlying weaknesses in the scientific method, this level of self-consciousness displayed in the public eye is hurting society, ignoring the value of rigor and letting nutjobs get away with their ridiculous claims for a quick buck.

naasking3• 4 years ago

I disagree 100%. Science journalism and unreproducible results of the last few decades is what has hurt science's credibility. These public conversations of science's failings and how to fix it are the only thing that can restore its reputation.

ModernMech4• 4 years ago

From my perspective as a scientist, a lot of the discussions here about "how to fix science" and "what's wrong with science" suffer from the same problem as science journalism; namely, people who are not experts in a subject trying to act as experts for others who are even less informed. For instance, the author of this TFA is a blogger and a journalist on "various topics". Don't you imagine he suffers the same problem as science journalists, which you call out as hurting science's credibility?

nurettin2• 4 years ago

Congratulations on being super strict, but this doesn't address half the problems. So you can only disagree 50%.

q-big2• 4 years ago

This is of course not a "fix" for science (I don't believe such a universal fix actually exists), but a method to highly increase the effectiveness of the scientific process:

Let the scientists simply do science or teaching (which is actually a method to educate the next generation of scientists) instead of having them handle an insane amount of academic bureaucracy in their work time.

hgomersall• 4 years ago

Right - novel idea: how about academics and post docs are paid a fair salary for their work. You could require that post docs need a position with a PI to be paid and that everyone is assessed on what they achieve not what they say they're going to achieve. Post docs can follow the PIs that interest them until they make tenure.

The only question then is around equipment, which though is not trivial is at least smaller than salaries and equipment.

mejutoco• 4 years ago

I feel like some suggestions try to fix the problem without changing much. We could instead change the roots of the problem. Education and access.

- Making public funded research publicly available (since it was paid with public funds): papers, etc. for anybody to access, legally, for free.

- Lowering the cost of textbooks.

With more people having access to the materials we might realize there are a lot of latent geniuses out there in need of a chance. The easier to get access to those papers and materials the easier it is to double check the results, too, and to cross-pollinate between fields.

dsqrt• 4 years ago

DOE [1], NASA [2], NSF [3] already require most research papers to be made publicly available.

---

[1] https://www.osti.gov/

[2] https://www.ncbi.nlm.nih.gov/pmc/funder/nasa/

[3] https://par.nsf.gov/

AstralStorm2• 4 years ago

That still leaves most papers in expensive journals or available via institutional access only.

(PubMed, arXiv and biorXiv help a bit with that, but it's still not enough.)

sazz2• 4 years ago

Science and software both have the problem of dealing with complexity.

Imagine what software development would look like if we applied the principles of science:

We wouldn't start from scratch with a new software program. Instead, we would refer to code from the 1960s that has been proven to be a development-theoretic consensus and accepted by developers.

We would not speak in software life cycles, but would either "revolutionize" a piece of software or extend the existing standard.

We would allow that we trust code from developers because some other developers say it will fit. However, we don't know if the "other developers" would have the same financial interests.

And we would allow interested parties with enough resources to influence this process at will by publishing permutations of similar software and trusting each other.

We would have our own marketing agencies that are adept at injecting arbitrary constructs of their own into the "developer consensus".

And every developer who refuses to do this, who develop their own frameworks and pursues their own software ideas in a detached manner, would be branded as a liar and their reputation be destroyed with the help of the "developer consensus" spoke persons under the payroll of corporate interests.

probably_wrong• 4 years ago

I started reading your list thinking I'd get an essay of how ridiculous software development would look like, only to get a list that more or less matches what a significant part of software development looks like.

I guess Poe's law strikes again.

sazz2• 4 years ago

There is no "right" way in how to deal with complexity.

Yes, both disciplines have their own way of handling that subject. And each discipline is struggeling how to find a way to gain evolution.

For me the key performance indicator is that in software development people controlled open source now takes a crucial part in the basis of most software projects whereas in science everything is much more corporate infected.

In science knowledge is created where money is. And money comes from corporate interests.

In software corporate interests tried to control the market but in the end had to capitulate. Most servers are run based on open source, most software is developed with tools being open source. Of course, there is no standard. You can write an application in Vue, React, Angular, JSP or whatever.

Yes, in the end software industry is insance. They have managed to get accepted that every five year the old software is scrapped and basically the same software is developed again.

But letting old software die and creating a new version with new ideas in mind is much closer to the way nature is doing things.

And to be honest - the only one who really can deal with complexity is nature hands down.

ponow• 4 years ago

How to fix science? Outside of that directly tied to government activities like national defense, stop all government funding. Consequently, any funding science will end up by people who have skin in the outcome, grounding the entire endeavor.

superchroma• 4 years ago

Corporate sponsors of research solicit favorably selective scientific literature all the time. Too often it's bad sample sizes, bad samples, imprecise methods, lack of rigor, overenthusiastic conclusions not supported by data, etc.

We don't know how to fix science (2021)

Related Stories