Harvard professor Francesca Gino was accused of faking data

https://www.businessinsider.com/harvard-francesca-gino-fake-data-fraud-allegations-scholar-2023-7

by robtherobber • 3 years ago

133 141 3 years ago

rdtsc3• 3 years ago

When discussing Gino with Insider, multiple people brought up the idea of "me-search" — that researchers gravitate to topics that are of personal interest to them. "We're our own therapists, in a sense," Gordon Pennycook, a behavioral-science professor, said.

I guess that might be common with students picking psychology as a major, too. Noticed it with a few acquaintances of mine.

In a study about "contagious dishonesty" that Gino coauthored with Ariely, the researchers found that students were more likely to cheat if they saw someone they believed to also attend their university cheating.

That’s fascinating since both of them fabricated data. At least they can add themselves as the N+1 data point to their study. It’s retracted already, but still good enough for a TED talk or a book probably. /s

Wonder if we have many sincere “coming clean” stories where they explain the thought process behind it. Why they did it, at what point did they decide to go through with it? Did they share their secret with others. Did they feel shame, etc.

thumbuddy• 3 years ago

I've never seen a coming clean story. But this has been going on at every scale for decades.

The scientific method isn't broken but the institutions that surround it and the people riding the tops of it tend to be. It'd be easy to fix but you'd instantly make a million high power enemies with psychopathic tendencies. Oh well.

iamandoni• 3 years ago

The book Faking Science by Diederik Stapel is an excellent view behind the scenes of a serial academic fraudster.

19870213• 3 years ago

That's one way of turning a vice into a virtue... "Right, I've been found out to have fabricated years of research, entire PhDs are now junk. How can I make some more money out of this?"

rdtsc3• 3 years ago

Thanks for the advice. I read a few pages and it was well written. I'll continue reading it.

makeset• 3 years ago

I guess that might be common with students picking psychology as a major, too. Noticed it with a few acquaintances of mine.

That's no secret. Any psych professor will tell you, it's rare to find a student (and consequently, psychologist) whose interest was not sparked by a personal history of mental illness.

andersco• 3 years ago

Podcast that describes many of the specifics of how her and another researcher faked their data: https://podcasts.apple.com/us/podcast/planet-money/id2907834...

ipsum22• 3 years ago

If you don't want to listen to it, the podcast description is a nice summary: https://www.npr.org/2023/07/27/1190568472/dan-ariely-frances...

MengerSponge• 3 years ago

Holy hell that statement from The Hartford is devastating. Insurance companies are famously risk-averse, so for their legal team to sign off on that...

runarb• 3 years ago

https://archive.is/dY6PE

ipsum22• 3 years ago

In a similar vein, UPenn professor Angela Duckworth of "Grit" fame was caught misrepresenting statistics to embellish her claims a few years ago: https://www.npr.org/sections/ed/2016/05/25/479172868/angela-.... Who knew psychology had such questionable science?

marginalia_nu2• 3 years ago

Who knew psychology had such questionable science?

Well given psychology is one of the two epicenters for the replication crisis, I didn't think this should be too unexpected.

tgv3• 3 years ago

It is to the merit of psychology. Other branches of social sciences don't dare to propose such a thing. They know that their replication numbers are going to be worse than psychology's.

implements• 3 years ago

Not in the field, but this recent podcast covers it:

https://podcasts.apple.com/gb/podcast/two-psychologists-four...

Harkening back to episode 73, Alexa and Yoel discuss recent evidence of fraud documented in the Data Colada blog post "Clusterfake." The post is the first in a series of four, which will collectively detail evidence of fraud in four papers co-authored by Harvard Business School Professor Francesca Gino. […] Finally, they consider what this means for a field still struggling to build a more trustworthy foundation.

HDMI_Cable• 3 years ago

What's the other epicentre?

tarvaina• 3 years ago

Based on the NPR article you linked this sounds like a normal scientific debate, though? One researcher says that data supports their theory, another says that it doesn't if you look at the data more carefully. I don't see accusations of fraud here, just different interpretation of facts, and a valid challenge to an earlier theory. This is how science is supposed to work, isn't it?

Gino, on the other hand, is accused of outright fraud. Data Colada claims she took real data and changed it to support her theory, or possibly created entirely fake data.

brmgb• 3 years ago

I mean, people don't like to say it because you get taxed of being snobbish at best or lacking culture and being close-minded at worst but everyone know that "soft" sciences are mostly bullshit.

I started my eductation with a bachelor of maths and an engineering degree before doing a lot of economics while in business school. The field is full of interesting ideas but methodologically it mostly is garbage even you stick to the more mathematically oriented part of it. And that's economy which is supposed to be the most serious of the soft science.

twobitshifter• 3 years ago

Agreed, I have a similar background. Science is done in economics, but a lot of economics ends up being thought experiments turned into a graph eventually taken as fact. Once you get into the science of the shapes of everyones’ multi dimensional personal utility curves and rational behavior you are drifting from actual observed behavior. That’s the pivot to the other side of irrational behavior that Dan Ariely loved to push based on observations, but now we find that’s full of false claims.

brezelnbitte• 3 years ago

Its not just psychology. All science is affected. Stuart Ritchie’s Science Fictions describes cases from across many disciplines. The worst are cases of misconduct in medical research with maybe the worst case being Paolo Macchiarini who fooled the Karolinska Institute of Nobel fame into letting him experiment on patients with trachea replacement surgeries: https://news.ki.se/the-macchiarini-case-timeline

veave• 3 years ago

Trust the science.

friendzis• 3 years ago

Believe in science*

dekhn• 3 years ago

Have faith that science is the least bad option we have

acatton• 3 years ago

Many researchers are privately terrified of being falsely accused by the "data cops," one scientist said. But no one wants to criticize them because no one wants a target on their own back — including this individual, who was granted anonymity for this reason. "Batman is a vigilante," the scientist said. "So is the Joker."
For collaborators, it's a stressful and infuriating time. When a paper is retracted, the research is erased, and all related citations are lost.
Others who attempted to build on Gino's studies are grappling with having wasted time, money, and energy.

It feels, to me, that they're making Data Colada (the data vigilante group) the bad guys here. It's like blaming the Police for stopping a criminal because they took away the criminal's source of income... I'm kinda uneasy with this line of editing :/

tgv3• 3 years ago

Indeed. That's nasty. They throw in a detracting bit of racist accusations as well.

all related citations are lost

What an extreme BS argument. It's really grasping at straws.

prepend4• 3 years ago

The citations aren’t lost though, are they. They still exist in the paper, they just point to a retraction. You can still find the papers that cited the retracted paper.

And I suspect you’ll still get new citations since it’s not unheard of that people cite a paper based on seeing it in some other paper, rather than reading the paper directly. I don’t think people should do that, but they do.

nelgaard• 3 years ago

Of course the citations should be lost (not counted).

But the paragraph is about collaboraters being angry and frustrated. And they seem to be angry at those who faked the data. Which is understandable.

You would hope that this would also mean that collaborators getting their name on a paper would put a little more effort into scutinizing the data provided by co-authors.

Maybe it is just a little to easy to work on paper and just accept the underlying data without asking any questions.

bakuninsbart• 3 years ago

That narrative is insane. It is one thing to produce shoddy research. Even good senior researchers sometimes just have a paper that... doesn't work out well, with a late found flaw in the methods that can't be fully mitigated or uninteresting findings, and then they'd rather still try to publish it than shelve months of work. That's not good, but almost unavoidable in the current academic environment.

The accusations here are very different: Producing fake data is fraudulent, of course the papers need to be retracted, and people involved and in the know need to be punished. The quote from the article describes well something that seems to be a growing group of people within academia:

It's possible "these people don't care and never cared about the science," Michael Sanders, a professor of public policy at King's College London, said. "They talk about science, and they talk about experiments, and they sort of wrap themselves in the fabric of the scientific method — just as a way of selling stuff."

gcoakes• 3 years ago

Is there a market for publishing the retrospective of the process which resulted in a "late found flaw"? I would expect uninteresting data would still be published if only to prevent someone from going down that rabbit hole in the future.

light_hue_15• 3 years ago

No one has ever been unfairly accused.

The bar for being accused is so incredibly high. People are given so many possible outs and are allowed to produce the most implausible explanations. And then, even having one or two faked papers basically has no impact.

Here's an example.

Ken Rogoff and Carmen Reinhart. Professors at Harvard. Despite having faked the paper that introduced austerity to the world, had an immense impact on the economy, and caused massive and needless suffering for hundreds of millions of people. That paper still continues to haunt countries today. To make their paper work they had to lie 3 times: to exclude some countries, exclude data from others selectively, and reweight all of the data. None of this is reported in their paper! Each of these took deliberate actions on their part, they cannot be mistakes. Every single "mistake" they made is carefully calculated to produce the result they want. Every one of them is needed otherwise their paper would be junk.

What happened to them? Well, Reinhart has been promoted to the Chief Economist of the World Bank and is still a professor at Harvard. Rogoff is still a professor at Harvard too. Their faked paper literally resulted in deaths, toppled governments, and the rise of the alt right in places like Greece.

But they got a promotion for it.

The only people who are afraid are the people who are faking their data. No real scientist is getting accused of anything here.

lukeasrodgers• 3 years ago

What do you make of Reinhart and Rogoff’s rebuttal to this criticism? They acknowledge the excel error but dispute the rest. https://www.nytimes.com/2013/04/26/opinion/reinhart-and-rogo...

stefan_• 3 years ago

Even if you are caught red-handed in fraud it's not the end. Dan Ariely, mentioned in the article, continues to be a professor at Duke.

diogocp2• 3 years ago

the paper that introduced austerity to the world

This is complete nonsense. The paper was published in 2010.

The principle of fiscal austerity for EU member states was enshrined in the Maastricht Treaty of 1992 (Article 104) and the accompanying Protocol No 20 on the excessive deficit procedure. [1]

The reference values referred to in Article 104(2) of this Treaty are:
— 3 % for the ratio of the planned or actual government deficit to gross domestic product at market prices;
— 60 % for the ratio of government debt to gross domestic product at market prices.

In fact R&R cite the Maastricht Treaty as the source of the 60% magic number in their paper!

These numbers were basically pulled out of thin air by a French civil servant in 1981. [2]

That is how government policy is actually decided. Not based on an academic paper published 20–30 years after the fact.

[1] https://eur-lex.europa.eu/eli/treaty/tec_1997/oj

[2] https://openeuropeblog.blogspot.com/2014/01/from-archives-st...

js8• 3 years ago

Despite having faked the paper that introduced austerity to the world, had an immense impact on the economy, and caused massive and needless suffering for hundreds of millions of people. [..] But they got a promotion for it.

There is a book from Clara Mattei, The Capital Order (https://press.uchicago.edu/ucp/books/book/chicago/C/bo181707...), which explains they were not the first, and probably are not the last.

amelius• 3 years ago

This reads like an accusation in disguise ...

huitzitziltzin2• 3 years ago

Completely and totally different case. You are wrong on the facts.

There is no one in the economics profession who thinks Reinhardt or Rogoff deliberately made that error bc they had something personal at stake in convincing policymakers that austerity policies were the right outcome. Quite unlike the cases of ariely or gino or various others.

They made an embarrassing error but ultimately it was a coding error.

Workaccount2• 3 years ago

I wonder if it's like a steroid doping situation:

What kinds of steroids and how much of them can I get away with using so I'm in the perfect spot where I am ahead my fellow doping athletes, but still under the bar of detection...

s1artibartfast3• 3 years ago

There's a pretty clear solution for that in athletics which is sometimes implemented. They take blood samples and store them for decades. New methods can come along and analyze the data and detect fraud long after the fact.

The same can be done in science, it just more disclosure of data.

next_xibalba• 3 years ago

it's like blaming the Police for stopping a criminal because they took away the criminal's source of income

Coming soon to a San Francisco near you!

peterfirefly3• 3 years ago

It's obviously already there in parts of California/Oregon.

burkaman• 3 years ago

Everyone, please read the whole article, not just this comment. The second two quotes here appear very out of context in this comment - they are about anger and frustration towards Gino, not towards Data Colada.

The first quote here is not a defense of Gino, it's a nuanced opinion of self-proclaimed vigilantes who are also human and might not always make perfect decisions. It is followed by this:

Discussing the allegations that Gino falsified data is seen almost unanimously as fair game, especially in light of Harvard putting her on leave. People expressed more mixed feelings toward disgraced researchers such as Cuddy, who failed to replicate her findings but did not fake her data.

I have no special insight here, but this at least seems like a reasonable thing to discuss.

It feels, to me, that they're making Data Colada (the data vigilante group) the bad guys here.

I strongly disagree. The light questioning of Data Colada was a very small part of the article, and even that section did not give me the impression that they are "bad guys".

jonhohle2• 3 years ago

If the findings can’t be replicated, were they worth publishing to begin with? I know there is pressure to publish, but the effect on society is significantly greater than a single researcher’s personal job security. If nothing else, papers with expensive or one-off experiments should be clearly marked as observations that should not be relied on as foundational for other work until replicated. When they’re cited peer reviewers could make sure they are referenced as such without requiring the peer reviewer to intimately know the details of the reference.

lo_zamoyski2• 3 years ago

Guilt is a very effective way of keeping corrupt status quos going. Even if you change your ways, there is still dirt on you. The only way to break the cycle is if someone from the outside exposes the filth, or someone guilty on the inside decides to pick up his cross and accept the consequences of admission and exposure. On top of that, there are many who won't even entertain the possibility that the accused could be guilty even with evidence, because they have an emotional interest in preserving a false image of decency. Or they're terrified of the consequences of reporting the problem.

This is what happens with sex abuse. When there's enough guilt to go around in the right places, the guilty form a kind of hellish camaraderie, a Mexican standoff of pointing fingers. Once such people assume positions of power, they also have a tendency to concentrate corruption in the institution by favoring the corrupt. Doing so entrenches the standoff and eliminates those who would oppose or expose them. The so-called "lavender mafia" is a well-known example, because the media hate nothing more than the RCC, but this occurs everywhere and at greater rates.

What's important is what we distinguish between institutions and the people that temporarily occupy them. Good institutions should generally be preserved. It's the corruption that should be dealt with. Of course, that's not always an option. Sometimes, corruption destroys the institution, or perhaps the institution is beyond repair. But generally, there is a presumption in favor of cleaning house rather than abolition.

DonsDiscountGas2• 3 years ago

Personally I'm less worried about outright-fake data than sloppiness. The article mentions Brian Wanskink; AFAIK he never deliberately invented anything but he did a whole bunch of p-hacking and his lab was so sloppy that data got mislabeled (one study allegedly done on 8-11 year olds was actually done on pre-schoolers). He was "caught" when he published a blog post[2] giving advice to young scientists and it went viral. Clearly no misconduct intended.

Most of this garbage research takes place in domains that don't matter, where people are hardly taking the results seriously anyway (see also power posing). Typically when somebody actually cares about the truth of a result, they kick the tires and vet the result pretty thoroughly. The fraudulent LeCour and Green study from a few years ago was exposed by Brookman and Kalla, who were attempting a related study (rather than being science "vigilantes").

But not always. Clearly people acting on Gino's bogus research. The Reinhart-Rogoff paper [0] was discussed globally, and may have actually influenced fiscal policy. They used Excel for analysis, made a click-and-drag mistake, and improperly excluded a couple datapoints. It appears this exclusion was accidental [1]. Nevertheless, including those points changes the conclusion.

Catching this error took 3 years. It probably would've been caught faster if they had published their data alongside the paper, although apparently they actually did provide it upon request, so if people checked these things more frequently it would've been caught earlier.

[0] https://en.wikipedia.org/wiki/Growth_in_a_Time_of_Debt

[1] "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis.5 The omitted countries are selected alphabetically and, hence, likely randomly with respect to economic relationships." http://peri.umass.edu/fileadmin/pdf/working_papers/working_p...

[2] https://web.archive.org/web/20170312041524/http:/www.brianwa...

juujian• 3 years ago

One of the problems is that if you run more studies, you are statistically more likely to eventually yield a statistically significant result. The tests that folks run are not designed for series of statistical tests, but for individual tests. So when you run a ton of experiments and throw away all the statistically insignificant results, you are engaging in the form of p-hacking that the discipline of psychology is currently struggling with. The overarching replication crisis.

And you may be thinking that's all harmless, but it's really not. Folks running experiments are competing for jobs with other folks who might do more worthwhile research. Because psychologists are good at running the pipeline, and they have found a nearly inexhaustible source of publications (experiments run on undergrad students), the up the bar for everyone else.

I remember just a few days ago a discussion on this site about how a hiring committee skipped over the "better" applicant with more publications and citations than the successful candidate. The reality is that there are plenty of faculty on hiring committees out there who are looking for candidates with a good publication record, because that is how they assess if someone is a good researcher. Psychologists have already succeeded in pushing out most of the less experiment-prone folks in their discipline. The pressure extends to adjacent fields.

DonsDiscountGas2• 3 years ago

100%. Theoretically one can account testing multiple hypotheses, but shockingly nobody wants to do that.

DrNosferatu• 3 years ago

She just faked it until she made it...

michaelhoney• 3 years ago

The Very Bad Wizards podcast talked a little about this case in a recent episode (https://verybadwizards.com/episode/episode-263-free-yoel - main topic is also interesting, about a professor not getting hired for what seem like not-great reasons). The podcast notes include a link to the Data Colada post about the Gino fraud: http://datacolada.org/109

LatteLazy• 3 years ago

It amazes me that in modern times people are not required to publish their raw data. Doing so would (a) prevent errors in stats going unnoticed (b) remove the incentive to use statistics creatively (c) make it much easier and faster to find errors (d) make fraud detection easier.

Increasingly I think we should be offering either direct funding or bounties to groups that DO actually check this stuff, attempt replication etc. The days of "a gentleman's word is his bond" are long gone (if they ever existed).

karencarits• 3 years ago

This becomes quite complicated if you do research on actual humans as privacy regualtions - especially for health related data - can be quite strict

macinjosh2• 3 years ago

Why aren't these scientists jailed for fraud? These people are low-life criminals and should face consequences for fraud, wasting grant money and peoples time. But they aren't poors so they get special treatment.

AnimalMuppet2• 3 years ago

They didn't steal money, so they don't face jail. They stole reputation, which is not a crime that the laws deal with.

macinjosh2• 3 years ago

They stole the money used to fund their fake study and the studies of people who cited them in their own studies. That is a big deal.

Knee_Pain• 3 years ago

Everyone in academia knows that academia is shit.

People are warned well in advance by peers, professors and everywhere online, and yet they still do it.

Then, once the sunk cost creeps in, you can't help but partake into the rotten system and wish to keep it as such.

Just leave and do private research, it's not worth it

sn41• 3 years ago

Even if you do private research, you still need your work to get peer-reviewed. The pressure comes because we need to make the results sound more dramatic, rather than "meh. some effect is visible, but nothing to lose sleep over" - that will not get you the awards, the grants, and the Nature publications. Almost all these carry over (more so) if you are doing private research, since in addition to publications, funding is a more pressing need.

Is corporate life any better? I've tried both. And in terms of autonomy, it is a huge relief not to be bound to quarterly evaluations and KPAs and trying to belittle others' work just so that you don't get axed and so on.

JoeAltmaier• 3 years ago

Everybody putting down academia as shit. But somehow we still have GPS satellites, fibre optics and AI machinery that can nearly think.

I'd guess the shittiness may be isolated to a few disciplines, but that's just a guess based on data.

mistermann• 3 years ago

What data type does isShit() return? A boolean? Is that adequate to accurately represent the inherent complexity of reality?

jahsome• 3 years ago

It's rather funny you seem to make the case that you can't generalize an entire population based on the actions of individuals, then go on to... generalize entire populations based on the actions of individuals.

kipchak• 3 years ago

There's also gov and private research, and the things in between. GPS for example is DoD, the gov gov has funded lots of companies that make new things, like In-Q-Tel funding Keyhole which made the precursor to Google Earth[1], or Oracle which gets it's name from the program it was developed for.[2] This is not to disparage academia and fundamental research, but it's not the only possible way to make new stuff.

[1]https://en.wikipedia.org/wiki/In-Q-Tel

[2]https://gizmodo.com/larry-ellisons-oracle-started-as-a-cia-p...

prepend4• 3 years ago

I’m a big fan of open data but struggle to articulate its value in a quantified way other than just my gut feeling of more transparency leads to more collaboration leads to more research leads to more knowledge.

But I think examples like this mean that we’re getting closer to if a study doesn’t release all its data- raw, cleaned, and analytical tables- then it will be discounted or ignored. And there’s fame and money to be made in finding data irregularities.

We still need some way to audit and confirm data regularities, but that is boring and I think no money there. I’m not sure if there will be some chore assignment for students to pick a dataset, analyze it for rigor, and post results somewhere so every paper has a link to people who checked data and found it appropriate. I hope this means 99.9% of studies have proper data and not just when we hear about the (hopefully) rare studies that falsify or just don’t work with properly and make inaccurate conclusions.

arsome• 3 years ago

Honestly, most of these studies are getting caught simply by not having the expected distribution on their data. I suspect improving this analysis will more often just lead to better fakes.

Unfortunately I think this means we need to disregard these popsci papers, especially in the social sciences unless there's a clear, uninterested party providing the data as well.

somedude8952• 3 years ago

Sociological studies should be disregarded and remain uncited until they're reproduced by an unaffiliated team that isn't ideologically aligned with the original researchers. Which isn't gonna happen for most studies since it's prohibitively expensive and most sociologists share the same biases.

hilbert424• 3 years ago

"We still need some way to audit and confirm data regularities,"

See my post that mentions Benford's Law, it's damn good a sussing out fraudulent data. Moreover, it's quite old and has stood the test of time.

hgsgm• 3 years ago

Why is is this old story getting frontaged so many times in one month?

https://hn.algolia.com/?dateRange=pastMonth&page=0&prefix=tr...

0xDEF• 3 years ago

Libertarian techbros are on a mission to make us doubt (climate) science.

fortran77• 3 years ago

Maybe our “top schools” really aren’t.

profsummergig• 3 years ago

Might be worth mentioning that while not considered "fake" data per se, Amy Cuddy's famous Power Poses research (one of the most popular TED talks) was debunked and she eventually left Harvard.

hilbert424• 3 years ago

I simply don't get this. I say the same thing every time I hear of research/scientific fraud.

I can understand why researchers want to commit fraud but why do they actually do it? Is it because they actually believe they can get away with it indefinitely, or are they gambling they can do so long enough to meet their goals—fool everyone long enough to get through their careers after which they couldn't care less about their reputations?

I find it hard to believe these smart people actually believe they can get away with it. Surely they don't and they must know their reputations will be ruined when they're found out.

As sure as day follows night, they'll eventually get caught out by others who are researching their papers and comparing their data with other researchers whether it's within weeks of publication or decades later. And they must know that AI will expose them as it can automate comparisons with similar research and their data will be found to be anomalous.

klntsky• 3 years ago

You are assuming that the rest of the researches are interested in exposing them. They may be in the same boat (institutions), paid by the same people to produce similar "research".

bwing• 3 years ago

A lot of factors:

1) People definitely did get away with this, all the time. Historically, researchers had no obligation (at least that was practically enforced) to maintain and share their data and code. Peer review would check for specific methodological flaws and nothing deeper. If someone emailed you about your 1995 study in 2005, you'd say "I no longer have the code," or, more likely, simply never email them back.

2) In a highly competitive landscape where cheating is effective, the "winners" will be heavily selected for willingness to maximize the use of cheating. Even if only 1% of the population is willing to commit explicit fraud, that 1% is going to be heavily overrepresented in a world where explicit fraud gets you the top-tier publications that bring you to prominence.

3) Ariely and Gino both made millions of dollars from their fraud that they will not have to give back. It's worth emphasizing how poorly Ariely's fraud was executed -- he did the laziest possible fraud and easily converted it into money and prestige.

4) Related to the first point, it's hard to overstate how much the culture has shifted since the '08-'12 time period. The replication crisis was just kicking into gear, and only among people who were paying attention to that type of thing. Ariely didn't come up reading Andrew Gelman's blog. There's simply far more light on any paper today than there would've been 10 years ago -- statistical and methodological understanding have come a long way as cohorts of academics came up in the shadow of the replication crisis. Having established credible groups like Data Colada to centralize these analyses has also been a big deal; tenured profs can't bully Data Colada by threatening their career progression the same way they could if accused via email by a random grad student.

hilbert424• 3 years ago

I agree with most of your comments but perhaps with a different emphasis, and you're right, some people do get away with falsifying data but the risks are very high when that data actually matters and is subject to scrutiny.

I amplify my comments below in my reply to droopyEyelids.

droopyEyelids• 3 years ago

I think the missing piece is your belief that "As sure as day follows night, they'll eventually get caught"

Academic research is sort of a "Tyranny of Structurelessness"[1] system where people can't risk alienating themselves from the underlying power structures or it'll destroy their career. On top of that, there isn't a good mechanism to report fraudulent research. So when someone finds a paper that can't be replicated, the sensible thing to do is ignore it and move on. Causing a ruckus will get them labeled a troublemaker, and there isn't any guarantee it'll make a difference.

And further, there are no material incentives for exposing a bad paper. It doesn't get the researcher another citation which is their currency. It just wastes time they could have spent working towards a new paper of their own.

https://www.jofreeman.com/joreen/tyranny.htm

hilbert424• 3 years ago

I essentially agree with what you say, similarly bweing's comment above but despite that committing scientific fraud is just such a risky business. If one falsifies results and the subject matter turns out to be irrelevant in the grand scheme of things one might get away with it because it is irrelevant. — no one would care anyway. If it's high profile and contentious then one's likely to get caught out either sooner or later.

Let's take two cases. The Piltdown man fraud took place in 1912 and it wasn't until over 40 years later in the '50s that it was conclusively determined as a fraud, it then took about another 60 years to determine who the fraudster was. Right, the fraudster got away with the deception in his lifetime but he goes down in history with his reputation in tatters. Whether he considered this or not or perhaps he was hoping to be discovered and wanted to get himself in the history books when his name would otherwise be lost to history is a moot point.

Now to the infamous Jan Hendrik Schön https://en.m.wikipedia.org/wiki/Sch%C3%B6n_scandal. He was caught out quickly. My background is electronics so I understand what he did and I can say his chances of getting caught were damned high and he was caught. He must have known the risk he was taking, how could he not when the fraud was intrinsically recognizable by its nature? Then he makes his situation worse by virtue of repeating the fraud many times, so he was left without a 'plausible deniability' defense on grounds of a mistake/error/mix-up etc.

He's obviously not completely stupid or he wouldn't have been in that job. I'll rephrase that, he's not so stupid that he wouldn't recognize the risks. So why did he ruin his reputation? He deliberately shot himself in the foot without good reason—unless his reason was to become notorious.

People may think they can get away with faking data but if it's in a high profile field with others hanging off their results things will go pear-shaped very quickly. Remember, we've had excellent tools like Benford's Law for years and that's damned good at detecting forged data: https://en.m.wikipedia.org/wiki/Benford%27s_law.

Knowing what I know there's no way I'd risk my reputation faking data, I'd forever be living fear of being caught out. Moreover, I wouldn't do it for ethical reasons, as in the end I'd be not only cheating others but also myself.

rdtsc3• 3 years ago

Notice the terrible punishment that happened to Ariely: https://scholars.duke.edu/person/dan

Hint: not much happened.

Still on the faculty roster. Still "Distinguished Professor of Behavioral Economics".

So why not fake data and cheat? You might get discovered, but by then, you'd have given tons of TED talks, speaking engagements and probably made millions of dollars. You get to keep that and you get coast relax as a professor for the rest of your life.

jp57• 3 years ago

The "multi-million dollar empire" is a big part of the problem here. When scientists build a lucrative business that is based on the truth of some hypothesis they've been studying it immediately creates a conflict of interest. How can they ever be trusted to report results that would falsify the hypothesis?

ColoursofOSINT• 3 years ago

For those of you interested in the accusations, here are the primary sources for them[1]. The are multiple (it is a 4-part series) and it looks quite damning.

[1] [[109] Data Falsificada (Part 1): "Clusterfake"](https://datacolada.org/109)

[2] [[110] Data Falsificada (Part 2): "My Class Year Is Harvard"](https://datacolada.org/110)

[3] [[111] Data Falsificada (Part 3): "The Cheaters Are Out of Order"](https://datacolada.org/111)

[4] [[112] Data Falsificada (Part 4): "Forgetting The Words"](https://datacolada.org/112)

efitz• 3 years ago

I find it ironic that we have a reproducibility problem in many scientific disciplines, that we find that many notable scientists have been less than 100% honest in their research, that peer review fails to catch many of these errors, and still there are many people running around saying stupid bumper sticker slogans like “trust the science” and “I believe in science” and “95% of scientists agree…”

What we’ve learned is that scientists are human and they have all the human weaknesses and venality as any other group. The practice of science in the modern day (publish or perish, beg for grants, non-blind peer review, financial conflicts of interest, politically charged university atmosphere) is rife with incentives to predetermine desired outcomes rather than let the chips fall as they may. Individual scientists of course will fall all across the distribution of integrity.

I think the scientific method is one of, if not the greatest gifts the minds of the Renaissance gave us. BUT I refuse to be browbeaten by priests in lab coats or their acolytes. If you cite scientific papers in front of me trying to make a political case for a course of action, I will use my own judgment, informed by he paper as well as other factors like motivations and so forth.

nerdponx• 3 years ago

It's not ironic. It's a reflection of the fact that "scientists" are not an homogeneous group, and we are currently living through a slow-moving revolution in which some scientists are, broadly, taking a stand against fraud and attempting to dismantle the power/incentive structures that make it possible to perpetuate. The system is imperfect, and the work is in progress.

"Trust the science" is a poorly-phrased attempt to counter the persistent conspiracy theories that various science disciplines, and the scientists that work therein, are part of this or that conspiracy to reduce your freedoms, mislead the public, etc. Or the related conspiracy theory that science broadly is bullshit and that the primary motivation of any scientific study is the continued justification of funding, rather than pursuit of knowledge.

The fact that scientists can be wrong, some studies are faked or their results are overstated, etc. has little or nothing to do with it.

I refuse to be browbeaten by priests in lab coats or their acolytes. If you cite scientific papers in front of me trying to make a political case for a course of action, I will use my own judgment, informed by he paper as well as other factors like motivations and so forth.

And this is exactly the kind of attitude that I described above. It's the ACAB approach applied to scientists, where it makes no sense if you look at what scientists actually do and care about. A huge number of scientists are persistently and actively fighting against the "priest in lab coat" archetype.

trompetenaccoun• 3 years ago

Do you work in science? Because since I have, I agree with some of the "conspiracy theories". I didn't before but as someone who'd always been fascinated with knowledge, research and human progress from a young age, I was thoroughly disappointed when I discovered the reality of it isn't as I had imagined in my almost religious belief in science. The reality was I found people who worked in it like a regular job. They were not driven by by a desire to further truth and expand human knowledge, but rather just like most other people they care about their salaries, funding, promotions, intrigues/political games, etc.

The mistake is thinking there must be a conspiracy to make it true. In reality it's just human nature and group behavior. One might think some of the most critical thinkers work in science, and it can be true in some cases, but more often than not those are actually the ones that get filtered out. Ironically, you must not question too much if you're a researcher who wants to receive funding. The safest thing to do is research one of the latest hypes and have findings similar to what other groups find. Don't go against the current, that gets you in trouble.

isaacremuant2• 3 years ago

"Trust the science" is a poorly-phrased attempt to counter the persistent conspiracy theories that various science disciplines, and the scientists that work therein, are part of this or that conspiracy to reduce your freedoms, mislead the public, etc. Or the related conspiracy theory that science broadly is bullshit and that the primary motivation of any scientific study is the continued justification of funding, rather than pursuit of knowledge

Actually no. It's a purely anti science/authoritarian approach to avoid any dissent when politically convenient.

Covid policies were the perfect example and labeling/censoring instead of debating (even other scientists like those on the great Barrington declaration) shows that it's not really about science but about control.

Corruption is not a grand conspiracy but a natural thing that occurs in bureocratic institutions and, nonetheless, accusing and exposing corruption and inconsistency was a big no in anything related covid (lockdowns, mandates, censorship, etc). Freedoms WERE limited unconstitutionally and selectively but people would be telling you "my fredumbs, grandmakiller, conspiracy theorist" because they couldn't really argue except via way of appeals to authority and slogans such as trust the science.

The science wasn't one and wasn't uniform, but you couldn't even compare "sciences". You only had to take it.

Those of us who engage in science are not afraid to expose our results and hypothesis and don't need censorship.

jonhohle2• 3 years ago

9x% of scientists agree has always struck me as profoundly stupid. Are these scientists in the same field? Has there been a Galilean discovery that is not widely accepted yet? Who surveyed a statistically significant portion of “scientists”? What credentials does one need to be considered in that population? Does a master of computer science have authority on biology, physics, chemistry, ecology, geology, or any other number of fields?

mcpackieh3• 3 years ago

I've found that a good rule of thumb is to ignore anybody who is self-describing as "a scientist". If they self-describe as some specific kind of scientist then I'm more than willing to listen. If somebody says "as a biologist" or "as a geologist" or "as a whateverologist", that's fine. If they identify themselves in an even narrower way, that's even better. But if somebody says "as a scientist", just generic 'scientist', then they're trying to imply a very broad sort of expertise that almost certainly is not justified. They're basically saying "as an expert in everything..." Such people are usually safe to ignore.

isaacremuant2• 3 years ago

and still there are many people running around saying stupid bumper sticker slogans like “trust the science” and “I believe in science” and “95% of scientists agree

That has all to do with authoritarianism and how in partisan politics and during the covid draconian policies push (in many different countries) any dissent or non agreement was treated as a terrible sin that HAD to be punished.

Even nowadays, in HN, you'll get people repeating the mantra, downvoting and acting super obfuscated that someone dare point out that just because govs/tech/pharma/media sayd so, doesn't mean it's true, specially when we there's money and politics involved and that security or hygiene theater is nothing new.

mcpackieh3• 3 years ago

Reminder that "peer review" is part of the scientific journal system, not part of the scientific process itself. Journals have it done to serve their own interests, to save on paper/printing expenses and maintain their prestige. Peer reviewer isn't replication (which is actually important to the scientific process.) Google n-grams for "peer review" and you'll see that nobody was talking about it before the 1960s.

linuxftw2• 3 years ago

Peer review can't catch lies. It reviews your methodology and whether your findings are supported by your own data. If you outright falsify your data, peer review won't catch that. Replication catches lies.

Many studies, even those relied on for medical products, don't get replicated by independent entities.

It's safe to assume, any non-replicated study is an outright fabrication at this point.

Georgelemental• 3 years ago

These frauds weren't caught by lack of replication, they were caught because statistical analysis of the reported data showed it to be implausible.

rcpt• 3 years ago

Yes, despite difficulty reproducing results in some fields, it still generally makes sense to trust scientific results. The scientific method is the best thing humanity has for establishing knowledge from empirical data.

Now if you'd prefer to put your faith in religion, or some demagogue, that's fine. And you're free to go eat all the horse paste you want. But overall dismissal of science as "stupid" is just a flat out ridiculous take.

AlbertCory3• 3 years ago

"horse paste" -- too bad. You were doing so well up to there.

TheRealPomax• 3 years ago

One paper is no paper. Until it's been replicated, your study hasn't concluded yet.

lo_zamoyski2• 3 years ago

Those who say "trust the science" in the typical way it seems to be meant are rarely people who understand what science is. They're merely virtue signalling fan boys and groupies trying to demonstrate their loyalty to the regime. Because that's why they say it. If the regime said "don't trust the science", they'd say that, but scientism happens to be the state religion.

beepbooptheory2• 3 years ago

I firmly believe that guys like Dawkins/Harris/Tyson have done substantial harm to a certain kind of person's perception of the world. One can quite easily see the worldview they espouse, in the broadest sense, is compatible with the worst of human tendencies and ideologies. I find people can gain utterly monstrous views about fellow human beings through the superiority they earn for simply "believing in Science."

vkou3• 3 years ago

One can quite easily see the worldview they espouse, in the broadest sense, is compatible with the worst of human tendencies and ideologies.

Could you name one or more major orthodox worldviews or institutions that are not, in practice, compatible with amplifying and perpetuating the worst of human tendencies and ideologies?

vkou3• 3 years ago

There's a lot of wrong with your argument, but let's go down the list.

1. The optimal rate of fraud is not 0%, just like the optimal rate of shoplifting in a store is not 0%, and the optimal amount of crime in a society is not zero. Science, just like the grocery, and the street, has to operate on some degree of trust, and not non-stop big-brother gestapo police state surveillance. Just like a zero-crime society, a zero-scientific fraud society would have a lot of negative externalities that aren't accounted for by 'bring fraud down to zero' paperclip-maximizaiton. That's why it makes sense to talk about what can be done to reduce fraud, but makes no sense to throw yourself off a roof when an instance of it actually happens. It's a statistical inevitability. Look at the blast radius, pick up the pieces, have a conversation about what, if anything, could be done better next time.

2. If even one bit of dishonesty is enough to distrust a system, you shouldn't trust anything. But nobody actually thinks or operates in that kind of insane solipsism. You wouldn't get anything done.

3. What actually happens is that people from low-integrity fields (Politics, sponsored media, lobbying, organizational PR, internal corporate politics) are trying their fucking hardest to use that argument to bring public perception of everyone else down to their level. If they are blatantly lying half the time, and someone working in science blatantly lies, say, 0.5% of the time, and is blatantly wrong 5% of the time, and is mostly-right, but subtly wrong[0] the rest of it, that solipsist from #2 would look at it and conclude 'well, there's no difference'. He couldn't be more wrong, and I would like to gamble on loaded-dice games with anyone who disagrees with me on this point.

4. You shouldn't use a single paper, or two papers, or even fifteen papers as an arbiter of expert consensus. Papers disagree. Expert consensus is reached by looking at the whole body of a field. This requires extensive understanding of a field, which you're not going to get from clicking a few links over an afternoon.

5. You shouldn't use expert consensus as a definitive arbiter of truth.

6. But you sure as hell should defer to an expert, over your own judgement, about non-trivial technical matters. I don't care how smart you are and how many papers you read last week[1], I have serious doubts that you're going to be any good at performing, say, an open-heart surgery. There's a reason we leave it to the experts, even if what they know is wrong. [2] [3]

[0] Almost all of our understanding of the world is at least subtly wrong. That's fine, though, many bits of that understanding are, while subtly wrong, useful.

[1] It's a different story if you read ten years worth of papers. And not just the ones that support your biases. And also spent five to ten years working with, and for experts in the field, making a lot of mistakes along the way. You don't get mastery of a subject by reading, you do it by doing, making mistakes, and correcting them.

[2] What experts know is almost certainly, objectively wrong. But it's often useful, and better than blind stumbling.

[3] If we treated the low-integrity fields with an iota of the scrutiny you want to apply to the lab-coated tech-priests, half the world wouldn't have a job.

AlbertCory3• 3 years ago

Well stated. Inevitably this discussion reverts to a practical one: what happened with COVID? A few powerful people used "trust the science" to elevate their own conclusions (which, to use your #3 and #5, are subtly wrong some of the time) to forcibly silence discussion. We have ample proof that top health officials and top political officials did exactly that.

People who spend an afternoon clicking links, to use your language, are entitled to voice their opinions, however ignorant they are. Just like you're entitled to say that the Earth is 6,000 years old. Most people will ignore it, but those who are receptive to those beliefs already will listen.

Again, using your language, "non-stop big-brother gestapo police state surveillance" of wrong opinions does have "a lot of negative externalities."

manvillej• 3 years ago

I agree with you that papers shouldn't be treated like some holy text. That would go against the core of scientific method. We should inspect the core research.

That said, there are many who would use the fact that "humans are flawed" to throw out scientific research and distrust it. Scientific research will still be more reliable and dependable than the spew of others because there is rigor to it.

but the fact that this research is public and available is why frauds like Gino can be caught. This is the scientific method working. Media tends to amplify the hell out of the fantastical, fraudulent or not. But this IS the system working.

We may step in the wrong direction here and there, but we will inevitably push towards truth.

ny711• 3 years ago

Lock em' up

jfghi• 3 years ago

I think it’d be quite nice to celebrate process more than results. Investigating a topic and demonstrating lacks of correlation or causation is remarkably useful for future investigators. The opposite can be said if a result falsely gets promoted which causes others to waste time in attempts of reproduction or continuing a mostly failed path of analysis.

wudangmonk• 3 years ago

In the 6th grade I had a science project due, I of course had done nothing until the day before so I made up weeks of data in the form of journal entries from which bar and line charts were created since I was good with computers back when having pictures off the internet was considered fancy. My aim was just a passing grade and not being found out.

Lo an behold do I not only get put into the school's science fair, I actually end up winning second place. This is actually how I became the science kid from then on and to live up to this name I went and invested more time into both math and science classes from then on. Eventually I went from an okay student to a top student and becoming a real scientist was something I always wished I had done.

Moral of the story?, there isn't one... I just never realized just how much 'real' science I was doing back then.

say_it_as_it_is• 3 years ago

Can everyone stop focusing on accused persons? The court of public opinion should not rely on accusations. Donald Trump made a lot of accusations, and did you believe all of them upfront?

Harvard professor Francesca Gino was accused of faking data

Related Stories