Fabricated data in research about honesty

https://www.npr.org/2023/07/27/1190568472/dan-ariely-francesca-gino-harvard-dishonesty-fabricated-data

by danso • 3 years ago

107 46 3 years ago

jvanderbot• 3 years ago

This is the latest in my most frequently learned lesson:

Nobody thinks they are doing anything wrong. They think they are shoring up a weakness in a fundamentally good argument or cause.

AnimalMuppet• 3 years ago

Ouch. That's not just scientific dishonesty. It's also politics, and the legal system, and advertising, and internet conversations, and...

munchler• 3 years ago

Yep. There are very few true villains in reality. Just a lot of people with different values and priorities. Even Vlad Putin probably thinks he’s on the side of the angels.

xdennis• 3 years ago

Please don't call that guy Vlad. Vlad is short for Vladislav. You're looking vor Vova.

It's like saying Ben is short for Bennett.

godelski2• 3 years ago

While I generally go with this, it definitely isn't an absolutely true statement and we should even modify it. People know they are doing wrong, they just justify it with reasons why it is okay to do the wrong thing. I'm pretty sure that these people knew they were doing a big wrong, but I'm sure they justified it with "well everyone fudges the data a little." (this wasn't little)

bena• 3 years ago

I'm going to go with both of you.

They know what they're doing is wrong. But I don't think they did it because "everyone does it", they did it because they think they're ultimately right.

They believe in their results, they know in their heart of hearts that their conclusions are true. It's just that not all of the data lines up. It's incomplete, inconclusive, etc. Yes, it is wrong to fabricate data, however, over time, results will bear out the truth of their statements and it won't matter.

They are of the camp that process doesn't matter as long as you reach the right conclusion. Which is wrong. Process is kind of the only thing that matters.

verisimi2• 3 years ago

"the greater good"

"THE GREATER GOOD"

https://www.youtube.com/watch?v=5u8vd_YNbTw

impalallama3• 3 years ago

The two worked together on a paper about how to "nudge" people to be more honest on things like forms or tax returns. Their trick: move the location where people attest that they have filled in a form honestly from the bottom of the form to the top.

Another strike against the entire concept of "nudges". Its become clear that it was a bit of a liberal pipe dream saying you could accomplish huge changes in people or even society with imperceptible "nudges" that conveniently meant you didn't need to address systemic issues and stone wall any attempts at grander change.

"Be more honest by changing a form." "Make people less fat by moving where the produce is in the store." Turns out things are more complicated.

jstx1• 3 years ago

conveniently meant you didn't need to address systemic issues and stone wall any attempts at grander change

Who is claiming this?

impalallama3• 3 years ago

https://en.wikipedia.org/wiki/Nudge_theory

gruez• 3 years ago

Its become clear that it was a bit of a liberal pipe dream saying [...] that conveniently meant you didn't need to address systemic issues and stone wall any attempts at grander change.

Really? I thought liberals were all about addressing systemic issues (eg. systemic racism), and all about "grander change" (eg. defund the police/ICE, eat the rich, etc.).

lo_zamoyski• 3 years ago

The word "liberal" is terribly overloaded to mean all sorts of things. The OP's intended meaning seems to be liberal in the sense of a lack of coercive measures, preferring incentives instead. Your meaning seems to mean "leftist".

FWIW, I use it to describe that which is Lockean in substance (which entails a kind of radical, atomistic individualism that takes the individual to be the basic unit of society, and not the family), which basically means that most of the political spectrum is implicated.

tantalor• 3 years ago

bit of a liberal pipe dream

Fail to see any connection here.

rgrieselhuber2• 3 years ago

I believe it’s a reference to these initiatives:

https://www.politico.com/agenda/story/2015/10/obamas-effort-...

sudosysgen• 3 years ago

Looking for solutions that do not involve any structural change is inherent to liberal politics in our society, since our current political economy is deeply liberal in structure (for a definition of liberal that contains both mainstream "liberals" and conservatives, closer to classical liberalism).

You can see this plainly by looking at the pattern of recuperation and dilution of any proposed structural change - making up non-structural alternatives that don't work but might and then haranguing over them for years is a common pattern in liberal democracies. See for example trying to avoid public healthcare by creating "optimized marketplaces", the entire idea of nudging in economics, the idea of self-regulation, avoiding providing free national tertiary education by instead giving progressively stringent and oddly targeted scholarships, etc...

Shugarl• 3 years ago

Make people less fat by moving where the produce is in the store

It doesn't? Not even a little bit ? Genuine question. One of the example of data exploitation I was given in university is that retail companies look for patterns in the thing their customers buy, and when they see that people who buy X-kind of thing also tend to buy Y-kind of thing, they tend to put X and Y right next to each other to push the customers to buy X and Y. Wouldn't doing the opposite work ?

impalallama3• 3 years ago

Moving single digit volume of sales and "solving" obesity are two orders of magnitude different things.

Calavar• 3 years ago

They may be referring to the research of Brian Wansink, who was exposed a few years ago for at the very least shoddy research practices and at worst active data manipulation [1, 2, 3]. A large body of his work centered around how to rearrange school lunchrooms to promote better dietary decisions. His advice quite literally includes placing healthy food first [4].

[1] https://www.wsj.com/articles/a-cornell-scientists-downfall-1...

[2] https://web.archive.org/web/20180307074049/http://www.timvan...

[3] https://www.buzzfeednews.com/article/stephaniemlee/who-reall...

[4] http://www.brianwansink.com/for-school-lunches.html

mmanfrin• 3 years ago

a liberal pipe dream

Non sequitur political flamebait in the top comment. HN continues its slide.

smcin• 3 years ago

What's a more neutral phrase for that, IYO?

shanusmagnus• 3 years ago

Good paper on this topic:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4046264

Chater, N., & Loewenstein, G. (2022). The i-frame and the s-frame: How focusing on individual-level solutions has led behavioral public policy astray. Behavioral and Brain Sciences, 1–60. https://doi.org/10.1017/S0140525X22002023

thimkerbell• 3 years ago

Are there countries where behavioral public policy has not been led astray? (I have not read the article)

sogen• 3 years ago

In my experience doing research, this question should go *at the bottom*.

Why: A basic 101 thing one learns in Market Research is that personal questions _always_ go last.

So, if you want to ask an interviewed person if their answers were honest, researchers should be placing them last, towards the end of the questionnaire.

But, this only works if the questionnaire is oriented towards asking first impersonal questions, then more personal ones.

As another commented said: This is a lot of work. Doing research, asking people questions, gathering and then analyzing data is no walk in the park. It's sad that this happened. And, most important of all: Always being honest, no matter what the results are, even if you don't like them.

jerrygenser2• 3 years ago

There are some types of nudges that have had measurable and durable societal benefits.

The primary one I'm aware of is making 401k a default, people are likely to stay enrolled and increase enrollment.

adammarples• 3 years ago

That's not even a nudge, you're not attempting to change anyone's behaviour (which is: do nothing) you're trying to take advantage of their default behaviour. Basically just putting the bin where people dump trash anyway isn't a "nudge".

boberoni• 3 years ago

The podcast “If Books Could Kill” has a great episode that breaks down and debunks the book “Nudge”. The podcast also factchecks various other pop-science and so called “airport books”.

gloryjulio• 3 years ago

I am glad that with internet I could cut out those word salad crap as much as possible, and stick with more reputable and updated sources.

Basically we have a better package management system to reduce the supply chain attack

firewolf34• 3 years ago

Now we just have to apply that concept to the watering-down which occurs when they try to take white-papers and disseminate them to the public through news articles and etc.

It is important for the public to learn - and they will need assistance to understand findings that are outside of their usual realm of knowledge/experience. Unfortunately, the resources for this are shaky, profit-motivated, and arbitrary.

verisimi2• 3 years ago

'debunks Nudge?'

Someone should tell the politicians and PR consultants...

rahimnathwani• 3 years ago

The two linked source posts are excellent:

http://datacolada.org/109

http://datacolada.org/98

This latest one has great sleuthing examples:

http://datacolada.org/112

Their analysis suggested that the data (originally collected using Qualtrics) had been modified, i.e. that the data in the data files would not match the original data (still stored on Qualtrics' servers).

It turned out they were right:

  We have received confirmation, from outside of Harvard, that Harvard's investigators did look at the original Qualtrics data file and that the data had been modified.

apienx• 3 years ago

Recommended video on the matter: https://youtu.be/d2Tm3Yx4HWI

cscheid2• 3 years ago

I’m a former academic who did a bit of work that involved statistical analysis of data in a similar way as these studies.

Fuck. These. People.

We put so much work making sure the analysis is right. Sweat the design, sweat the data collection, the consent forms, and dropped the studies where the result wasn’t there, as well as reporting null results. I specifically remember having to have a terrible conversation with a phd student that, after doing a power analysis, setting up a pilot study, and collecting enough data, we got p=0.054 on the relevant test. “Sorry, we can’t add additional participants; we can’t try a new analysis. We did the right thing and we should report a non significant result. It sucks.”

And these fuckers just go ahead and forge data to land TED talks?

Yes, let’s also have a conversation about incentives and the system. But individually, I reserve my sympathy for better people. Salt the earth they walk on, and rake them over coils. This is beyond reprehensible. Fuck. Them.

nyokodo• 3 years ago

This is beyond reprehensible.

It is easy enough to make honest mistakes and spread misinformation before it can be properly replicated that we do still require scientists to be honest. Active deception should come with a high cost. There should be a scientific equivalent of disbarment.

cscheid2• 3 years ago

Active deception should come with a high cost

Yes, I wish so too. For what's worth, I'd buy "either of Dan Ariely and Francesca Gino write a NYT bestseller book in 5 years, go on a redemption tour, and nothing new happens" for $0.80 on a prediction market.

thumbuddy• 3 years ago

These people are called psychopaths, sociopaths, and narcissists. They sit at the top of almost every single organization because our/America culture breeds them and their personality disorders are effectively a human virus to do exactly that.

These people would be doing this anywhere else regardless of the incentives. Titles and power to harm others and bolster themselves are their only motivators.

People who have done real research like you and me, we'll never sit where they sit.

naijaboiler2• 3 years ago

I think this is a misuse of p-value.

crackedbassoon• 3 years ago

What do you mean?

dang• 3 years ago

Related. Others?

Fraudulent data raise questions about superstar honesty researcher (2021) - https://news.ycombinator.com/item?id=36726485 - July 2023 (33 comments)

UCLA professor refuses to cover for Dan Ariely in issue of data provenance - https://news.ycombinator.com/item?id=36684242 - July 2023 (131 comments)

Harvard ethics professor allegedly fabricated multiple studies - https://news.ycombinator.com/item?id=36665247 - July 2023 (215 comments)

Harvard dishonesty expert accused of dishonesty - https://news.ycombinator.com/item?id=36424090 - June 2023 (201 comments)

Noted study in psychology fails to replicate, crumbles with evidence of fraud - https://news.ycombinator.com/item?id=28264097 - Aug 2021 (102 comments)

A Big Study About Honesty Turns Out to Be Based on Fake Data - https://news.ycombinator.com/item?id=28257860 - Aug 2021 (90 comments)

Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments)

whynotmaybe2• 3 years ago

Kinda ironic that my most remembered detail about Dan Ariely's book is that "everyone cheats a little".

https://www.wired.com/2009/02/ted-1/

vacuity• 3 years ago

Wouldn't that be the opposite of irony?

whynotmaybe2• 3 years ago

I guess that Alanis Morissette tainted my perception of irony.

naijaboiler2• 3 years ago

Obviously he was studying himself

Fabricated data in research about honesty

Related Stories