A Criminologist Accused of Cooking the Books

https://www.chronicle.com/interactives/20190924-Criminology?key=mi0Bff1vaLHL09_no2Emgy5Y-dAeAI88Vkl3OR4ZPajd6ssrFAxNVnafDFjqu4AWZkpfVDdwM3pGT1E3SjBzbm5pVUNTc0FPRXB6UHY4UTctY0xWOC0xUVlIWQ

by Hooke • 7 years ago

80 31 7 years ago

chronicle.com

classicsnoot3• 7 years ago

I got my major credits for both History and Political Science but was a few credits short for my core requirements so I decided to pick up a minor in Criminal Justice during my last semester. I thought PoliSci was bad in terms of opinions being treated as facts, alchemical postulations being regarded as high science, and a general culture of trendy cults around particular texts/authors being the deciding factor in who and what gets taught and believed; CJ makes PoliSci look like Biology.

Sociology in general is far closer, metaphorically speaking, to alchemy than chemistry. The problem is that Criminology is used to make very important decisions about criminal law, incarceration, sentencing, community standards. One could make a claim that PoliSci has a similar relationship with policy making, but it doesn't hold up. Political Scientists spend more time doing interviews and tweeting than they do running countries, or even experiments (case in point: my idea of using North Dakota and South Dakota as a geographical laboratory to run 10 year long, competitive governing structure experiments was perceived as a joke). Criminologists are actively involved in policing standards/regs, sentencing, etc.

With the stuff I've come into contact with, the allegations leveled in this article do not seem surprising.

misterdoubt4• 7 years ago

A minor in Criminal Justice is not necessarily a useful point of reference for the field. Most undergraduate Criminal Justice degrees are geared more pragmatically toward training police, parole officers, and such. CJ majors (and certainly minors) can graduate without ever being in shouting distance of actual criminological research.

That's not to say that the field is without problems, but what you're describing is not well representative of what we're taking about when we talk about the journal Criminology.

classicsnoot3• 7 years ago

A fair point, and something I considered before posting. There are a few factors that lead me to believe my experience is quite representative.

1) Every single teacher I have (5) are PhDs in criminology.

2) 4 out of 5 classes use their readings/assignments exclusively criminological research papers and peer reviewed journal articles. The fifth is exclusively case law and comments/treatises thereof.

3) Each of the 4 classes covers a rough average of 15 papers, the lowest being ten and the highest 20 or more.

4) The institution I am at is a leader in the field of criminology. In addition, the CJ program is paired very tightly with the law school.

5) The institution I am at is decidedly anti-LEO in both form and content. Though there are a number of matriculating cops and a smattering of future LEOs, the vast majority of undergrads in the program are sociology students aiming for lawschool, social work, or criminology.

I understand your point, and I would wager that it is generally true, but not for my case. TBH, it is the only reason I posted.

I also shared this article with every one of my professors, a few of the guest lecturers, and the dean of the school.

EDIT: clarity of point 02

spangry2• 7 years ago

It's quite shocking that the editor in chief of the 'Criminology' journal seems unconcerned about papers in his journal that rely on fabricated data. It calls into question the veracity of all papers published in his journal. Does anyone have any insight into criminology as an academic field?

misterdoubt4• 7 years ago

Yes. He hasn't been editor in chief for long; the Crim editorship rotates around through top departments every few years. I wouldn't take his position as particularly determinative.

Shawn Bushway's take is probably most representative of the field. If we're going to be taken seriously (insert Arrested Development shot) we need to be able to deal with this.

Criminology has thus far mostly avoided the replication crisis, so there hasn't been much reckoning with sloppy research -- much less with a major figure in the field just making up data. And the co-author network here touches on several of the top programs, so there's plenty of drama.

bjornsing• 7 years ago

Yea... Maybe I’m taking it a bit far here, but my spontaneous thought reading this was “How can this McDowall character call himself a scientist...? What the fuck is happening to the proud tradition of scientific research...?”

Simulacra• 7 years ago

I think this is indicative of the publish or perish mentality that universities have instilled among their researchers and professors. It creates an incentive to fake data, to fake papers, anything to publish. Those who get upset at closer inspection of their papers and their data sound more like they have something to hide, or they are afraid that the spotlight of closer inspection will turn on them. Then again this could speak to the larger issue that universities are having with aversion to critique, and disagreement.

elliekelly• 7 years ago

The author collecting the data is a pretty clear conflict of interest that I guess I’ve never given much thought. Considering how difficult/expensive/time consuming it is to interview people it’s kind of odd the publications don’t require some sort of certification from whichever organization collected and/or funded the collection of the data.

Nasrudith2• 7 years ago

I suspect it hasn't been hiven much thought because of how logistically difficult it would be communications wise and the value of context. Take even a simple yet still complex example as say Microsoft declares that coding and software design should be kept absolutely separate on a personal level. You can make a design or write of anything but not both. That would be a major pain for no concrete gain - like trying to produce bottled water from hydrogen and oxygen burning on the grounds it is "purer" that way - ignoring that apparatus also introduces comparable parts per billion contaminants as a reverse osmosis filter.

As for interviews what would another layer of gatekeeping even add that calling one or both parties to verify that the transcript wasn't from a satirical news site add?

jessaustin• 7 years ago

“I don’t even quite know what retraction is. I imagine that it could occur. I would think there would be legalistic implications.”

-- David McDowall, Editor-in-chief of Criminology

brokenkebab• 7 years ago

That's horrible.

Also, "McDowall is no fan of the move toward more scrutiny in the social sciences". Pretty damning.

LocalTrust• 7 years ago

recently saw a tweet by an academic (political) economist who proposed "you are not a real empirical social scientist unless you have published a null results paper." science cannot advance without them, but an individual cannot build career on them.

_edo2• 7 years ago

If you look at the author's published papers[0] just about every one involves highly sensitive political and social topics. That means they're likely to be quoted outside of the field where people will say things like "Look, it's scientific it was published in a peer-reviewed journal!"

Young adults have this guy as a professor and they believe that surely their professor knows what he's talking about.

This story is a few months old, the retraction request looks serious[1], and I'm left thinking that either Picket has gone off the rails or this entire field looks awful.

The paper in question has 78 citations and Stewart (the one accused of fabricating data) has 5,712 citations according to Google Scholar.

[0] http://criminology.fsu.edu/research/type/eric-stewart/ [1] - https://cj.fiu.edu/student-resources/resources-for-graduate-...

edit: this thread of some sociologists discussing this issue is interesting - https://www.socjobrumors.com/topic/co-author-requests-retrac...

spangry2• 7 years ago

Very interesting. Here is a list of the 5 articles from that discussion:

· Johnson, Brian D., Eric A. Stewart, Justin Pickett, and Marc Gertz. (2011) Ethnic threat and social control: Examining public support for judicial use of ethnicity in punishment. Criminology, 49(2), 401-441.

· Stewart, Eric A., Ramiro Martinez, Jr., Eric P. Baumer, and Marc Gertz. (2015) The social context of latino threat and punitive Latino sentiment. Social Problems, 62(1), 69-92.

· Mears, Daniel P., Eric A. Stewart, Patricia Y. Warren, Miltonette O. Craig, and Ashley N. Arnio. (2019) A legacy of lynchings: Perceived black criminal threat among whites. Law & Society Review, 53(2), 487-517.

· Stewart, Eric A., Brian D. Johnson, Patricia Y. Warren, Jordyn L. Rosario, and Cresean Hughes. (2019) The social context of criminal threat, victim race, and punitive black and latino sentiment. Social Problems, 66(2), 194-221.

· Stewart, Eric A., Daniel P. Mears, Patricia Y. Warren, Eric P. Baumer, and Ashley N. Arnio. (2018) Lynchings, racial threat, and whites’ punitive views toward blacks. Criminology, 56(3), 455-480.

All centre around racial issues in the criminal justice context.

misterdoubt4• 7 years ago

The field doesn't look great. But in most corners there is serious concern, e.g., http://grumpy.skardhamar.no/2019/09/25/the-former-flagship-j...

From that post: "I do not have any solutions to the systemic problems here, but improvements should be easy. Criminology as a field has to improve in terms of making data available with full documentation and reproducible code. That would make errors detectable sooner."

Anyone not agreeing with this needs the boot.

_edo2• 7 years ago

I think the field has a cultural problem that's causing their data problem: impact is valued over empiricism.

hdfbdtbcdg• 7 years ago

One of the features of academia is that those on the inside of the club know which papers are to good to be true - but no one says anything publicly.

justinjlynn2• 7 years ago

I am dumbfounded that respectable journals still don't regularly engage in data escrow with third party adjudication precisely for this reason. A credible question about the veracity or interpretation of the data, in terms of objectively presented statistical heuristics, should automatically grant confidential data access for further peer review. This could be accomplished easily with threshold-type cryptographic tools, even amongst hierarchies of authors. These studies are the basis of our science and are its direct credibility. It's absurd we don't take precautions against the malefactors who would pervert it.

dekhn• 7 years ago

even without any of the overly complicated tech you describe, some journals maintain statisticians on staff who evaluate everything in every paper and then help correct errors before publication. My stats prof, Stan Glantz did this for a major cardiology journal. He said (and I agree, having reviewed plenty of papers myself) that statistical errors that affect the conclusion of the paper were found in the majority of papers.

justinjlynn2• 7 years ago

overly complicated tech

It solves a problem, just like any other engineering solution. There are trade-offs involved like always. Obviously you understand the problem I'm trying to solve, since you're passing judgement on my suggestion - do you have a better one? Even given all of your verbiage, I'm afraid I couldn't devine one.

Merrill• 7 years ago

Does Wiley-Blackwell as publisher have any responsibility, or are they just a print shop?

ironSkillet• 7 years ago

The sad truth is that the majority of social sciences likely have similar problems surrounding data integrity. Good data is often expensive or hard to acquire, and once you have it, you often have to apply some sort of ad hoc interpretation in order to make it quantifiable for performing statistics. E.g. in psychology there are entire manuals for how you "encode" interactions between a therapist and a patient along various dimensions.

And then once you've got quantitative data, many social scientists don't actually apply statistics correctly, either out of ignorance or willful desperation to have some sort of positive result.

We probably need to have to have some sort of standard for what constitutes acceptable quality data, and have a government funded repository of such data to encourage reproducibility. But that may be way easier said than done.

I think I'll just stick to mathematics.

bjourne• 7 years ago

If you think the problem is limited to social sciences then you are sadly mistaken. It affects all disciplines which deal with statistical evidence, from comp sci to medicine to criminology. Not pure math though, but that field has other problems such as proofs being published that are incorrect and never gets corrected.

Nasrudith2• 7 years ago

I haven't taken any Criminonology classes so I cannot comment on the level of field rigor in even a subset but this resembles a stereotypical case of "non-Mathematics Liberal arts majors who think they were done with math".

There is a sadly significant culturally innumerate population who will look at you with scorn and anger for even daring to bring up numbers even for something straightforward like "does a year of typical catfood or small dog breed dogfood cost more" instead of trying to guess based on concepts like "beef is more expensive than fish" or "crazy cat ladies are willing to pay more". Their reaction seems more ignorance than "caught in a lie bullshit more".

It also sounds like TV news screenshots mocked online for poll percentages which overflow 100%. I have done research and replicated sources in high school. They often took numbers from multiple related sources and questions and miscombined them. "Like they had the unpaid interns who knew how to use a computer source the data" was a trope of an explanation but it may well have been the senior professionals who did so.

ineedasername2• 7 years ago

This isn't just an issue about fabricating a research paper, it also represents an enormous failure of the peer review process. 2 or 3 reviewers is common, and yet none of them pushed back on any of the issues presented here. And yet when others took a look the issues became apparent. It's hard not to come away from this with the idea that the system is broken. I also don't understand why some level of sharing anonymous data isn't a requirement for all research. If replication is a cornerstone of scientific research, the very first piece of replication should be obtaining the same results from the same set of data.

ineedasername2• 7 years ago

I really don't understand, especially given the risk of being found out, why someone would go through all of the trouble to fabricate research like this. With a little extra effort it should be possible to produce legitimate research. Heck, even if you're results are negative you can write the research and say "Results here showed no significant difference on the question of X. However there is a need for further research, and these results simply represent the first step along that path" blah blah blah. It's really not that hard!

EricE2• 7 years ago

Anyone who says anything as inane as "The science is settled" deserves the highest form of ridicule and mockery, yet as you can see in this article the wagon circling is more about politics and power than performing good science.

The more that gets revealed about the whole academic journal scene points to it being more a scam and house of cards credibility wise vs. sincerely advancing human knowledge.

And we aren't talking about "journals" like these: https://www.nytimes.com/2018/10/04/arts/academic-journals-ho...

Sigh...

A Criminologist Accused of Cooking the Books

Related Stories