Only positive reinforcement for researchers in some fields

https://statmodeling.stat.columbia.edu/2022/11/21/only-positive-reinforcement-for-researchers-in-some-fields/

by thetan • 4 years ago

77 52 4 years ago

statmodeling.stat.columbia.edu

kenjackson6• 4 years ago

There probably is some reasonable etiquette that needs to be established. I'd probably feel blindsided if someone criticized my work in public w/o first talking to me about it. The counterargument is that -- it's a public paper, we should be able to criticize it in any venue we like. But in that world it's very easy for those with the loudest microphone to effectively kill research they don't (or the research of people they don't like).

In a small enough community, it seems reasonable to go to the researcher and say, "we're going to make this statement about that research -- do you have a comment". Almost like a newspaper -- both mediums are trying to present truth, rather than create a narrative, so that's probably why it's so similar.

automatoney• 4 years ago

I definitely agree this is an etiquette rules discussion - and that maybe this change has to do with how loud the loudest microphones can get. Maybe pre-internet, the only people who would see criticism would be other experts in the field. But now maybe even things offline are more public. I've heard many academics use twitter - which means a critique in public could easily become a tweet, and then viral in the community, etc.

I do really like that newspaper comparison, especially because it could prevent misunderstanding what an author was trying to say.

periheli0n5• 4 years ago

Twitter is poison for academic debate. It is already impossible to discuss a complex matter over email—most nuance gets lost. Twitter is much worse, a) because of the character limit, b) because of the crossfire that is a wild mix of relevant and irrelevant comments, plus pure trolling.

biomcgary• 4 years ago

My biggest learning opportunities in science have come from watching two intellectual heavyweights engaged in robust debates. Unfortunately, it seems increasingly common for people to infer mental or emotional states of their fellow debater, which derails good debate.

Reasonable etiquette in science is critiquing the expressed ideas and methods rather than the person (or what has been inferred).

renewiltord2• 4 years ago

On the other hand, perhaps we could sit through half a book of refutation of refutation (cf. Conjectures and Refutations, literally the entire second half of the book), sometimes with not quite an impersonal tone.

JamesianP3• 4 years ago

The etiquette is if someone catches you in a real mistake, you should thank them (yes it can be hard, etiquette isn't always the same as reacting naturally). After all they could have ignored your existence but bothered to read your work.

If you think they are wrong you are expected and encouraged to respond publicly with why. Perhaps many people misread your work just same as that person so would benefit from the debate.

EDIT: and by the way, if you think they are wrong you should thank them for the same reason.

kenjackson6• 4 years ago

Mistakes are common in doing any non-trivial work. On HN coding is a great example. We have code reviews and it is standard practice to have issues raised there. It’s less common for someone to find and issue at CR and send it to the CEO to call an all hands to have him point out your bug to the company. In fact it would seem absurd to even raise this as a hypothetical. That is more the equivalent of what we’re discussing here.

Would you thank your colleague for raising your bug to the CEO or posting a LinkedIn article about your bug before talking to you about it?

xyzzyz2• 4 years ago

I'd probably feel blindsided if someone criticized my work in public w/o first talking to me about it.

Why, though? If you present your work in front of audience, it is extremely reasonable to expect the audience offer questions or criticism. If the mere fact of people criticizing your work after your presentation makes you feel blindsided, it’s simply a matter of adjusting expectations: you simply should expect criticism.

Then, if you expect criticism, but feel blindsided by the contents of the criticism, this means that you failed your job as a researcher. If someone can come up with a point after listening to your presentation for a few minutes, that you didn’t have thought of in advance and prepared answer for it, it means that you’re doing bad job, and should do better. You should actually thank the critic for bringing the issue to your attention, because in doing so, he helped you.

jstarfish• 4 years ago

You should actually thank the critic for bringing the issue to your attention, because in doing so, he helped you.

Heckling is not "helping." Nobody wants to be humiliated by public criticism they weren't prepared for, which may or may not itself be valid-- but a poor response from the presenter makes one look incompetent in front of their peers nonetheless.

Callouts are sadistic and unprofessional. That anyone should thank someone for punching them in the face as criticism for not seeing it coming is ludicrous. It's bully logic.

kenjackson6• 4 years ago

To be clear, I wasn’t talking about during a presentation. I was thinking after, for example on Twitter or in a magazine editorial.

The one place I’d be more OK with it is in another paper. But even then if the criticism is rather scathing, I think there is a courtesy to them reaching out and saying — “we’re planning on showing how your method was wrong and your results misleading for this reason in our forthcoming paper”.

Because once the argument goes public people tend to dig in their heels and true progress is not likely to be made.

jasonhansel3• 4 years ago

Doesn't praising people create the same issue?

It's easy for those with the loudest microphone to promote the work of the researchers they like or are connected to, or research that confirms their preexisting beliefs. This risks making the field insular by excluding new voices with new perspectives and without such preexisting interpersonal ties.

I think that you need criticism to be able to overcome this issue: to allow disinterested parties to push back on praise given by biased sources, and to ensure that current researchers aren't always elevated above newer ones.

kenjackson6• 4 years ago

It doesn’t create quite the same issue because being criticized can make the outside voices not only unheard, but people may want to actively avoid even association.

One environment discourages research in certain fields while the other encourages it in certain fields. It’s similar, but not the same. And I think the former is worse.

jmugan• 4 years ago

This seems worrisome. Sure, don't destroy a new PhD student giving their first talk in front of everyone for sport, but open and constructive criticism is the long conversation that moves science forward.

michael_j_x2• 4 years ago

This is very unlike my experience (PhD Computational optimization) where aggressive criticism was too be expected and anticipated during a presentation. I remember running "pre-mortems" before talks with my supervisor to prepare for as much criticism as we could possibly predict.

periheli0n5• 4 years ago

Engaging with critique is an invaluable skill that is relevant way beyond academia. They key is to never take it personal, otherwise one has lost the debate already.

Understanding the main angle of critique on the spot is hard enough, coming up with a rebuttal even harder, especially when the critique comes from a senior researcher.

Accepting points of critique that are valid as opportunities to improve the research, or at least as limitations to the generality of the findings reported is highly valued.

I always tended to think that a PhD is awarded for the ability to accept and counter critique towards work that one has spent years of their life with.

glial• 4 years ago

It seems like academic criticism can be binned into a few categories:

1. legit criticism, e.g. you missed something or did something incorrectly

2. reveals an opportunity to communicate better on your part

3. self-promotion by the criticizer

1 and 2 are directly valuable. Distinguishing between 1, 2, and 3 is a skill, especially since 3 is so common.

mistrial9• 4 years ago

lots of places still do hazing, even at the high levels.. honestly

michael_j_x2• 4 years ago

but I found this approach quite fitting to the environment. The math didn't care about my emotions, or how much effort I put into the work. The criticism was (most of the time) mathematically valid, and was delivered extremely efficiently in its brutality. We could argue (politely) but without worrying about hurting egos. It forced me to learn to become emotionally detached from my specific work, allowing me to accept mistakes and take corrective actions.

dekhn2• 4 years ago

There is a difference between hazing and aggressive questioning. Hazing is done entirely to maintain power structures through humiliation, while aggressive questioning is done to improve the field by reducing the impact of emotion on rationality.

yummypaint• 4 years ago

I never encountered the mentality described in the article during my physics PhD, either. Though it was stressful at the time, having committee members who were known for asking tough questions was invaluable as a motivator to learn more about underlying theory. It's of course very important that the process be conducted respectfully and constructively, and I had the fortune of being in a department with a healthy culture in that regard.

All the people I know who were not so fortunate experienced unduly harsh, personalized, and non-constructive criticism. I'm not sure whether or not that's worse than never being critical, but they are kind of equivalent in that they create barriers to students engaging with the science.

What is described in the article is not at all ubiquitous as some commenters are implying, and afaik is essentially unknown in the hard sciences and math.

periheli0n5• 4 years ago

Some journals encourage the submissions of comments to published papers. After all, a little controversy attracts traffic and gives both the critique and the critiqued paper more visibility.

But not all authors like being critiqued. This is even worse when competing interests are involved, e.g. patents, or an author is employed at a research-active company that sponsored that paper.

This is particularly bad when a paper is accompanied by press releases that dramatically oversell the findings. Unfortunately many researchers seem to read press releases and confuse their content with the actual science reported in the paper.

In that case it must be possible to critique a paper and set things right, if only to have a citable record for the next time a reviewer tries to shoot down a paper or grant based on a skewed understanding of prior research.

thetan• 4 years ago

I posted this because it's mentioned in https://statmodeling.stat.columbia.edu/2022/11/23/bigshot-ch... (https://news.ycombinator.com/item?id=33719763)

bjornsing• 4 years ago

It’s like people assume that critique cannot be helpful unless its somehow balanced with positives or provided in the context of some anonymous format or at a time when authors hace prepared themselves to hear comments and will therefore not be surprised if someone says something critical.

I think this is spot on. This is the norm, and it’s seeping in everywhere. You see it in the corporate environment too.

Perhaps it can even be generalized one more step: People increasingly expect to be able to react completely instinctively to stimuli, outside of a few clearly defined contexts such as the Q&A after a scientific talk where they will “brace themselves”.

omginternets• 4 years ago

People increasingly expect to be able to react completely instinctively to stimuli

This is a good framing, and one that fits a cluster of related behaviors and emerging cultural norms I've witnessed. One of these is hardly original: the tendency to speak about what one "feels" versus what one "thinks". We should be careful, because "I feel" is often used as a synonym of "I think", but I am increasingly under the impression that American culture has become more accepting of unarticulated, unreasoned, stream-of-consciousness thought.

But this is now ratcheting up a notch. Not only is this kind of impulsive speech acceptable in the workplace (no big deal, in and of itself), there is now an emerging social norm such that we may no longer expect more. That, I think, is detrimental.

throwawaylinux• 4 years ago

I am increasingly under the impression that American culture has become more accepting of unarticulated, unreasoned, stream-of-consciousness thought.

Politicians speaking about grave matters will spout absolute gibberish with a straight face, and nobody calls them out or asks them to explain what they mean. I think it is because the individual words and phrases used, and the intonation they are spoken with convey the right feelings and emotions.

What seems to matter is that they are feeling appropriately happy, sad, angry, about something. After that, people don't care too much or maybe don't even realize that what is being said does not actually mean anything or provide them with any real information.

It's bizarre, but it must work. Lots of politicians do it on all sides of politics, I'm not talking about the thing where they say a lot of words without answering a difficult question, and it's not just occasional mis-speaking or conversation taken out of context, because they do this in their speeches too.

periheli0n5• 4 years ago

It may seep in in the corporate environment. But I cannot see it in academia.

A relevant presentation will be followed by a critical discussion. If there is no discussion or only positive comments, the presentation was is very likely irrelevant, or shallow at best.

I would consider an academic environment without critical discussion as broken.

athrowaway3z• 4 years ago

People increasingly expect to be able to react completely instinctively to stimuli

I think this is true. If we're doing hypothesis about human behavior, you might be able to argue that our methods of communication, always forever online, have spawned this approach as an adaptation to stress of having to embody more personas for more numerous interaction. ( You can take it a step further and question the origin of 'acting' out a persona but i belief that in itself is extremely valuable )

On a further tangent, i still sometimes think about an Asian person commenting on the traits of US society to be obsessed with behaving as 'your true self' everywhere and all the time, whereas their culture it was noticeably less so.

ellen364• 4 years ago

I sometimes wonder if people have become more resistant to critique or if uninvited critique has become more common. But early twentieth century books like How to Win Friends and Influence People emphasise that people respond well to positive reinforcement, which makes me suspect that people have long given unsolicited criticism and long been annoyed by it.

JamesianP3• 4 years ago

Sure if your goal is to get something from a person then you should use another tactic. People have always had egos and taken criticism personally. Often it becomes an argument. But those arguments can still be very good for everyone else in the field to hear and for research progress.

In this article the problem was the new "weapons" people have created by which the person being criticized can automatically win that argument based on emotion basically.

throwayyy479087• 4 years ago

We have a state sponsored religion now. That the priesthood of this religion does not tolerate critique is bog-standard throughout human history. So is the hysteria in response to heresy and public apostasy (Rogan, Gabbard, etc).

Animats• 4 years ago

"HCI" as in "human-computer interface"? Or something else.

"not possible to be both an inclusive field and a field that embraces criticism."

Hm. Over time, will the field will stop producing useful new results?

A comment from an actor, years ago: "In New York, they tell you if you suck. In LA, they don't call you back if you suck. In SF, nobody tells you that you suck. So you suck forever."

bombcar• 4 years ago

Tough love is sometimes the best thing you can be given, like a five foot tall player told he might want to stop trying to play basketball professionally.

jamessb• 4 years ago

"HCI" as in "human-computer interface"? Or something else.

Human Computer Interaction: https://en.wikipedia.org/wiki/Human%E2%80%93computer_interac...

i.e., the sort of research that gets presented at CHI: https://chi2022.acm.org/

renewiltord2• 4 years ago

Doesn't this likely sound like an unintentional Goodhart's Law? Our heroes are those who have never erred, and anyone who ever errs can never be great. Given that we have created this ideal, it is unsurprising that no one ever wants to set that "have erred" bit. And given that, it is unsurprising that anyone would want someone they like or respect to have the "have erred" bit set on them. And given that, it is best to avoid any discussion of error at all.

Perhaps, and I speculate, it is our culture of dunking on targets. My personal favourite is that I disregard anything regarding Colin Powell. Someone might say "Well, he was quite the X" or "he would have made a good Y" and I cannot resist but say "But the mindless clown will just repeat nonsense if you give him nonsense". Others might decide that errors I consider insignificant in others are the equivalent of Colin Powell's.

Consequently, it is important to never state that another has erred.

periheli0n5• 4 years ago

Famous example: Ragnar Granit studied color vision in the cat for 10 years. It turned out cats are colorblind. He carried on and later was awarded the Nobel Prize for related research.

dekhn2• 4 years ago

Cats aren't really colorblind, they just have reduced color perception compared to humans, but still are sensitive to many hues. In fact that's exactly what his paper in 1943 on cat vision said: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1748-1716....

dr_dshiv• 4 years ago

“This is sometimes called “white knighting.” I discussed one such example a few years ago. The white knight in question (a senior faculty member, whom I’d never met, who taught at a different university) sent me some very angry emails.

It seemed that, from his point of view, I was breaking the rules by publicly criticizing published work. The whole thing seemed so ridiculous to me, but to him it was all about good guys and bad guys. So frustrating.”

AtlasBarfed• 4 years ago

If the paper is public, and the critique is based in data or standard procedure, then just make the critique anonymously.

... Unless of course someone is critiquing it to gain some sort of notoriety.

If it's about the fundamental science, then make the critique on fundamental grounds and do it anonymously.

goatcode• 4 years ago

Suggest changing the title to "Only positive reinforcement [is permitted,] for researchers in some fields" for clarity.

karmakurtisaani• 4 years ago

In my time in academia (math/cs), public criticism of other people's work was extremely rare. Not for the (somewhat strange) reasons listed in the article, but because it never ever reflects well on anyone involved.

I knew a researcher who publicly challenged some obviously bullshit research, and basically got absolutely nothing out of it, apart from a certain reputation. The correct thing to do is just ignore the bullshit. If you want to challenge good work with some gaps, there certainly are ways to do it without appearing critical.

jasmer• 4 years ago

Self awareness in criticism on both sides of the equation is really hard.

Some people derive ego out of criticizing, some are angry, spiteful, ideological, personal - and frankly, I think all criticism have a least small hints of all of these things.

But as someone suggested 'etiquette' with a hint of self-awareness, even as simple as 'Do I need to do this?' 'How can I make this about the work and not about either of us?' etc. might help.

Wrapped in a bit of 'tone shifting' to make it feel a bit nicer ... then debate ought to be encouraged.

On the receiving end, I really think that a bit of 'adversity training' would go a long way. I think people are incredibly resilient, and you can 'pretend expose' people to all sorts of stuff (even some personal attacks) and they will get used to it very quickly. In sports, even in the military, you see people develop certain kinds of resiliency that I think comes from doing challenging things, camaraderie, teammates chirping you when you get to big for yourself (or for any other reason). Confident people don't perceive questions as threats, even if they are reactionary in the moment.

leoh• 4 years ago

There’s a way to give criticism that is based on understanding and context, which can be kind a generous; many don’t master this approach and fall into fail modes of glib positivity or scouring negativity.

Only positive reinforcement for researchers in some fields

Related Stories