No Paper Is That Good

https://www.econlib.org/no-paper-is-that-good/

by zt • 8 years ago

162 99 8 years ago

econlib.org

B1FF_PSUVM• 8 years ago

'We are all in a mud moat, but some of us are looking at the stars', as comrade Oscar Wilde almost said, then?

User232• 8 years ago

What happens when you apply this author’s standard to this work?

tzs• 8 years ago

He's asking for two papers to demonstrate that it is worth delving into the literature of an entire field. It's not clear that it is even meaningful to ask how it applies to the work of any particular individual author within the field.

bpchaps• 8 years ago

Heh. From the article:

Does all of this hold for my papers, too? Of course. The most I can claim is that I am hyper-aware of my own epistemic frailty, and have a litany of self-imposed safeguards. But I totally understand why my critics would look at my best papers and say, “Meh, doesn’t really prove anything.”

User232• 8 years ago

I stopped reading before I got to that point thanks to the author’s argument on account of I took his advice. Thanks for the datum.

will42742• 8 years ago

Allow me to disagree. Chris Okasaki's thesis, "Purely Functional Data Structures" is That Good for the field of functional programming. There are other exemplary papers, like Godel's incompleteness theorem, Cantor's diagonal argument, or Einstein's statement of special relativity. Or Satoshi's paper on Bitcoin.

These papers are not exhaustive summaries of a field. But a reader comes away understanding the type of problems a field is devoted to solving and many of the existing ideas. And I believe that each is a paragon of their field.

xevb3k• 8 years ago

The article limits itself to discussing “empirical paper”s not theoretical papers.

In a theoretical paper, it’s possible to make statements that stand on their own merits. In empirical science, a single paper is never really enough to support an entire field. Empirical sciences generally rely on the development of scientific consensus.

antt• 8 years ago

That seems like an artificial distinction made up for economics. In physics there are plenty of theory papers that make very clear and concise predictions. If these papers hold then they are very much both. E.g. the original special relativity paper goes out of its way to make predictions and calculations at the state of the art of the time, which are quite jarring today.

will42742• 8 years ago

Hmm. I guess Mendal wrote a journal, not a paper per se. Millikan's oildrop experiment?

anonytrary• 8 years ago

If you want me to read the vast literature, cite me two papers that are exemplars of that literature. I will read them. If these two papers are full of mistakes and bad reasoning, I will feel free to skip the rest of the vast literature. Because if that’s the best you can do, I’ve seen enough.

This is one of worst ideas I've ever read. Wouldn't you just be cheating yourself?

How is tying your own learning to someone else's ability to find the best papers in any way a smart thing to do? It would be much better to doubt the person who gave you the papers (perhaps he miscalculated which two papers were the best), than to dismiss the entire field.

jldugger• 8 years ago

It's not about dismissing the field, as much as the claim that you must have read the literature to constructively participate in a debate.

skywhopper• 8 years ago

I think it’s reasonable to expect familiarity with “the literature” beyond “two exemplary papers” to be given any respect in an academic debate. If you aren’t reading the lit reviews and broadly in the topic how can you possibly contribute to the discussion productively? Almost certainly any points you want to make will have been examined and discussed already. This rule is about FOMO for wannabe rennaisssance-man know-it-alls who think their gut reactions are God’s own truth and demand a response from actual experts. If you don’t know the topic and don’t want to learn about it then please find something else to do.

mesozoic• 8 years ago

I don't know of any actual person that believes this they are always happy to jump in a debate with little to know knowledge

jonathanstrange6• 8 years ago

That's ostensibly silly, though. What if someone mentions paper X and you haven't read it? How would you know a priori that X is bad?

You always need to read all relevant literature to be able to constructively participate in a debate.

Gatsky• 8 years ago

Yeah it’s silly. If you don’t want to do any research/generate new knowledge, then sure apply this rule. But then, who cares?

amelius• 8 years ago

What the author wants to read is probably a survey ...

hessenwolf• 8 years ago

Yes - most papers just present one idea, as part of a field.

Although, the original paper of an idea, I often find, has the most human-readable explanation. Pearl's 1986 paper of Bayesian Networks, Gauss' 1800-whatever paper on the normal distribution, the original bitcoin/blockchain paper by that Japanese-sounding name, etc. They're still trying to sell the idea so they really need to talk to the idiots.

jonathanstrange6• 8 years ago

It's an anti-intellectual stance. These become more and more popular today, even among scholars. The same attitude is often behind rejecting foreign lean words or technical jargon, it's often based on the idea that dumbing things down could somehow make you smarter. It won't. You are as smart as you are, getting gradually a bit dumber but also more experienced over the years, and the rest is education. If you reject education, the only result is that you'll be uneducated.

In my opinion, a real intellectual is interested in the topic and assorted problems and will critically read whatever fuels their interest. Even papers that are not good can be interesting and worthwhile reading. Wise men know the principle "garbage in, garbage out", though, and that their time during life is limited, so they choose their readings carefully. However, that doesn't mean that they shouldn't include fairly bad literature, too. For example, it's necessary to read seminal papers that are part of a canon, no matter how stupid they seem to be.

When I started studying philosophy a long time ago, some tutor whose name I've forgotten gave us two pieces of advice that would have been very valuable to me if I had taken them seriously earlier:

1. Never dismiss a book or paper quickly because you believe you've found a mistake. Read them until the end and take the authors seriously!

2. Aim for writing 1 DIN A4 page a day, no matter what you write (could be personal, could be professional, doesn't matter). You will write much less but as a goal about 1 page a day is good.

mattmanser2• 8 years ago

I'm curious if you read the reasoning as you seem to be presenting an argument based purely on the title. Certainly the original proposal doesn't seem anti-intellectual.

For example he says "full of mistakes", not one mistake.

JackFr• 8 years ago

It's an anti-intellectual stance.

It's not - it's a realization that life is short and much research is just writing.

naasking• 8 years ago

How is tying your own learning to someone else's ability to find the best papers in any way a smart thing to do?

If they are supposed to be an expert in the field, that seems like a fair test.

nerdponx2• 8 years ago

One conclusion you could draw from this: economics might actually be a giant mud moat.

BurningFrog• 8 years ago

Economics is several separate sciences, awkwardly placed in the same department.

allenz2• 8 years ago

Most departments are heterogeneous, with mutually unintelligible subfields. CS encompasses both information theory and robotics. Cultural anthropology has little in common with medical anthropology.

traverseda3• 8 years ago

Just like cognitive science has a much higher replication rate than social psychology, you will get different results with different subfields.

barry-cotter• 8 years ago

You’d be wrong. I’m not all that impressed with macroeconomics and it’s far from a mud moat. Microeconomics is making more progress and is on much sounder footing, hardly surprising given how many more empirical observations there are.

If you’re only going to read one microeconomics textbook read Varian’s Intermediate Microeconomics if you can do calculus. If you can’t Cowen and Tabarrok’s Principles is pretty good and has a load of wonderful accompanying videos.

Comparative Advantage

https://www.mruniversity.com/courses/development-economics/c...

nerdponx2• 8 years ago

Here's Smith's definition of a mud moat:

And in the world of intellectual debate, this vast literature can function as a mud moat. That is a term I just made up, sticking with the metaphor of political arguments as medieval castles requiring a defense. A mud moat is just a big pit of mud surrounding your castle, causing an attacking army to get trapped in the mud while you pepper them with arrows.

https://leaders.economicblogs.org/noah-smith/2017/smith-lite...

I was only half-serious in my original comment. What mud moat exists today is maintained less by economists and more by politicians and businesspeople, who find certain "classical" economic conclusions convenient. "Free markets are good - just read the economics literature!"

spruciefic3992• 8 years ago

I was sort of on board with the two-paper rule until I got to the arguments about the reviews. Maybe in his mind integrating literature into a cohesive summary (ala meta-analysis) is a novel contribution, but if not, the attitude behind the two-paper rule is part of why science is in such crisis. The author is right, that the interpretation of the literature should be based on an accumulated read, and not one or two papers (unless they're reviews or meta-analyses).

hzhou321• 8 years ago

I would agree that "no paper is that good" -- on the first try or even the second try. Just as we need constantly refactor code, just as we need constantly re-edit a book, a paper that explains a scientific idea also needs constant re-write to become good. The sad reality is that scholars simply do not do that.

It is understandable though. Once the paper is published, then it is no-longer novel, and there is obviously lack of reward and motivation to write the same idea again -- just to write it better. ... unless you are actually writing a review paper, but then, it appears there is little value unless the review is "complete". A good illustration of the idea need be able to high-light the key idea while avoid having minor ideas obscure the key. Therefore, a "complete" review paper rarely provide a good read.

Compared to papers, a good text-book often is a much better read than all the original papers. It is not really because of the size, rather it is because a good text-book is written from the reader's point of view and focuses on conveying the idea itself (vs. selling the idea). And a good text-book takes many rounds to develop.

There is no reason papers cannot be developed in a similar way as text books: once a ground-breaking paper is published, it shall be constantly updated, each new edition reflects what the author has newly learned and incorporating new development including the entire community...

Alas, that is not the culture, and there is no motivation to do such.

allenz2• 8 years ago

In addition to the incentive problem, you would also need to get people to peer review any changes to a paper. Most of the changes would be minor and annoying to peer review.

I think that courses remain the best way of distilling and communicating knowledge. The only problem is that they're not always available to a lay audience, since there's little incentive to make them available.

The other alternative for the lay audience is science/economics journalism (Economist, Scientific American, Discover, etc). This works to some extent but mostly just scratches the surface, since even regular journalism is struggling to be profitable these days. With deep technical topics there are too few readers and too few qualified, willing writers.

jacques_chester• 8 years ago

I think it's more useful for papers to be what they are: diffs. A textbook is a snapshot view. Both kinds of embodiment are necessary in their native form for different purposes.

nazgulnarsil• 8 years ago

Is there any cross disciplinary repository of excellent research reviews?

6ak74rfy• 8 years ago

On a slightly tangential note: how do you guys read research papers? Do you go through them word by word or you only skim them to get a general understanding?

I try to do the former but the work seems so boring that I am hardly motivated to do this more.

MereInterest• 8 years ago

Most of the time, skimming through, picking out the parts that are different from others in the field.

For physics: "Okay, this paper is from so and so, yup, same facility and equipment as the previous one. Theory section, same basic overview, skip to the end for what makes this experiment worthwhile? Anything interesting in their analysis, and how good are their statistics? Did they show the usual set of diagnostics plots for the type of analysis they did, and if not, what are they trying to downplay?"

rflrob• 8 years ago

For the most part, the way to read papers in biology is to go through the figures. When I was quickly trying to figure out if a project I was about to start up just got scooped (it did, and good for them for saving me the effort), I did it by glancing at figure 1, then reading the title of figure 1, then closely examining figure 1 while cross-referencing with the detailed caption. Then move on to 2, 3, etc. By figure 5, I had a hard time telling what was going on, so I found the part of the text that referenced it, and read it carefully.

I've heard arguments that one should leave the abstract until the end, since it sometimes pushes a story harder than the data necessarily warrants, and so if you come to your own conclusion about what's going on in the paper, then check to make sure the authors claim what you feel is supported, you're less likely to be misled. Probably this makes sense for reviewing a paper, but for most things, I'd rather find out quickly if the paper is even relevant to what I'm working on.

dekhn2• 8 years ago

I've found looking at figures and captions to be really misleading. I find that I have to read the methods section first to convince myself that the data collection was correct before I can trust any figures.

You have to read the abstract. It's the intro and conclusion you can skip.

pmiller22• 8 years ago

In mathematics:

Read the title and abstract. Then read the statements of the main theorems and corollaries (skip the lemmas and interstitial commentary at first). At each step, evaluate whether you want to continue or abandon the paper for whatever reason (not interesting, not relevant, whatever).

Finally, if you made it this far, read the whole paper word for word with a pen and paper (or chalk + chalkboard) handy.

For longer papers or books/theses (say, 20+ pages), do this whole process on each individual section/part/chapter that you want to consume.

It also helps to have a reason to read the paper besides curiosity. I find things that are relevant to current work to be easier to get through than more peripheral sorts of things. Having a colleague to discuss things with also helps.

sweezyjeezy• 8 years ago

FWIW, I don't think this article really applies to areas like maths or computer science, where the main output is theorems or algorithms that can be verified line by line. It's more about empirical science where one is trying to prove/disprove hypotheses through rigorous experimentation.

enriquto• 8 years ago

how do you guys read research papers

my phd advisor started by looking only at all the displayed formulas. If there was nothing interesting or new, he would not bother reading any sentence of the paper.

PeterisP• 8 years ago

It's worth to note that "reading papers" consists of two activities - actually reading a paper indenting to understand it, and skimming a large number of papers or abstracts to filter out which papers to actually read in detail - and it's the second part that seems to take the majority of time, at least for me.

dalbasal2• 8 years ago

I assume this relates mostly to economics or the humanities (social science, if you insist) in general. These apply to sciences too, but are less debilitating in the long run.

Fifth, most researchers’ priors are heavily influenced by some extremely suspicious factors.

Academics in the humanities identify themselves "I am an X." X can be post structuralist, rational materialist, classical liberal or some other broad, hairy, intellectual identity. This is bad news for objectivity. Everyone has a dog in the fight.

ci5er3• 8 years ago

It's interesting. As an engineering type (EE, primarily, but I had several other undergrad degrees), everything was about what is true. How do electrons move with their respective related fields across geometries.

This other stuff seems to be about "what is useful to us as humans in a group matrix (culture/community". That seems like it should be studied - but I don't think we all have the same objectives. Also - some people keep trying to make it "true" vs. "useful".

What say ye all? Do you think that "True" has any place in the social sciences vs. the observational "I observe this to be useful in this context" or the almost psycho-analytical "I notice that when people believe X together, society does better"?

I don't know - it just seems (to me) that we spend too much time on the truth as opposed to (subjectively) communally useful. i.e. What should we decide to believe in as a community to hang together (because otherwise, we shall surely hang separately)?

westoncb2• 8 years ago

What say ye all? Do you think that "True" has any place in the social sciences vs. the observational "I observe this to be useful in this context" or the almost psycho-analytical "I notice that when people believe X together, society does better"?

The problem with that is whoever takes it upon themselves to discard truth and consider 'what will have the best effect on people/society if they were to believe it' —is that you have to manipulate other people into believing it's true on the hope that your assessment of how it's going to affect people is accurate. It's almost certainly not.

It's interesting considering the human brain's criteria for accepting beliefs. It works pragmatically, not with strict adherence to truth—but, from what I can tell (and I've seen some research supporting this), the amount it's willing to deviate from (what it estimates to be) truth is proportional to the immediate demands of present circumstances. If you are in a crisis, it is willing to compromise and adopt a belief that conflicts with evidence, as long as it solves some immediate problem you're facing. But when that happens, you are accumulating something like technical debt. You become sort of 'out of harmony' with your circumstances, and problems will arise given enough time.

Why would it evolve to act in that way? It seems clear the answer is: because it's generally most beneficial to survival to adopt beliefs which best fit the observed data. It's a fair assumption that society holding true beliefs is also going to be most effective for survival.

So when someone manipulates someone else's beliefs because they think it will be beneficial, they are forcing that technical debt on them, and that's a decision only they should make for themselves. The same is true but amplified when considering spreading these ideas to wider audiences.

That said, much of contemporary social science and the humanities are rife with the opposite sort of thinking. This is heavily fueled by the fact that much of it doesn't consider 'correspondent' truth to be a real thing, so in their framework it is meaningless to say they are compromising the truth. Which truth? You can see this in Pragmatism's approach of literally redefining 'truth' to refer to something like 'the most effective belief to adopt'; and even more pervasive are all the schools of thought influenced by a kind of radical relativism which denies the possibility of truth. (It also clarifies a lot of philosophy once you realize the philosophers are actually thinking about the best mind-programs to put into people, rather than seeking truth as a typical person would recognize it. If only the sneaky bastards would say so to their readers' faces. But of course it would not be pragmatic to do so.)

It's a bad idea, and I sincerely hope the fields presently employing it fall into ruin, or are otherwise convinced to abandon it.

dalbasal2• 8 years ago

I'm in. Not everything needs to be true in the electrons-in-a-circuit sense to be useful. A lot of things mgs are subjective, both in the sense that it's arguable, but also in the sense that how people see it is important that in itself.

Man may not be the measure of *all" things, but he (she) is the measure of some things and that's ok. I think a lot of arguments (politics and political ideologies in particular) would get a lot nicer if we just accepted that we aren't arguing about absolute truth.

OlympusMonds• 8 years ago

I would say assuming sciences are somehow not affected by these sorts of factors is worse than what the humanities do - at least they admit a bias. From my own experiences in the hard sciences, the human-factor dominates over objectivity, from the individual through to peer review.

salty_biscuits• 8 years ago

Papers aren't meant to be this crystallized nugget of truth, they are a progress report on an ongoing piece of living research. They aren't meant to be infallible.

traverseda3• 8 years ago

On the psychology replication crisis

Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies.

https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...

Of course if you point out that psychology research is strongly biased, the immediate response is that you must be an anti-intellectual. As if the system that produced such misleading results is a representation of all intellectual pursuits, and not just a mistake. The same way you get called an anti-intellectual for questioning the post-modern-analyses of whatever.

I'm really sick of people claiming that criticizing obviously bad science is "anti-intellectual". When you produce that many useless papers you're obviously, as a field, lacking an understanding of why science is good and useful in the first place.

I'm presuming that economics has similar problems with replication, and that you can only trust the most basic and obvious of their findings.

sarbaz• 8 years ago

A lot of this is simply that p < 0.05 is a very low bar for research. If we want to understand how the world works, we need to be able to construct mechanisms that explain why the results we measure in experiments come about. Without this thing, it's not science.

A good mechanism that accurately explains some part of the world can be validated with a much better result than p < 0.05, because it can be properly isolated from other factors and its effect size can be strengthened.

Without a mechanism to explain how things work, the experimental results that get published are probably just noise. And because p < 0.05 is such a low bar to pass, lots of noise gets published. Trying to reproduce an experiment whose results were drawn from /dev/urandom is pointless.

In fact I'm surprised the replication was so high. I was expecting it to be around 5% (i.e. results are totally random). So maybe there is some hidden merit to all of this

nicoburns• 8 years ago

If we want to understand how the world works, we need to be able to construct mechanisms that explain why the results we measure in experiments come about.

This, 1000 times over. Unfortunately we seem to teach that statistical analysis is sufficient to infer causation in a lot our undergraduate science programs.

traverseda3• 8 years ago

I wouldn't expect totally random because people have common sense. I'd expect it to be similar to the folk-wisdom of the time, since a lot of it is going to be repackaging of that same folk-wisdom, and negative results tend not to get published, or get massaged until they're positive.

I'd expect social-psychology to mirror the dominant opinions of people in other segments of the humanities, although it is of course hard to determine the direction of that relationship.

UncleMeat2• 8 years ago

A serious problem is when people assume that because the replication crisis in psych is widely publicized that it is exclusive to psych and then they dismiss all of the findings in the field. In my experience the replication crisis is rarely used to criticize method but instead used to dismiss research out of hand, whether or not it is supported by other research that limits the likelihood of error.

I think that psych should be commended for attacking this problem. As a field they are funding replications, which basically never happens in any other field. We don't even know what the "replication crisis" would be in empirical CS. Given discussions with ML people I'd wager that the "replication crisis" in ML is at least as dramatic.

Science is messy. But this is why we should defer to experts who have vision over an entire field rather than focusing on individual papers as conclusive data points.

spruciefic3992• 8 years ago

To be consistent, you'd have to leverage your arguments against other fields with replication crises, which include neuroscience and oncology. It's really true of the biomedical sciences in general.

Psychology has a history of being sort of self-analytical (which makes sense if you're a field studying human behavior). Meta-analysis, for example, was largely incubated in psychology before it started expanding into other disciplines.

The criticism I'm aware of is really being leveled at the idea that this is somehow unique to psychology. It's like killing the messenger, or someone who is trying to fix the problem.

dekhn2• 8 years ago

Well, Watson & Crick (Nature 1953) is that good. It does contain an error (the actual DNA structure is slightly wrong) And Avery (JEM 1944) http://jem.rupress.org/content/79/2/137) which "proved" that DNA is the molecule of heredity, is also there.

But to be qualified to read these papers and appreciate why they are points of quality within a sea of crap? That's hard.

acoye• 8 years ago

Bitcoin's paper was that good.

Simple.

Easy to read.

Easy to replicate (a poc in python is easy)

It is a recent key innovation (the blockchain tech).

coldtea3• 8 years ago

The best papers get up to around .20. Again, No Paper Is That Good. If you demur, consider this: In twenty years, will you still hold up the best papers of today as “paragons” or “exemplars” of compelling empirical work?

If the field is not totally vague, then yes. We can still consider certain papers in physics, or chemistry, or medicine, computer science etc. as exemplary decades, or even centuries, later, even when they deal with empirical work.

Soft sciences need not apply.

AnimalMuppet• 8 years ago

Absolutely. Others have cited Shannon's paper on information theory. I would add Einstein's paper on Special Relativity. I read a translation of it on the 100th anniversary of its release, and I learned something, even though I already knew Special Relativity.

(What did I learn? I don't remember precisely - it's been 15 years - but I think it was that something on the electromagnetic side was caused by time dilation.)

UncleMeat2• 8 years ago

Soft sciences need not apply

Why not? Historians still revere Braudel and The Mediterranean was published nearly 70 years ago. Are you certain that there are no such exemplary works outside of the fields you like?

coldtea3• 8 years ago

No, but I'm certain that whether revered or not, Braudel's work is still "full of mistakes and bad reasoning" -- it's just that in history those are harder to tell.

Jweb_Guru• 8 years ago

Nonsense. Plenty of old computer science papers don't hold up well to scrutiny or are largely irrelevant nowadays, such as those that were trying to optimize for bottlenecks in hardware that no longer exist or have shifted to different places. Even in pure mathematics, something that was considered a great insight a long time ago might be considered fairly trivial now, and not just because the other paper came out first. Just because a paper is correct doesn't make it interesting or worthwhile--a lot of the best papers have errors in them, but propose something that's a genuinely new contribution to the field.

coldtea3• 8 years ago

Nonsense. Plenty of old computer science papers don't hold up well to scrutiny or are largely irrelevant nowadays

Nonsense, knee-jerk answer.

First, we don't need all of them to "hold up well to scrutiny" but just 2 (as per TFA challenge). And we have way more than just 2 -- hundreds of great papers.

Second, the papers "that were trying to optimize for bottlenecks in hardware that no longer exist or have shifted to different places" can still be perfectly valid as per the challenge we have, which was:

1) that they were not "full of mistakes and bad reasoning", 2) that they did not "contain little or no original work" (and where thus just references and meta-papers)

The question wasn't if we have "plenty of" papers that are bad, or plenty of papers that were very good but have been super-ceded by changes in technology.

Just that we have at least 2 (and I argue we have way more than two) seminal papers that have original work, and are not full of mistakes and bad reasoning.

kabouseng• 8 years ago

Claude Shannon's paper on information theory [1] is possibly an example of such a paragon of work. No citations, 50 pages of pure awesome, spawned a whole research field of its own and 70 years on just as valid.

[1] A Mathematical Theory of Communication - http://math.harvard.edu/~ctm/home/text/others/shannon/entrop...

eboyjr• 8 years ago

You said it. He proposed the model for digital AND analog communication still in use today. Although being a digital software person I admit I've never been motivated to read the part about continuous channels.

I was motivated however to install an archway for my house from the exact shape in Figure 7 (binary entropy function)

thidr0• 8 years ago

First paper I thought of too.

xurukefi• 8 years ago

No citations

The paper contains citations. But they are hidden in the footnotes - see page 1 or page 5 for example.

beagle3• 8 years ago

And it's easily readable with basic math/engineering knowledge. Highly recommended.

Erdos has a few papers that are "that good" as long as you know all the theory leading to the new insight, but none (that I read) are as clear and freestanding as Shannon's AMToC

joker3• 8 years ago

Math is different from empirical work. Yes, Shannon's paper is great, but that's not really relevant here because it's in a completely different paradigm.

turc1656• 8 years ago

I suspect this is limited to economics and the social sciences. There is absolutely no way this holds up in the natural sciences like math, physics, etc.

People forget the original name for what we call economics was "political economy". That alone should tell you all you need to know about the dangers of treating that field like a science. If you ever want to know why expert economists can't seem to agree on things that happened 50-100 years ago or make accurate predictions for the future, the original name is very telling. Wouldn't it be ridiculous if we were still debating the validity of f=ma or e=mc^2? Wouldn't it be crazy for someone to claim general relativity is just flat out wrong, even though GPS systems would not work properly without humans accepting it?

Why is nothing remotely approaching reasonable standards used in the social sciences before acceptance?

nitwit005• 8 years ago

Following the link through to the original proposal, this seems like a misinterpretation of the suggestion. The problem this was supposed to "fix" was people using the existence of their being "vast literature" to shut down arguments.

From that perspective, the suggestion seems fine. You should be able to dig out two examples that show your field isn't nonsense. I don't think it was meant to be a high bar:

There are actual examples of vast literatures that contain zero knowledge: Astrology, for instance. People have written so much about astrology that I bet you could spend decades reading what they've written and not even come close to the end. But at the end of the day, the only thing you'd know more about is the mindset of people who write about astrology. Because astrology is total and utter bunk.

gjm11• 8 years ago

It seems like a lot of commenters here are under the impression that the "two paper rule" (if someone says you should read the literature in field X, ask them for the two best papers in that field they can think of, and if those aren't impressive then don't bother looking further) is a proposal of Bryan Caplan, who wrote the OP here.

That is incorrect. The two-paper rule is Noah Smith's, and Bryan Caplan is disagreeing with it.

(It's scarcely possible to read any of Caplan's post without realising that; I conclude that many commenters here have not bothered to read the OP before commenting.)

No Paper Is That Good

Related Stories