80% of current drug applications in China failed to meet analysis requirements

http://www.sciencealert.com/80-of-the-data-in-chinese-clinical-trial-is-fabricated

by ting_bu_dung • 10 years ago

127 41 10 years ago

Odenwaelder3• 10 years ago

In chinese culture, losing one's "face" is one of the worst things that can happen to a person. I'm wondering whether the fact that 80% of data has been fabricated is a result of that. As in, "I have to make my findings significant to not lose my face", or, "Presenting negative findings to my superior will make me lose my face". Maybe someone more familiar with this culture could elaborate.

ravenstine• 10 years ago

Very interesting analysis. If this is accurate, they should get rid of that attitude, pronto, as they have the highest potential of being on the scientific forefront(or at least that of medical science).

finid2• 10 years ago

Wait a minute!

That seems to be backward. Doesn't "losing face" come from being caught doing the wrong thing or being dishonest, so that the incentive to fabricate data for something as important as scientific research should be very, very low?

Odenwaelder3• 10 years ago

You might be right, depending on the culture's definition of "wrong". Let's hope you're right...

HillRat• 10 years ago

I'm not sure that reputational risk is the same thing; I suspect one way to look at it is that a researcher who returns the null case is disrupting the organization by putting a lot of other peoples' work and assurances at risk. So there would be a significant amount of implicit social pressure to show results and maintain organizational harmony.

psyc• 10 years ago

No, because losing face is not really about morality or honesty in the first place - it's more important than honesty. The primary thing here is the sharp and pervasive divide between facade and reality. The facade is more important. There are a lot of cultures where fronting is the main thing. It's just particularly evident in China.

mike_hock• 10 years ago

Being caught is the keyword.

radicsge• 10 years ago

It's not about the face. It was clearly written there "local pharmaceutical companies trying to produce Western drugs struggle to turn a profit"... What about Theranos in the US? Same..

Isn't it because the turn everything into profit within two years "success or nothing" attitude?

If something ruins the world, it will be two countries that under the surface is way too similar to each other.

brudgers2• 10 years ago

Radio Free Asia (RFA) is a private, nonprofit international broadcasting agency of the United States government[2] that broadcasts and publishes online news, information, and commentary to listeners in East Asia while "advancing the goals of U.S. foreign policy."

https://en.wikipedia.org/wiki/Radio_Free_Asia

The source cited is Radio Free Asia not the report itself. Radio Free Asia does not link to any additional sources.

That is not to say that the story is inaccurate, but rather that the claims are not well substantiated and that the source is not apolitical.

eveningcoffee2• 10 years ago

This is good catch, but considering the history of such programs (https://en.wikipedia.org/wiki/Radio_Free_Europe/Radio_Libert...), it is that they publish rather information that has been suspended by local governments than something that has been made up.

Of course you should be cautious with what they publish, but this applies to every published information.

brudgers2• 10 years ago

From the RFA story:

data gathered during clinical trials were incomplete, failed to meet analysis requirements or were untraceable, the paper cited a source in the agency as saying.

Outside of China, clinical data is often not shared, suppressed when it does not support the claims, and p-value hacking is used to justify 'off-label' use. This is not usually spun as fabricated, and in commercial drug development my intuition is that 80% of proposed drugs having at least one of these features would be a plausible hypothesis

tlb• 10 years ago

It mentions 1622 current clinical trials. Of those, how many would be expected to be approved in a rational system?

I wonder if the huge %age is due to a lot of low-quality applications mixed in with the serious ones. If the barrier to apply is sufficiently low, you would expect a Sturgeon's law dynamic.

Alex3917• 10 years ago

I wonder if the huge %age is due to a lot of low-quality applications mixed in with the serious ones.

I doubt it. Most clinical trial data in the U.S. is similarly fraudulent, but there isn't really any concept of high-quality vs low-quality applications like you might see with YC or whatever.

Instead, pharma companies tend to rely on the same set of 20 or 30 tricks to take chemical compounds that might be no better than placebos and make them look like wonder drugs. I've been keeping a tally of these tricks on my website:

http://www.alexkrupp.com/Citevault.html#pharmaceuticals

eveningcoffee2• 10 years ago

Considering past occurrences https://en.wikipedia.org/wiki/2008_Chinese_milk_scandal and general behaviour in online market places (my last experience was that the product was advertised to be shipped from Germany (expect fast delivery), but arrived a month later from China) then it should not come as very big surprise (I am not saying that this is normal or acceptable, but that it is not very surprising).

My question is that what is the cause of this general fraudulent behaviour? Does it have some historical roots? Does it come from lack of actual functioning oversight?

Another anecdotal case: try asking for directions in China. Even when person does not know the correct directions, they do not mind to direct you, and when confronted, they refuse to admit that they do not know. Is this mindset result of constant fear of repressions, or it runs deeper into the history of the society (say it is also present in Taiwan?).

I am not accusing anybody in anything, I just would like to understand.

_yvjs• 10 years ago

There was an interesting study looking at measures of honesty and economic growth: https://www.uea.ac.uk/documents/3154295/7054672/Honesty+pape....

Chinese nationals were in the top 2 of both measures of dishonesty (which didn't correlate all that well in general, so I'm not sure "honesty" is all that easy to measure), while British nationals were in the bottom 2 of both. One of the correlations they found was that rates of honesty were correlated with rates of Protestant Christianity.

rootkit55• 10 years ago

It would be interesting to see results of that game played for a million dollars. You go into a room alone and flip a coin. If you flip heads you win the million dollars. If you flip tails you win nothing.

It's also interesting that in situations like the Wells Fargo scandal, honesty didn't even matter. If you were honest, you were simply fired.

asddddd• 10 years ago

general behaviour in online market places (my last experience was that the product was advertised to be shipped from Germany (expect fast delivery), but arrived a month later from China)

Sounds more like a drop shipper reselling stuff from Aliexpress or similar, not necessarily anything to do with a Chinese seller. They add a hefty markup and ship directly to you. It's particularly popular with counterfeits ("replicas").

hzhou3213• 10 years ago

Historically scientists are lonely people who are (mostly) motivated by seeking truth or knowledge and satisfying curiosity instead of money or fame. Most people are practical and are motivated by money or fame more than satisfying curiosity. So most people are pursuing jobs rather than science. Fraudulent behavior is simply a result of seeking shortest path (shortcut) to a goal. When the goal is (short term) money and fame, and when the gain of taking the shortcut of fraud is bigger than the negative impact, then the popularity of the short cut is well expected.

So the problem is we reward so-called scientist too much (in terms of money or status). Scientists were historically not rewarded by money or status. In fact I don't think there is traditionally a career of science. People then were doing science on the side of their regular jobs (such as engineers or teachers or nobles who don't really need a job). Only when the reward of doing science is detached (to certain degree) from the reward of money and fame, can we truly prevent the fraudulent behavior.

On the other hand, the allure of money and fame probably can never be truly detached for any normal human, so fraudulent behavior is part of the human nature. It ebbs and flows. There is no need to make a big fuss about it.

PS: from what I heard, China today has so much (relative to its capacity) science money floating around and that got to distort the goal of science significantly.

maxander2• 10 years ago

I would have a hard time swallowing the argument that we pay scientists too much, or that society respects them too much. A distant observer might think this could be the case since, from their perspective, what they see as "scientists" are celebrated professors, showered with job stability, upper-middle-class incomes and book deals. But those lucky sods are vastly outnumbered by postdocs and tenure-track faculty, who work thankless hours at minimal pay and who could put their brains to work anywhere else for better reward. Those are the "typical scientist," on average, these days, and if we paid them any less they could barely afford to eat.

The problem is the imbalance- a postdoc really really wants to get tenure someday, because the only alternative is for his career to peter out and to end up working ignominiously in industry (as a second-rate software engineer, most likely.) And the number of tenure positions that could possibly be available is an order of magnitude fewer than the number of smart people competing for them. So he'll possibly "cut corners" to get there, despite the devotion to his field that made him choose that career and not something reasonable, like finance or software engineering. Academia is tenure or bust- there's no stable middle option.

There is a correction under way as people start to realize that academia is no longer a feasible career path, in the same way that the NFL isn't. So I'm hopeful that things will stabilize in regards to scientific integrity regardless- but, then we'll have to figure out who is going to do the science when we can't rely on an army of poorly paid geniuses.

clort• 10 years ago

asking for directions in China

If you don't know something and you show your ignorance in this way, then you lose face massively.

If they are involved in this (you ask them for directions) then you are tainting them with your lack of face.

They are hideously embarrassed and will tell you anything to make you go away.

rasz_pl• 10 years ago

Face. In China (and Asia in general) it is perfectly acceptable to LIE in your face if the alternative could be embarrassing. Dont know something? Lie. Competition is better? Lie. Will miss the deadline? Lie with a wide smile while shaking your hand. Lying itself is _not_ embarrassing.

daodedickinson• 10 years ago

Socialism, in the sense that is completely salvagable amid Chinese capitalism, is the situation of everyone having to talk like a politician.

DanielBMarkham2• 10 years ago

Science is having an astounding crisis -- I don't think there's been anything like it in history before.

If this is true about Chinese clinical data? More generally, if there is any simple tag that covers fraud at this scale? Other people working under that tag will experience deleterious effects for years to come.

The only way I see out of this is a rigorous, self-imposed body of standards. Even then I'm not sure that this will be sufficient.

Think of all those people, all of that money, all of the hundreds of thousands of hours of research. For nothing. Actually, it's worse than nothing because it's been flooding the system with fake data.

Odenwaelder3• 10 years ago

It's not only about fake data, it's also, maybe even most importantly, the low quality data. Publications with low data quality decrease the signal-to-noise ratio, leading to a slower scientific progress. Purely from a scientific standpoint: If you identify fake data, you just take them out of your model training set. But identifying low quality data is much harder.

DanielBMarkham2• 10 years ago

Agreed. Poor quality data is worse.

A couple of years ago, as this story started to break, I would mention it to scientist friends. "Don't worry," they'd say, "We always have the meta analysis to fall back on."

I don't hear that anymore.

ADD: A critical issue here, which you touch on tangentially, is the mix of motivation and milieu. If I'm making stuff up -- bad data -- other people can identify it and delete it. But if my goal is to appear to be doing good science instead of actually doing it, then it can become extremely problematic both to identify and remedy what I've done.

A lot of "normal" science is done around the margins, with not-so-incredible hypotheses and pedestrian-looking data. Bad science with really poor data fits seamlessly into that model without having any distinguishing characteristics.

AlexCoventry• 10 years ago

It's reminiscent of the role played by the Sophons in Cixin Liu's The Three-Body Problem, except in this case it's self-inflicted.

https://en.wikipedia.org/wiki/The_Three-Body_Problem#First_C...

ksk• 10 years ago

Title should make it clear that this article is about currently pending applications, not about all clinical trials.

sctb• 10 years ago

Thanks, we've updated the title.

projektfu2• 10 years ago

In the couple years I spent doing biology research, about 8 years ago, I was skeptical of most research from China because it already had a reputation already of being untrustworthy. But why stop at China?

I spent a good part of a year trying and failing to replicate an English study, and looking around for materials referenced in another study that simply don't exist, finally being told by the principal on that study that they mischaracterized the materials in the paper. Sometimes when you get the results you are hoping for, you publish, and you don't care if they were wrong, right?

Perhaps it would be better if scientists generally took a more skeptical view of others' publications and tried to replicate them in house. At conferences, instead of asking, "Where do we go from here", with the first study, ask, "What do I need to do to replicate this?"

nonbel• 10 years ago

I had a very similar experience. And you are correct that these problems are not at all limited to "China", although those authors tend to be a little less sophisticated in their (what may generously be called) shortcuts.

For example, if you do a real literature review on a medical topic (not just credulously take their word for it) you are almost sure to find that one of the primary assumptions/claims was actually never studied. One paper refs a second which refs a third that speculated about it in the introduction, but now "everyone knows it".

projektfu2• 10 years ago

So true. This is especially important as people study ways of improving metrics of markers of disease instead of actually improving disease endpoints. For example, there is a supplement marketed in the veterinary space that may lower blood urea nitrogen in uremic animals, according to a study. Whoop-de-do! I don't care about urea, it doesn't cause the disease of uremia. It is just a marker. You could convert all the urea to ammonia and now have a patient with two diseases and a BUN of 0.

msie• 10 years ago

Ting bu dung...

jza00425• 10 years ago

lol, after checking comments and stories of this guy, what a deep love to China

canada_dry• 10 years ago

No wonder wealthy Chinese go out of their way to buy not-made-in-China food and medicine.

80% of current drug applications in China failed to meet analysis requirements

Related Stories