Why Most Published Research Findings Are False

The Crisis of Science

James Corbett (2019)

Film Review

This documentary is loosely based on the 2005 paper by John Ioannidis (Why Most Published Research Findings Are False). According to Ioannidis, modern research faces four major crises: the crisis of fraud, the crisis of replication, the crisis of peer review and the crisis of publication.

The Crisis of Fraud: the number of scientists basing their findings on fraudulent data and/or methodology is clearly on the increase. This stems partly from the sad reality that corporations, which presently fund most research, tend to pay their researchers to produce favorable outcomes. However the pressure universities place on researchers to continuously publish also plays a significant role. The film features a 2008 clip of Robert F Kennedy Jr exposing emails he uncovered regarding a conspiracy between the CDC (Centers for Disease Control) and vaccine manufacturers to conceal a proven link between thimerosal (a mercury-based vaccine preservative) and autism and other neurodevelopmental disoders.

The Crisis of Replication: this relates both to outright fraud and a peer review process that places more weight on statistical significance than on sample size and reproducibility. Statisticians are well aware that measuring a large number of variables in a small group inevitably produces at least one “statistically significant” correlation. In the research community, this is known as “p-hacking”* or “data dredging.”

The Crisis of Peer Review: research has become so super specialized that only a handful of scientists are qualified to perform peer review in any given field. This allows them to act as gatekeepers. Not only are they more inclined to approve work by scientists they know or work with, but they’re also less inclined to check closely for data fraud and statistical manipulation in a friend’s work.

The Crisis of Publication: the pressure to publish (three times a year at some universities) creates a robust market for scientists who produce dodgy research.

---

*P-value or probability value is one important measure of statistical significance.