Weak Statistical Standards Implicated in Scientific Irreproducibility
By Erika Check Hayden,
Nature
| 11. 11. 2013
The
plague of non-reproducibility in science may be mostly due to scientists’ use of weak statistical tests, as shown by an innovative method developed by statistician Valen Johnson, at Texas A&M University in College Station.
Johnson compared the strength of two types of tests: frequentist tests, which measure how unlikely a finding is to occur by chance, and Bayesian tests, which measure the likelihood that a particular hypothesis is correct given data collected in the study. The strength of the results given by these two types of tests had not been compared before, because they ask slightly different types of questions.
So Johnson developed a method that makes the results given by the tests — the P value in the frequentist paradigm, and the Bayes factor in the Bayesian paradigm — directly comparable. Unlike frequentist tests, which use objective calculations to reject a null hypothesis, Bayesian tests require the tester to define an alternative hypothesis to be tested — a subjective process. But Johnson developed a 'uniformly most powerful' Bayesian test that defines the alternative hypothesis in a standard way...
Related Articles
President Trump scored a bunch of generally favorable mainstream headlines recently by announcing that he was ordering expanded access to in vitro fertilization (IVF). He had announced in October that he was “the father of IVF” although he also said he had only just learned what it was from Senator Katie Britt (R-AL), who explained it to him over the phone. “And within about two minutes, I understood it.”
The executive order, as reporter Susan Rinkunas wryly noted at...
By Staff, The Economist | 02.21.2025
One of the greatest scandals in modern science began with a late-2010s advertisement for HIV-positive couples looking to have children through in-vitro fertilisation (IVF). The ad had been put out by a scientist named He Jiankui, a biologist then at...
By Ewan Bolton, The Telegraph | 02.18.2025
Just last week, in the Miyagi prefecture of Japan, the local governor met victims of an official eugenics policy to apologise for the suffering they underwent.
“On behalf of the prefecture, I would like to offer my heartfelt apologies for...
By Anne Branigin, The Washington Post [cites CGS' Marcy Darnovsky] | 02.04.2025