Item talk:Q146375

Opinion: A preferred approach for dealing with reproducibility and replicability in science

Science impacts our daily lives and guides national and international policies (1). Thus, results of scientific studies are of paramount importance; yet, there are concerns that many studies are not reproducible or replicable (2). To address these concerns, the National Research Council conducted a Consensus Study [NASEM 2019 (3)] that provides definitions of key concepts, discussions of problems, and recommendations for dealing with these problems. These recommendations are useful and well considered, but they do not go far enough in our opinion. The NASEM recommendations treat reproducibility and replicability as single-study issues, despite clear acknowledgement of the limitations of isolated studies and the need for research synthesis (3). We advocate a strategic approach to research, focusing on the accumulation of evidence via designed sequences of studies, as a means of dealing more effectively with reproducibility, replicability, and related problems. These sequences are designed to provide iterative tests based on comparison of data from empirical studies with predictions from competing hypotheses. Evidence is then formally accumulated based on the relative predictive abilities of the different hypotheses as the sequential studies proceed.