Quantifying Replicability and Consistency in Systematic Reviews

Iman Jaljuli, Yoav Benjamini, Liat Shenhav, Orestis A. Panagiotou, Ruth Heller

Research output: Contribution to journalArticlepeer-review

Abstract

Systematic reviews and meta-analyses are important tools for synthesizing evidence from multiple studies. They serve to increase power and improve precision, in the same way that large studies can do, but also to establish the consistency of effects and replicability of results across studies. In this work we propose statistical tools to quantify replicability of effect signs (or directions) and their consistency. We suggest that these tools accompany the fixed-effect or random-effects meta-analysis, and we show that they convey important information for the assessment of the intervention under investigation. We motivate and demonstrate our approach and its implications by examples from systematic reviews from the Cochrane Library. Our tools make no assumptions on the distribution of the true effect sizes, so their inferential guarantees continue to hold even if the assumptions of the fixed-effect or random-effects models do not hold. We also develop a version of this tool under the fixed-effect assumption for cases where it is crucial and justified.

Original languageEnglish
JournalStatistics in Biopharmaceutical Research
DOIs
StateAccepted/In press - 2022

Keywords

  • Cochrane collaboration
  • Drug discovery
  • Heterogeneity
  • Meta-analysis
  • Partial conjunction analysis
  • r-value

Fingerprint

Dive into the research topics of 'Quantifying Replicability and Consistency in Systematic Reviews'. Together they form a unique fingerprint.

Cite this