You are here: Home DFG - Research Unit Projects Review


Evaluation of bias in systematic reviews

Duration: 2004-2010

Principal investigator: Dr. Guido Schwarzer (IMBI)

Researchers: Dr. Guido Schwarzer (IMBI), Dr. James Carpenter (IMBI), Dipl. Math. Gerta Rücker (IMBI)



The overall aim of the project was to evaluate and develop `modern statistical methods for the detection, quantification and correction of bias in the results of systematic reviews', with a focus on methods for binary outcomes. These purposes were achieved, partly even exceeded. During the funding period, we pursued:

  • Citation bias in psychiatry: We conducted an empirical investigation of citation bias in four psychiatric journals (Nieminen et al., 2006, 2007)
  • Accurate tests for publication bias: We proposed arcsine tests for publication (Rücker et al., 2008). James Carpenter is a coauthor of the chapter on publication bias of `The Cochrane Handbook of Systematic Reviews of Interventions' (Higgins, Green, 2011, Guido Schwarzer, Gerta Rücker and James Carpenter are coauthors of a consensus statement on the use of tests for publication bias which has been published in the British Medical Journal (Sterne et al., 2011).
  • Sensitivity analysis for publication bias and other sources of bias: This article has been published (Carpenter et al., 2011).
  • Cooperation with Non-randomized}: Gerta Rücker, Erika Graf and Martin Schumacher published a letter on randomized/non-randomized hybrid designs (Rücker et al., 2010).

Our work will be also disseminated by further developing the R packages meta and copas (Carpenter et al., 2009), an educational article on publication bias in German (Schwarzer and Rücker, 2010), and by giving talks and courses in meta-analytic methods.


We did not pursue a few minor goals:

  • methods for missing data in meta-analysis,
  • studying other models proposed by John Copas,
  • cooperation with Pharmacoepi project, since this project was not part of the research group in the second funding period.


  • motivated by our idea of arcsine tests, we studied the arcsine difference as a treatment effect measure, particularly in the rare event setting (Rücker et al., 2009)
  • we published an article on measuring heterogeneity (Rücker et al, 2008) and visualizing confounding in meta-analyses (Rücker and Schumacher, 2008),
  • we contributed a new method of estimating treatment effect effects adjusted for small-study effects, of which publication bias is a special case (Rücker et al., 2010),
  • we compared this and other methods in examples and simulations (Carpenter et al., 2009; Schwarzer et al., 2010; Rücker et al., 2011)
  • we contributed to the discussion of the fixed vs the random effects model (Carpenter et al., 2008; Rücker et al., 2010)
  • we started working on meta-analysis of diagnostic accuracy studies (Rücker and Schumacher, 2010), and
  • we started a cooperation with the High-dimensional project for an application of capture-recapture techniques, including a boosting approach, in literature search for systematic reviews (Rücker et al., 2011).