Publication Bias, Missing Data, and Sensitivity Analysis

Publication bias, missing data, and sensitivity analysis matter in meta-analysis because unavailable or selectively reported evidence can affect the validity of pooled estimates and the conclusions drawn from a systematic review. This session begins by distinguishing missing data within individual studies from entire studies or results that are unavailable for synthesis. Students will explore publication bias, selective reporting, and small-study effects, recognizing that studies with statistically significant or favorable findings may be more likely to be published or fully reported. These mechanisms can create an incomplete representation of the available evidence and potentially distort the estimated overall effect. Understanding these issues is essential for evaluating the reliability of meta-analysis findings and recognizing that the number of included studies does not necessarily reflect the full body of relevant research.

The second component focuses on methods for detecting and assessing potential publication bias and related patterns. Students will learn how to construct and interpret funnel plots, which display study effect estimates against a measure of their precision, and how asymmetry may indicate small-study effects or other sources of systematic differences. The session introduces funnel plot asymmetry tests, including Egger’s test, while emphasizing their limitations, particularly when only a small number of studies are available or when heterogeneity is substantial. Students will also examine the trim-and-fill method, which estimates potentially missing studies under specific assumptions and evaluates how their inclusion might affect the pooled estimate. In addition, they will consider approaches for addressing incompletely reported statistical information and understand why neither statistical tests nor adjustment methods can definitively establish or eliminate publication bias.

The final component introduces sensitivity analysis as a systematic approach to evaluating whether meta-analysis conclusions remain consistent under alternative analytical assumptions and methodological decisions. Students will explore sensitivity analyses based on model assumptions, study eligibility decisions, risk-of-bias assessments, and the exclusion or inclusion of studies that may strongly influence the pooled result. Comparing these alternative analyses helps identify robust findings and those that depend heavily on particular studies or analytical choices. By the end of this session, students should be able to recognize potential sources of bias and missing information, interpret diagnostic methods cautiously, conduct and explain sensitivity analyses conceptually, and report uncertainty and methodological limitations transparently. These skills are essential to ensure that meta-analysis conclusions reflect the strength and limitations of the available evidence, not an unjustified degree of certainty.