News - Interdyscyplinarne Centrum Etyki (INCET)

Skip banner
INCET logo

BIOUNCERTAINTY - ERC Starting Grant no. 805498

ERC logo

[Important! Event date changed] 26th October 2020: Research seminar online: Is meta-analysis a reliable source of evidence for decisions in the clinic?

[Important! Event date changed] 26th October 2020: Research seminar online: Is meta-analysis a reliable source of evidence for decisions in the clinic?

We have the pleasure to invite you for a research seminar in the ‘BIOUNCERTAINTY’ research project. The subject of the seminar is 'Is meta-analysis a reliable source of evidence for decisions in the clinic?'. The seminar will take place on Monday, 26th of October, at 5:30pm on MS Teams (link below).

 

Abstract: Literature-based meta-analysis is a standard technique applied to pooling results of individual studies used in medicine and social sciences. It has been criticized for [1] being too malleable to constrain results, [2] averaging incomparable values, [3] lacking a measure of strength of evidence, and [4] problems with systematic bias of individual studies. Despite these epistemic concerns, the current practice of decision-making and preparing clinical guidance relies to a high degree on evidence from meta-analysis. We argue against the use of literature-based meta-analysis of RCTs for assessment of treatment efficacy by showing that therapeutic decisions based on meta-analytic average of individual studies are not optimal given the full scope of existing evidence. The argument proceeds with discussing examples and analyzing properties of some common meta-analytic techniques. First, we demonstrate that the use of meta-analysis can lead to reporting statistically significant results despite limited efficacy of the treatment. This, in turn, leads to a huge number needed to treat (NNT) and an unfavorable tradeoff between benefits and harms of the treatment. Second, we show that meta-analytic confidence intervals (CI) are too narrow comparing to the variability of treatment outcomes reported by individual studies. Third, based on a review of studies comparing results of literature-based meta-analysis and IPD meta-analysis that report non-systematic differences in treatment effect estimates, we conclude that literature-based meta-analysis lacks reliability. Finally, we analyze amalgamating conflicting RCTs (i.e., studies reporting both positive and negative treatment effects). In this case, meta-analysis averages out the differences among studies and leads to a loss of information  if no mediating variable can be found. Despite these problems, literature-based meta-analysis is useful for the assessment of harms. We support two alternative approaches to concluding systematic literature review: meta-analysis of individual patient data (IPD) and qualitative review employing mechanistic evidence.

Recommended
23rd of October 2026  — dr Amber Comer  — Life Unworthy of Life: Professional Reorientation and the Object of Protection in Nazi Medicine
23rd of October 2026 — dr Amber Comer — Life Unworthy of Life: Professional Reorientation and the Object of Protection in Nazi Medicine
19th of November 2026  — dr. Guilherme Almeida — Dual character concepts and metalinguistic negotiations in legal philosophy and beyond
19th of November 2026 — dr. Guilherme Almeida — Dual character concepts and metalinguistic negotiations in legal philosophy and beyond
12th of November 2026  — dr Diego Almonacid Almarza — The Causal Efficacy of Legal Norms: An Interventionist Approach
12th of November 2026 — dr Diego Almonacid Almarza — The Causal Efficacy of Legal Norms: An Interventionist Approach
New paper by Tomasz Żuradzki
New paper by Tomasz Żuradzki