The Interpretation of Random-Effects Meta-Analysis in Decision Models

Abstract

This article shows that the interpretation of the random-effects models used in meta-analysis to summarize heterogeneous treatment effects can have a marked effect on the results from decision models. Sources of variation in meta-analysis include the following: random variation in outcome definition (amounting to a form of measurement error), variation between the patient groups in different trials, variation between protocols, and variation in the way a given protocol is implemented. Each of these alternatives leads to a different model for how the heterogeneity in the effect sizes previously observed might relate to the effect size(s) in a future implementation. Furthermore, these alternative models require different computations and, when the net benefits are nonlinear in the efficacy parameters, result in different expected net benefits. The authors’ analysis suggests that the mean treatment effect from a random-effects meta-analysis will only seldom be an appropriate representation of the efficacy expected in a future implementation. Instead, modelers should consider either the predictive distribution of a future treatment effect, or they should assume that the future implementation will result in a distribution of treatment effects. A worked example, in a probabilistic, Bayesian posterior framework, is used to illustrate the alternative computations and to show how parameter uncertainty can be combined with variation between individuals and heterogeneity in meta-analysis.