Scope, Methods, and Overview Findings for the Making Numbers Meaningful Evidence Review of Communicating Probabilities in Health: A Systematic Review

Abstract

Background. The format in which probabilities are presented influences comprehension and interpretation. Purpose. To develop comprehensive evidence-based guidance about how to communicate probabilities in health and to identify strengths and weaknesses in the literature. This article presents methods for the review of probability communication and is accompanied by several results articles. Data Sources. MEDLINE, Embase, CINAHL, the Cochrane Library, PsycINFO, ERIC, ACM Digital Library; hand search of 4 journals. Study Selection. Two reviewers conducted screening to identify experimental and quasi-experimental research that compared 2 or more formats for presenting quantitative health information to patients or lay audiences. Data Extraction. In our conceptual framework, people make sense of a stimulus (data in a data presentation format) by performing cognitive tasks, resulting in perceptual, affective, cognitive, or behavioral responses measured as 1 of 14 distinct outcomes. The study team developed custom instruments to extract concepts, conduct risk-of-bias evaluation, and evaluate individual findings for credibility. Data Synthesis. Findings were grouped into tables by task and outcome for evidence synthesis. Limitations. Reviewer error could have led to missing relevant studies despite having 2 independent reviewers screening each article. The granular data extraction and syntheses slowed the work and may have made it less replicable. Credibility was evaluated by only 2 experts. Conclusions. After reviewing 26,793 titles and abstracts, we identified 316 articles about probability communication. Data extraction produced 1,119 individual findings, which were grouped into 37 evidence tables, each containing evidence on up to 10 data presentation format comparisons. The Making Numbers Meaningful project required novel methods for classifying and synthesizing research, which reveal patterns of strength and weakness in the probability communication literature. Highlights The Making Numbers Meaningful project conducted a comprehensive systematic review of experimental and quasi-experimental research that compared 2 or more formats for presenting quantitative health information to patients or other lay audiences. The current article focuses on probability information. Based on a conceptual taxonomy, we reviewed studies based on the cognitive tasks required of participants, assessing 14 distinct possible outcomes. Our review identified 316 articles involving probability communications that generated 1,119 distinct research findings, each of which was reviewed by multiple experts for credibility. The overall pattern of findings highlights which probability communication questions have been well researched and which have not. For example, there has been far more research on communicating single probabilities than on communicating more complex information such as trends over time, and there has been a large amount of research on the effect of communication approaches on behavioral intentions but relatively little on behaviors.