Variations in the results of nutritional epidemiology studies due to analytic flexibility: Application of specification curve analysis to red meat and all-cause mortality

Abstract

AbstractObjectiveTo present an application of specification curve analysis—a novel analytic method that involves defining and implementing all plausible and valid analytic approaches for addressing a research question—to nutritional epidemiology.Data sourceNational Health and Nutrition Examination Survey (NHANES) 2007 to 2014 linked with National Death Index.MethodsWe reviewed all observational studies addressing the effect of red meat on all-cause mortality, sourced from a published systematic review, and documented variations in analytic methods (e.g., choice of model, covariates, etc.). We enumerated all defensible combinations of analytic choices to produce a comprehensive list of all the ways in which the data may reasonably be analyzed. We applied specification curve analysis to NHANES data to investigate the effect of unprocessed red meat on all-cause mortality, using all reasonable analytic specifications.ResultsAmong 15 publications reporting on 24 cohorts included in the systematic review on red meat and all-cause mortality, we identified 70 unique analytic methods, each including different analytic models, covariates, and operationalizations of red meat (e.g., continuous vs. quantiles). We applied specification curve analysis to NHANES, including 10,661 participants. Our specification curve analysis included 1,208 unique analytic specifications. Of 1,208 specifications, 435 (36.0%) yielded a hazard ratio equal to or above 1 for the effect of red meat on all-cause mortality and 773 (64.0%) below 1, with a median hazard ratio of 0.94 [IQR: 0.83 to 1.05]. Forty-eight specifications (3.97%) were statistically significant, 40 of which indicated unprocessed red meat to reduce all-cause mortality and 8 of which indicated red meat to increase mortality.ConclusionWe show that the application of specification curve analysis to nutritional epidemiology is feasible and presents an innovative solution to analytic flexibility.LimitationsAlternative analytic specifications may address slightly different questions and investigators may disagree about justifiable analytic approaches. Further, specification curve analysis is time and resource-intensive and may not always be feasible.