Predictability of buprenorphine‐naloxone treatment retention: A multi‐site analysis combining electronic health records and machine learning

Abstract

AbstractBackground and aimsOpioid use disorder (OUD) and opioid dependence lead to significant morbidity and mortality, yet treatment retention, crucial for the effectiveness of medications like buprenorphine‐naloxone, remains unpredictable. Our objective was to determine the predictability of 6‐month retention in buprenorphine‐naloxone treatment using electronic health record (EHR) data from diverse clinical settings and to identify key predictors.DesignThis retrospective observational study developed and validated machine learning‐based clinical risk prediction models using EHR data.Setting and casesData were sourced from Stanford University's healthcare system and Holmusk's NeuroBlu database, reflecting a wide range of healthcare settings. The study analyzed 1800 Stanford and 7957 NeuroBlu treatment encounters from 2008 to 2023 and from 2003 to 2023, respectively.MeasurementsPredict continuous prescription of buprenorphine‐naloxone for at least 6 months, without a gap of more than 30 days. The performance of machine learning prediction models was assessed by area under receiver operating characteristic (ROC‐AUC) analysis as well as precision, recall and calibration. To further validate our approach's clinical applicability, we conducted two secondary analyses: a time‐to‐event analysis on a single site to estimate the duration of buprenorphine‐naloxone treatment continuity evaluated by the C‐index and a comparative evaluation against predictions made by three human clinical experts.FindingsAttrition rates at 6 months were 58% (NeuroBlu) and 61% (Stanford). Prediction models trained and internally validated on NeuroBlu data achieved ROC‐AUCs up to 75.8 (95% confidence interval [CI] = 73.6–78.0). Addiction medicine specialists' predictions show a ROC‐AUC of 67.8 (95% CI = 50.4–85.2). Time‐to‐event analysis on Stanford data indicated a median treatment retention time of 65 days, with random survival forest model achieving an average C‐index of 65.9. The top predictor of treatment retention identified included the diagnosis of opioid dependence.ConclusionsUS patients with opioid use disorder or opioid dependence treated with buprenorphine‐naloxone prescriptions appear to have a high (∼60%) treatment attrition by 6 months. Machine learning models trained on diverse electronic health record datasets appear to be able to predict treatment continuity with accuracy comparable to that of clinical experts.