External Validation of DeepBleed: The first open-source 3D Deep Learning Network for the Segmentation of Intracerebral and Intraventricular Hemorrhage

Abstract

AbstractObjectivesDeepBleed is the first publicly available deep neural network model for the 3D segmentation of acute intracerebral hemorrhage (ICH) and intraventricular hemorrhage (IVH) on non-enhanced CT scans (NECT). The aim of this study was to evaluate the generalizability in an independent heterogenous ICH cohort and to improve the prediction accuracy by retraining the model.MethodsThis retrospective study included patients from three European stroke centers diagnosed with acute spontaneous ICH and IVH on NECT between January 2017 and June 2020. Patients were divided into a training-, validation- and test cohort according to the initial study. Model performance was evaluated using metrics of dice score (DSC), sensitivity, and positive predictive values (PPV) in the original model (OM) and the retrained model (RM) for each ICH location. Students’ t-test was used to compare the DSC between the two models. A multivariate linear regression model was used to identify variables associated with the DSC. Pearson correlation coefficients (r) were calculated to evaluate the volumetric agreement with the manual reference (ground truth: GT). Intraclass correlation coefficients (ICC) were calculated to evaluate segmentation agreement with the GT compared to expert raters.ResultsIn total, 1040 patients were included. Segmentations of the OM had a median DSC, sensitivity, and PPV of 0.84, 0.79, and 0.93, compared to 0.83, 0.80, and 0.91 in the RM, adjusted p-values > 0.05. Performance metrics for infratentorial ICH improved from a median DSC of 0.71 for brainstem and 0.48 for cerebellar ICH in the OM to 0.77 and 0.79 in the RM. ICH volume and location were significantly associated with the DSC, p-values < 0.05. Volumetric measurements showed strong agreement with the GT (r > 0.90), p-value >0.05. Agreement of the automated segmentations with the GT were excellent (ICC ≥ 0.9, p-values <0.001), however worse if compared to the human expert raters (p-values <0.0001).ConclusionsDeepBleed demonstrated an overall good generalization in an independent validation cohort and location specific variances improved significantly after model retraining. Segmentations and volume measurements showed a strong agreement with the manual reference; however, the quality of segmentations was lower compared to human expert raters. This is the first publicly available external validation of the open-source DeepBleed network for spontaneous ICH introduced by Sharrock et al.