Multimodal Deep Learning for Classifying Diabetes: Analyzing Carotid Ultrasound Images from UK and Taiwan Biobanks and Their Cardiovascular Disease Associations

Abstract

Abstract Objective Clinical evidence has shown that carotid intima-media thickness (CIMT) is a robust biomarker for determining the thickness of atherosclerosis, which in turn increases the risk of cardiovascular disease (CVD). Additionally, diabetes mellitus (DM) is linked to the acceleration of atherosclerosis. Thus, as measured by carotid ultrasound (US), CIMT exhibits a significant association with both DM and CVD. This study examines the potential of US image features, beyond CIMT, in enhancing DM classification and their subsequent association with CVD risks. Specifically, we aimed to determine if these US image features could contribute to DM classification in conjunction with traditional predictors such as age, sex, CIMT, and body mass index (BMI). Additionally, we evaluated the relationship between the probabilities derived from the DM classification model and the prevalence and incidence of CVD in DM patients.Materials and Methods Utilizing carotid US image data from the UK Biobank (UKB) and Taiwan Biobank (TWB), we developed and trained a custom multimodal DM classification model. This model employed a Convolutional Neural Network (CNN) deep learning approach, using data from the UKB. We assessed the model's performance by comparing it with traditional models that incorporate only clinical features (age, sex, CIMT, BMI). The same comparative analysis was performed on the TWB data. Logistic regression was utilized to analyze the associations between the DM classification model's probability outcomes and CVD status.Results Our comprehensive performance evaluation across both the UKB and TWB datasets revealed that the multimodal DM classification model, which considers both image and clinical features (Age, Sex, CIMT, BMI), outperformed models that rely solely on clinical features. This was evidenced by an improved average precision of 0.762, recall of 0.655, specificity of 0.79, and accuracy of 0.721. Furthermore, in the UKB dataset, we identified a statistically significant association between the probabilities derived from the DM model and CVD status in DM patients, both prevalent (P-value: 0.006) and incident (P-value: 0.058), particularly on the left side.Conclusions The study provides robust evidence that carotid US image features, in addition to traditional parameters like CIMT, significantly enhance the capability of the multimodal DM classification model. The probability outcomes from this model could serve as a promising biomarker for assessing CVD risk in DM patients, offering a novel approach in the medical imaging field.