Performances of machine learning algorithms in discriminating sacroiliitis features on MRI: a systematic review

Abstract

ObjectivesSummarise the evidence of the performance of the machine learning algorithm in discriminating sacroiliitis features on MRI and compare it with the accuracy of human physicians.MethodsMEDLINE, EMBASE, CIHNAL, Web of Science, IEEE, American College of Rheumatology and European Alliance of Associations for Rheumatology abstract archives were searched for studies published between 2008 and 4 June 2023. Two authors independently screened and extracted the variables, and the results are presented using tables and forest plots.ResultsTen studies were selected from 2381. Over half of the studies used deep learning models, using Assessment of Spondyloarthritis International Society sacroiliitis criteria as the ground truth, and manually extracted the regions of interest. All studies reported the area under the curve as a performance index, ranging from 0.76 to 0.99. Sensitivity and specificity were the second-most commonly reported indices, with sensitivity ranging from 0.56 to 1.00 and specificity ranging from 0.67 to 1.00; these results are comparable to a radiologist’s sensitivity of 0.67–1.00 and specificity of 0.78–1.00 in the same cohort. More than half of the studies showed a high risk of bias in the analysis domain of quality appraisal owing to the small sample size or overfitting issues.ConclusionThe performance of machine learning algorithms in discriminating sacroiliitis features on MRI varied owing to the high heterogeneity between studies and the small sample sizes, overfitting, and under-reporting issues of individual studies. Further well-designed and transparent studies are required.