A deep learning-based system for real-time image reporting during esophagogastroduodenoscopy: a multicenter study

Abstract

Background and study aims Endoscopic reports are essential for the diagnosis and follow-up of gastrointestinal diseases. This study aimed to construct an intelligent system for automatic photo documentation during esophagogastroduodenoscopy (EGD) and test its utility in clinical practice. Patients and methods Seven convolutional neural networks trained and tested using 210,198 images were integrated to construct the endoscopic automatic image reporting system (EAIRS). We tested its performance through man-machine comparison at three levels: internal, external, and prospective test. Between May 2021 and June 2021, patients undergoing EGD at Renmin Hospital of Wuhan University were recruited. The primary outcomes were accuracy for capturing anatomical landmarks, completeness for capturing anatomical landmarks, and detected lesions. Results The EAIRS outperformed endoscopists in retrospective internal and external test. A total of 161 consecutive patients were enrolled in the prospective test. The EAIRS achieved an accuracy of 95.2% in capturing anatomical landmarks in the prospective test. It also achieved higher completeness on capturing anatomical landmarks compared with endoscopists: (93.1% vs. 88.8%), and was comparable to endoscopists on capturing detected lesions: (99.0% vs. 98.0%). Conclusions The EAIRS can generate qualified image reports and could be a powerful tool for generating endoscopic reports in clinical practice.