Author:
Madadi Yeganeh,Delsoz Mohammad,Lao Priscilla A.,Fong Joseph W.,Hollingsworth TJ,Kahook Malik Y.,Yousefi Siamak
Abstract
ABSTRACTPurposeTo evaluate the efficiency of large language models (LLMs) including ChatGPT to assist in diagnosing neuro-ophthalmic diseases based on case reports.DesignProspective studySubjects or ParticipantsWe selected 22 different case reports of neuro-ophthalmic diseases from a publicly available online database. These cases included a wide range of chronic and acute diseases that are commonly seen by neuro-ophthalmic sub-specialists.MethodsWe inserted the text from each case as a new prompt into both ChatGPT v3.5 and ChatGPT Plus v4.0 and asked for the most probable diagnosis. We then presented the exact information to two neuro-ophthalmologists and recorded their diagnoses followed by comparison to responses from both versions of ChatGPT.Main Outcome MeasuresDiagnostic accuracy in terms of number of correctly diagnosed cases among diagnoses.ResultsChatGPT v3.5, ChatGPT Plus v4.0, and the two neuro-ophthalmologists were correct in 13 (59%), 18 (82%), 19 (86%), and 19 (86%) out of 22 cases, respectively. The agreement between the various diagnostic sources were as follows: ChatGPT v3.5 and ChatGPT Plus v4.0, 13 (59%); ChatGPT v3.5 and the first neuro-ophthalmologist, 12 (55%); ChatGPT v3.5 and the second neuro-ophthalmologist, 12 (55%); ChatGPT Plus v4.0 and the first neuro-ophthalmologist, 17 (77%); ChatGPT Plus v4.0 and the second neuro-ophthalmologist, 16 (73%); and first and second neuro-ophthalmologists 17 (17%).ConclusionsThe accuracy of ChatGPT v3.5 and ChatGPT Plus v4.0 in diagnosing patients with neuro-ophthalmic diseases was 59% and 82%, respectively. With further development, ChatGPT Plus v4.0 may have potential to be used in clinical care settings to assist clinicians in providing quick, accurate diagnoses of patients in neuro-ophthalmology. The applicability of using LLMs like ChatGPT in clinical settings that lack access to subspeciality trained neuro-ophthalmologists deserves further research.Summary Highlights-The goal of this study was to explore the capabilities of ChatGPT for the diagnoses of different neuro-ophthalmic diseases using specific case examples.-There was general agreement between ChatGPT Plus v4.0 and two neuro-ophthalmologists in final diagnoses.-ChatGPT was more general while neuro-ophthalmologists were more methodical and specific when listing diagnoses.
Publisher
Cold Spring Harbor Laboratory
Reference16 articles.
1. Levin LA , Nilsson SF , Ver Hoeve J , Wu S , Kaufman PL , Alm A. Adler’s Physiology of the Eye E-Book: Expert Consult-Online and Print. Elsevier Health Sciences; 2011.
2. Patient harm due to diagnostic error of neuroophthalmologic conditions;Ophthalmology,2021
3. The human resource crisis in neuro-ophthalmology;Journal of Neuro-Ophthalmology,2008
4. Mismatch in supply and demand for neuro-ophthalmic care;Journal of Neuro-Ophthalmology,2022
5. A guide to deep learning in healthcare
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献