Skip to main content

Advertisement

SpringerLink
Log in

Early prediction of sepsis using chatGPT-generated summaries and structured data

  • 1243: Multi-modal Information Analysis and Applications based on Chat-GPT
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a large language models (LLMs) assisted algorithm that uses ChatGPT to summarize clinical notes and then concatenate these generated summaries with structured data to predict sepsis. We perform a human evaluation of the summaries generated by ChatGPT and evaluate our algorithm using an independent test set. Our algorithm achieves a high prediction AUC of 0.93 (95% CI 0.92-0.93), accuracy of 0.92 (95% CI 0.91-0.92), and specificity of 0.89 (95% CI 0.88-0.90) 4 hours before the onset of sepsis. The ablation study demonstrated a 2% improvement in predicted AUC score when utilizing ChatGPT for clinical notes summarization compared to traditional methods, 4 hours before the sepsis onset. The experiment results in turn revealed the remarkable performance of ChatGPT in the domain of clinical notes summarization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The datasets generated during and analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche J-D, Coopersmith CM et al (2016) The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 315(8):801–810

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, Suppes R, Feinstein D, Zanotti S, Taiberg L et al (2006) Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 34(6):1589–1596

    Article  PubMed  Google Scholar 

  3. Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ, Phillips GS, Lemeshow S, Osborn T, Terry KM, Levy MM (2017) Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med 376(23):2235–2244

    Article  PubMed  PubMed Central  Google Scholar 

  4. Islam MM, Nasrin T, Walther BA, Wu C-C, Yang H-C, Li Y-C (2019) Prediction of sepsis patients using machine learning approach: a meta-analysis. Comput Methods Programs Biomed 170:1–9

    Article  PubMed  Google Scholar 

  5. Duan Y, Huo J, Chen M, Hou F, Yan G, Li S, Wang H (2023) Early prediction of sepsis using double fusion of deep features and handcrafted features. Applied Intelligence, 1–17

  6. Reyna MA, Josef C, Seyedi S, Jeter R, Shashikumar SP, Westover MB, Sharma A, Nemati S, Clifford GD (2019) Early prediction of sepsis from clinical data: the physionet/computing in cardiology challenge 2019. In: 2019 computing in cardiology (CinC), p 1 . IEEE

  7. Jensen K, Soguero-Ruiz C, Oyvind Mikalsen K, Lindsetmo R-O, Kouskoumvekaki I, Girolami M, Olav Skrovseth S, Augestad KM (2017) Analysis of free text in electronic health records for identification of cancer patient trajectories. Sci Rep 7(1):46226

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  8. Goh KH, Wang L, Yeow AYK, Poh H, Li K, Yeow JJL, Tan GYH (2021) Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nat Commun 12(1):711

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  9. Amrollahi F, Shashikumar SP, Razmi F, Nemati S (2020) Contextual embeddings from clinical notes improves prediction of sepsis. In: AMIA annual symposium proceedings, vol 2020, p 197. American medical informatics association

  10. Qin F, Madan V, Ratan U, Karnin Z, Kapoor V, Bhatia P, Kass-Hout T (2021) Improving early sepsis prediction with multi modal learning. arXiv preprint arXiv:2107.11094

  11. Horng S, Sontag DA, Halpern Y, Jernite Y, Shapiro NI, Nathanson LA (2017) Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning. PLoS ONE 12(4):0174708

    Article  Google Scholar 

  12. Apostolova E, Velez T (2018) Toward automated early sepsis alerting: identifying infection patients from nursing notes. arXiv preprint arXiv:1809.03995

  13. Culliton P, Levinson M, Ehresman A, Wherry J, Steingrub JS, Gallant SI (2017) Predicting severe sepsis using text from the electronic health record. arXiv preprint arXiv:1711.11536

  14. Alsentzer E, Murphy JR, Boag W, Weng W-H, Jin D, Naumann T, McDermott M (2019) Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323

  15. Yan MY, Gustad LT, Nytrø Ø (2022) Sepsis prediction, early detection, and identification using clinical text for machine learning: a systematic review. J Am Med Inform Assoc 29(3):559–575

    Article  PubMed  Google Scholar 

  16. Zhang T, Ladhak F, Durmus E, Liang P, McKeown K, Hashimoto TB (2023) Benchmarking large language models for news summarization. arXiv preprint arXiv:2301.13848

  17. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901

    Google Scholar 

  18. Leiter C, Zhang R, Chen Y, Belouadi J, Larionov D, Fresen V, Eger S (2023) Chatgpt: A meta-analysis after 2.5 months. arXiv preprint arXiv:2302.13795

  19. Wiering MA, Van Otterlo M (2012) Reinforcement learning. Adapt Learn Optim 12(3):729

    Google Scholar 

  20. Nie W, Wen X, Liu J, Chen J, Wu J, Jin G, Lu J, Liu A-A (2023) Knowledge-enhanced causal reinforcement learning model for interactive recommendation. IEEE Transactions on Multimedia

  21. Goyal T, Li JJ, Durrett G (2022) News summarization and evaluation in the era of gpt-3. arXiv preprint arXiv:2209.12356

  22. Liu Y, Liu P, Radev D, Neubig G (2022) Brio: Bringing order to abstractive summarization. arXiv preprint arXiv:2203.16804

  23. Sanh V, Webson A, Raffel C, Bach SH, Sutawika L, Alyafeai Z, Chaffin A, Stiegler A, Scao TL, Raja A, et al. (2021) Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207

  24. Polat G, Ugan RA, Cadirci E, Halici Z (2017) Sepsis and septic shock: current treatment strategies and new approaches. Eurasian J Med 49(1):53

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Jaimes F, Garcés J, Cuervo J, Ramírez F, Ramírez J, Vargas A, Quintero C, Ochoa J, Tandioy F, Zapata L et al (2003) The systemic inflammatory response syndrome (sirs) to identify infected patients in the emergency room. Intensive Care Med 29:1368–1371

    Article  PubMed  Google Scholar 

  26. Levy MM (2003) Sccm/esicm/accp/ats/sis. 2001 sccm/esicm/accp/ats/sis. international sepsis definitions conference. Crit Care Med 31:1250–1256

    Article  PubMed  Google Scholar 

  27. Jones AE, Trzeciak S, Kline JA (2009) The sequential organ failure assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med 37(5):1649

    Article  PubMed  PubMed Central  Google Scholar 

  28. Calvert JS, Price DA, Chettipally UK, Barton CW, Feldman MD, Hoffman JL, Jay M, Das R (2016) A computational approach to early sepsis detection. Comput Biol Med 74:69–73

    Article  PubMed  Google Scholar 

  29. Desautels T, Calvert J, Hoffman J, Jay M, Kerem Y, Shieh L, Shimabukuro D, Chettipally U, Feldman MD, Barton C et al (2016) Prediction of sepsis in the intensive care unit with minimal electronic health record data: a machine learning approach. JMIR Med Inform 4(3):5909

    Article  Google Scholar 

  30. Mao Q, Jay M, Hoffman JL, Calvert J, Barton C, Shimabukuro D, Shieh L, Chettipally U, Fletcher G, Kerem Y et al (2018) Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and icu. BMJ Open 8(1):017833

    Article  Google Scholar 

  31. Yang M, Wang X, Gao H, Li Y, Liu X, Li J, Liu C (2019) Early prediction of sepsis using multi-feature fusion based xgboost learning and bayesian optimization. In: The IEEE conference on computing in cardiology (CinC), vol 46, pp 1–4

  32. Moor M, Horn M, Rieck B, Roqueiro D, Borgwardt K (2019) Early recognition of sepsis with gaussian process temporal convolutional networks and dynamic time warping. In: Machine learning for healthcare conference, pp 2–26 . PMLR

  33. Shashikumar SP, Josef C, Sharma A, Nemati S (2019) Deepaise–an end-to-end development and deployment of a recurrent neural survival model for early prediction of sepsis. arXiv preprint arXiv:1908.04759

  34. Liu R, Greenstein JL, Sarma SV, Winslow RL (2019) Natural language processing of clinical notes for improved early prediction of septic shock in the icu. In: 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 6103–6108 . IEEE

  35. Armi L, Abbasi E, Zarepour-Ahmadabadi J (2021) Texture images classification using improved local quinary pattern and mixture of elm-based experts. Neural Computing and Applications, 1–24

  36. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems. 30

  37. Nie W, Chang R, Ren M, Su Y, Liu A (2021) I-gcn: incremental graph convolution network for conversation emotion detection. IEEE Trans Multimedia 24:4471–4481

    Article  Google Scholar 

  38. Wen X, Nie W, Liu J, Su Y (2023) Mrft: Multiscale recurrent fusion transformer based prior knowledge for bit-depth enhancement. IEEE Trans Circ Syst Video Technol

  39. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  40. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

  41. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  42. Johnson AE, Pollard TJ, Shen L, L-wH Lehman, Feng M, Ghassemi M, Moody B, Szolovits P, Anthony Celi L, Mark RG (2016) Mimic-iii, a freely accessible critical care database. Scientific data 3(1):1–9

    Article  Google Scholar 

  43. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, Rubenfeld G, Kahn JM, Shankar-Hari M, Singer M et al (2016) Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 315(8):762–774

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Futoma J, Hariharan S, Heller K, Sendak M, Brajer N, Clement M, Bedoya A, O’brien C (2017) An improved multi-output gaussian process rnn with real-time validation for early sepsis detection. In: Machine learning for healthcare conference, pp 243–254 . PMLR

  45. Subbe CP, Kruger M, Rutherford P, Gemmel L (2001) Validation of a modified early warning score in medical admissions. QJM 94(10):521–526

    Article  CAS  PubMed  Google Scholar 

  46. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  47. Aşuroğlu T, Oğul H (2021) A deep learning approach for sepsis monitoring via severity score estimation. Comput Methods Programs Biomed 198:105816

    Article  PubMed  Google Scholar 

  48. Li X, Xu X, Xie F, Xu X, Sun Y, Liu X, Jia X, Kang Y, Xie L, Wang F et al (2020) A time-phased machine learning model for real-time prediction of sepsis in critical care. Crit Care Med 48(10):884–888

    Article  Google Scholar 

  49. Rosnati M, Fortuin V (2021) Mgp-atttcn: An interpretable machine learning model for the prediction of sepsis. PLoS ONE 16(5):0251248

    Article  Google Scholar 

  50. Wang Z, Yan W, Oates T (2017) Time series classification from scratch with deep neural networks: A strong baseline. In: 2017 International joint conference on neural networks (IJCNN), pp 1578–1585 . IEEE

  51. Frazier PI (2018) A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811

  52. Hongyi Li G (1996) Maddala: Bootstrapping time series models. Economet Rev 15(2):115–158

    Article  Google Scholar 

  53. Liu S, Fu B, Wang W, Liu M, Sun X (2022) Dynamic sepsis prediction for intensive care unit patients using xgboost-based model with novel time-dependent features. IEEE J Biomed Health Inform 26(8):4258–4269

    Article  PubMed  Google Scholar 

  54. Ding R, Rong F, Han X, Wang L (2023) Cross-center early sepsis recognition by medical knowledge guided collaborative learning for data-scarce hospitals. arXiv preprint arXiv:2302.05702

  55. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, Kumar A, Sevransky JE, Sprung CL, Nunnally ME et al (2017) Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med 43:304–377

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by the Foundation of State Key Laboratory of Ultrasound in Medicine and Engineering (Grant No.2022KFKT004) and Tianjin Health Science and Technology Project (Grant NO TJWJ2023ZD006).

Author information

Authors and Affiliations

  1. School of Microelectronics, Tianjin University, Tianjin, 300072, China

    Qiang Li & Hanbo Ma

  2. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    Dan Song

  3. Department of Cardiac Surgery, Chest Hospital, Tianjin University, and Clinical School of Thoracic, Tianjin Medical University, Tianjin, 300070, China

    Yunpeng Bai

  4. Department of Critical Care Medicine, Tianjin Medical University General Hospital, Tianjin, 300070, China

    Lina Zhao

  5. Department of Critical Care Medicine, Department of Anesthesiology, and Tianjin Institute of Anesthesiology, Tianjin Medical University General Hospital, Tianjin, 300070, China

    Keliang Xie

Authors
  1. Qiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanbo Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunpeng Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lina Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Keliang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dan Song.

Ethics declarations

Ethical standard

The authors state that this research complies with ethical standards. This research does not involve either human participants or animals.

Conflicts of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Ma, H., Song, D. et al. Early prediction of sepsis using chatGPT-generated summaries and structured data. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18378-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18378-7

Keywords

Search

Navigation