Patient-reported inpatient experiences and natural language processing: Unsupervised topic modeling to identify care experiences in free-text comments (Preprint)

Abstract

Patient-reported experience surveys allow administrators, clinicians, and researchers to quantify and improve healthcare by receiving feedback directly from patients. Existing research has focused primarily on quantitative analysis of survey items, but these measures may collect optional free-text comments. These comments can provide insights for health systems but may not be analyzed due to limited resources and the complexity of traditional textual analysis. However, advances in machine learning-based natural language processing (NLP) provide opportunities to learn from this traditionally underutilized data source.

To apply natural language processing to model topics found in free-text comments of patient-reported experience surveys.

CAHPS-derived patient experience surveys were collected and linked to administrative inpatient records by the provincial health services organization responsible for inpatient care. Unsupervised topic modeling with automated labeling was performed with BERTopic in Python. Sentiment analysis was performed using {sentimentr} in R to assist in topic description.

Between April 2016 and February 2020, 43.4% (n = 43,522) adult patients and 46.9% (n = 3,501) pediatric caregivers completed patient experience surveys that included free-text responses. Topic models identified 86 topics among adult survey responses and 35 topics among pediatric responses that included elements of care not currently surveyed by existing questionnaires. Frequent topics were generally positive.

We found that with limited tuning, BERTopic identified care experience topics with interpretable automated labeling. Results are discussed in the context of person-centered care, patient safety, and healthcare quality improvement. Further, we note the opportunity for the identification of temporal and site-specific trends as a method to identify patient care and safety concerns. As the use of patient experience measurement increases in healthcare, we discuss how machine learning can be leveraged to provide additional insight on patient experiences.