Assessing Nursing Homes Quality Indicators’ between-Provider Variability and Reliability: A Cross-Sectional Study Using ICCs and Rankability

Abstract

Nursing home quality indicators are often used to publicly report the quality of nursing home care. In Switzerland, six national nursing home quality indicators covering four clinical domains (polypharmacy, pain, use of physical restraints and weight loss) were recently developed. To allow for meaningful comparisons, these indicators must reliably show differences in quality of care levels between nursing homes. This study’s objectives were to assess nursing home quality indicators’ between-provider variability and reliability using intraclass correlations and rankability. This approach has not yet been used in long-term care contexts but presents methodological advantages. This cross-sectional multicenter study uses data of 11,412 residents from a convenience sample of 152 Swiss nursing homes. After calculating intraclass correlation 1 (ICC1) and rankability, we describe between-provider variability for each quality indicator using empirical Bayes estimate-based caterpillar plots. To assess reliability, we used intraclass correlation 2 (ICC2). Overall, ICC1 values were high, ranging from 0.068 (95% confidence interval (CI) 0.047–0.086) for polypharmacy to 0.396 (95% CI 0.297–0.474) for physical restraints, with quality indicator caterpillar plots showing sufficient between-provider variability. However, testing for rankability produced mixed results, with low figures for two indicators (0.144 for polypharmacy; 0.471 for self-reported pain) and moderate to high figures for the four others (from 0.692 for observed pain to 0.976 for physical restraints). High ICC2 figures, ranging from 0.896 (95% CI 0.852–0.917) (self-reported pain) to 0.990 (95% CI 0.985–0.993) (physical restraints), indicated good reliability for all six quality indicators. Intraclass correlations and rankability can be used to assess nursing home quality indicators’ between-provider variability and reliability. The six selected quality indicators reliably distinguish care differences between nursing homes and can be recommended for use, although the variability of two—polypharmacy and self-reported pain—is substantially chance-driven, limiting their utility.