Do the effects of acupuncture vary between acupuncturists? Analysis of the Acupuncture Trialists’ Collaboration individual patient data meta-analysis

Abstract

Objectives: The degree to which the effects of acupuncture treatment vary between acupuncturists is unknown. We used a large individual patient dataset of trials of acupuncture for chronic pain to assess practitioner heterogeneity. Methods: Individual patient data linked to identifiable acupuncturists were drawn from a dataset of 39 high-quality trials of acupuncture, where the comparators were either sham acupuncture or non-acupuncture controls, such as standard care or waitlist. Heterogeneity among acupuncturists was assessed by meta-analysis. Results: A total of 1206 acupuncturists in 13 trials were included. Statistically significant heterogeneity was found in trials with sham-control groups (p < 0.0001) and non-acupuncture control groups (p <0.0001). However, the degree of heterogeneity was very small, with the observed distribution of treatment effects virtually overlapping that expected by chance. For instance, for non-acupuncture-controlled trials, the proportion of acupuncturists with effect sizes half a standard deviation greater or less than average was expected to be 34%, but was observed to be 37%. A limitation is that the trials included a relatively limited range of acupuncturists, mainly physician-acupuncturists. Discussion: Although differences in effects between acupuncturists were greater than expected by chance, the degree of variation was small. This suggests that most chronic pain patients in clinical practice would have similar results to those reported in high-quality trials; comparably, we did not find evidence to suggest that greater standardization of acupuncture practice would improve outcomes. Further research needs to be conducted exploring variability using a sample of acupuncturists with a broader range of practice styles, training and experience.