Hawthorne Control Procedures in Educational Experiments: A Reconsideration of Their Use and Effectiveness

Abstract

This article reports on a descriptive analysis of research practices and a meta-analysis of effect sizes associated with control groups employed to address Hawthorne effects in educational experiments. The descriptive analysis of 86 studies revealed a diversity of practices designed to control one of three artifact variables: special attention, activity related to the experimental task, or awareness of participation in an experiment. The meta-analysis provided no evidence for a Hawthorne versus no-treatment control difference. Moreover, a detailed analysis of these studies by their control procedure, and subsequently by other moderator variables, also revealed no systematic trends to suggest a specific artifact source. A within-study analysis of the pattern of treatment/Hawthorne/control group effect sizes suggested that the artifact controls were of limited utility. Increased research is urged into other artifacts and alternative control procedures that the educational researcher should consider.