Real and Artificial Differential Item Functioning

Abstract

The literature in modern test theory on procedures for identifying items with differential item functioning (DIF) among two groups of persons includes the Mantel–Haenszel (MH) procedure. Generally, it is not recognized explicitly that if there is real DIF in some items which favor one group, then as an artifact of this procedure, artificial DIF that favors the other group is induced in the other items. Using the Rasch model for dichotomous responses as a theoretical basis, this article proves that the source of artificial DIF in the MH procedure is that estimates of the person locations are substituted for their unknown values. The article then demonstrates that the formalization of artificial DIF implies mathematically (a) a particular sequential, iterative procedure for detecting items with real DIF and for identifying a set of items that may have no DIF and (b) a resolution of the items with real DIF for quantifying the DIF on the same metric as the items showing no DIF and provides expected value curves and tests of fit for the item for each of the groups. Finally, because the source of artificial DIF in the MH procedure is the substitution of a parameter with its estimate, it is suggested that all procedures that use the substitution of an estimate for a parameter are likely to induce artificial DIF.