Generalizability Analysis for Performance Assessments of Student Achievement or School Effectiveness

Abstract

Evidence of the uncertainty attached to school and individual scores is required to avoid over interpretation of results. Generalizability analysis provides in the standard error a suitable indicator of uncertainty. Assessments depart from traditional measurements in ways that require extensions and re interpretatons of generalizability analysis. The authors find, for example, that in many analyses the variance component interpreted as Pupil x Task interaction actually arises in part from instability in pupil performance. It is necessary to recognize in the school-level analysis that pupils are nested in classes and schools. Whether to treat the population of a school's pupils as infinite or as limited to the student body assessed also requires careful consideration.