Affiliation:
1. Arizona State University 1050 S Forest Mall Tempe AZ 85281 USA
2. Duolingo 5900 Penn Ave. Pittsburgh PA 15206 USA
3. School of Education, Indiana University Bloomington 201 N Rose Ave Bloomington IN 47405 USA
4. Departments of Psychology and Statistics, University of Illinois Urbana‐Champaign 603 E Daniel St. Champaign IL 61820 USA
Abstract
AbstractThe recent surge of machine learning (ML) has impacted many disciplines, including educational and psychological measurement (hereafter shortened as measurement). The measurement literature has seen rapid growth in applications of ML to solve measurement problems. However, as we emphasize in this article, it is imperative to critically examine the potential risks associated with involving ML in measurement. The MxML project aims to explore the relationship between measurement and ML, so as to identify and address the risks and better harness the power of ML to serve measurement missions. This paper describes the first study of the MxML project, in which we summarize the state of the field of applications, extensions, and discussions about ML in measurement contexts with a systematic review of the recent 10 years’ literature. We provide a snapshot of the literature in (1) areas of measurement where ML is discussed, (2) types of articles (e.g., applications, conceptual, etc.), (3) ML methods discussed, and (4) potential risks associated with involving ML in measurement, which result from the differences between what measurement tasks need versus what ML techniques can provide.
Reference70 articles.
1. A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective;Ackerman T. A.;Journal of Educational Measurement,1992
2. Using big data and machine learning in personality measurement: Opportunities and challenges;Alexander L.;European Journal of Personality,2020
3. American Educational Researchers Association (AERA) American Psychological Association (APA) & National Council for Measurement in Education (NCME). (2014).The standards for educational and psychological testing.https://www.testingstandards.net/open‐access‐files.html
4. Evaluating Content‐Related Validity Evidence Using a Text‐Based Machine Learning Procedure
5. Random forests;Breiman L.;Machine learning,2001