Affiliation:
1. GAZI UNIVERSITY, GAZİ FACULTY OF EDUCATION
2. University of Alberta
Abstract
The purpose of this study is to generate non-verbal items for a visual reasoning test using templated-based automatic item generation (AIG). The fundamental research method involved following the three stages of template-based AIG. An item from the 2016 4th-grade entrance exam of the Science and Art Center (known as BİLSEM) was chosen as the parent item. A cognitive model and an item model were developed for non-verbal reasoning. Then, the items were generated using computer algorithms. For the first item model, 112 items were generated, and for the second item model, 1728 items were produced. The items were evaluated based on subject matter experts (SMEs). The SMEs indicated that the items met the criteria of one right answer, single content and behavior, not trivial content, and homogeneous choices. Additionally, SMEs' opinions determined that the items have varying item difficulty. The results obtained demonstrate the feasibility of AIG for creating an extensive item repository consisting of non-verbal visual reasoning items.
Publisher
International Journal of Assessment Tools in Education
Reference52 articles.
1. Adji, T.B., Pribadi, F.S., Prabowo, H.E., Rosnawati, R., & Wijaya, A. (2018). Generating parallel mathematic items using automatic item generation. ICEAP 2019, 1(1), 89-93. https://doi.org/10.26499/iceap.v1i1.78
2. Arendasy, M.E., & Sommer, M. (2012). Using automatic item generation to meet the increasing item demands of high-stakes educational and occupational assessment. Learning and individual differences, 22(1), 112-117. https://doi.org/10.1016/j.lindif.2011.11.005
3. Atli, S. (2007). Matematiksel mantıksal yetenek ile ritimsel yetenek arasındaki ilişkiler [Relations between matematical-logical talent and rhythmic intelligence] [Unpublished master's thesis]. Gazi University.
4. Balboni, G., Naglieri, J.A., & Cubelli, R. (2010). Concurrent and predictive validity of the raven progressive matrices and the Naglieri Nonverbal Ability Test. Journal of Psychoeducational Assessment, 28(3), 222 235. https://doi.org/10.1177/073428290934376
5. Bildiren, A., Bıkmaz Bilgen, Ö., & Korkmaz, M. (2021). National non-verbal cognitive ability test (BNV) development study. SAGE Open, 11(3). https://doi.org/10.1177/2158244021104694