Interpretation of Course Conceptual Structure and Student Self- efficiency: An Integrated Strategy of Knowledge Graphs with Item Response Modeling

Abstract

Abstract Background A lack of studies quantitively measures the difficulty and importance of knowledge points depending on students’ self-efficacy for learning (SEL). The study aims to verify the practical use of psychological measurement tools in physical therapy education by analyzing student SEL and course conceptual structure. Methods We extracted 100 knowledge points (KPs) from the “Therapeutic Exercise” course curriculum and administered a difficulty rating questionnaire of KPs to 218 students after their final exam. The pipeline of the non-parametric IRT and parametric IRT was employed to estimate student SEL and describe the hierarchy of KPs in terms of difficulty. Additionally, Gaussian Graphical Models with Non-Convex Penalties were deployed to create a Knowledge Graph (KG) and identify the main components. Finally, a visual analytics approach was proposed to understand the correlation and difficulty level of KPs. Results We identified 50 KPs to create the Mokken scale, which exhibited high reliability (Cronbach’s alpha = 0.9675) and showed no gender bias at the overall or each item level (p > 0.05). The three-parameter logistic model (3PLM) demonstrated good fitness with questionnaire data, whose Root Mean Square Error Approximation < 0.05. Besides, item-model fitness unveiled good fitness, as indicated by each item with non-significant p-values for chi-square tests (p > 0.05). The Wright map revealed item difficulty relative to SEL levels. SEL estimated by the 3PLM correlated significantly with the high-ability range of average Grade-Point Average (p < 0.05). The KG backbone structure consisted of 58 KPs, with 29 KPs overlapping with the Mokken scale. Visual analysis of the KG backbone structure indicated that discrimination of knowledge concepts in the IRT could not replace their position parameters in the KG, suggesting KG and IRT methods offer distinct perspectives to visualize correlations and hierarchical relationships among the KPs. Conclusion This study integrated IRT modeling and the KG method through a questionnaire on student self-perceived knowledge difficulty, quantitatively assessing student SEL and the importance of KPs. Based on real-world teaching empirical data, this study laid a research foundation for updating course contents and customizing learning objectives. Trial registration Not applicable.