Knowledge Of Problem Solving (KOPS) scale: Design and evaluation of a scalable competence measure for a common practice element in task-shared youth mental health interventions

Abstract

Abstract Background There is a dearth of reliable and feasible quality assurance methods to support scaling up of psychological interventions delivered by non-specialist providers. This paper reports on the phased development and validation of a digitally-administered measure (Knowledge of Problem Solving; KOPS scale) to assess competencies associated with a “task-shared” problem-solving intervention for adolescents with diverse mental health problems in India. Methods Phase 1 established key competencies required to deliver the intervention, followed by item generation for a corresponding knowledge-based competency measure that could be administered efficiently through e-learning systems. In Phase 2, items were refined based on responses from an "experienced" reference sample comprising 17 existing counsellors with direct experience of the problem-solving intervention, and a “novice” sample with 14 untrained university students and NGO staff. In Phase 3, we evaluated two parallel versions of the measure in a validation sample (N = 277) drawn from universities and NGOs. Results The resulting 17-item measure was structured around a hypothetical case, followed by multiple-choice questions that asked about the most appropriate response to a practice-based scenario. The difficulty level of the test items was well matched to the ability level of participants (i.e., most items being of moderate difficulty and few items being easy or difficult). Only one item showed a negative discrimination index and was removed from the 17-item forms. Conclusion The final 16-item version of the KOPS measure provides a scalable digital method to assess key psychotherapeutic competencies among non-specialists, particularly in relation to a transdiagnostic problem-solving intervention. Similar formats could be deployed more widely alongside e-learning programmes to expand the global workforce capable of delivering evidence-based psychological interventions.