Physics-Based Tool Usage Simulations in VR

Abstract

The need for scalable, immersive training systems is universal and recently has been included in fields that rely on complex, hands-on processes, such as surgery operations, assembly operations, construction processes training, etc. This paper examines the potential to support immersive training via digital tool manipulation in the domain of traditional handicrafts. The proposed methodology employs Finite Element Method simulations to compute material transformations and apply them to interactive virtual environments. The challenge is to accurately simulate human–tool interactions, which are critical to the acquisition of manual skills. Using Simulia Abaqus (v.2023HF2), crafting simulations are authored, executed, and exported as animation sequences. These are further refined in Blender (v3.6) and integrated into Unity to create reusable training components called Action Animators. Two software applications—Craft Studio (v1.0) and Apprentice Studio (v1.0)—are designed and implemented to enable instructors to create training lessons and students to practice and get evaluated in virtual environments. The methodology has wide-ranging applications beyond crafts, offering a solution for immersive training in skill-based activities. The validation and evaluation of the proposed approach suggest that it can significantly improve training effectiveness, scalability, and accessibility across various industries.