Facial Affect Recognition in Immersive Virtual Reality: Where is the Participant Looking?

Abstract

The recognition of facial expression of emotions in others is essential in daily social interactions. The different areas of the face play different roles in decoding each emotion. To find out which ones are more important, the traditional approach has been to use eye-tracking devices with static pictures to capture which parts of the face people are looking at when decoding emotions. However, the ecological validity of this approach is limited because, unlike in real life, there is no movement in the face that can be used to identify the emotion. The use of virtual reality technology opens the door to new experiences in which the users perceive that they are in front of dynamic virtual humans. Therefore, our main aim is to investigate whether the user’s immersion in a virtual environment influences the way dynamic virtual faces are visually scanned when decoding emotions. An experiment involving 74 healthy participants was carried out. The results obtained are consistent with previous works. Having confirmed the correct functioning of our solution, it is our intention to study whether emotion recognition deficits in patients with neuropsychiatric disorders are related to the way they visually scan faces.