Emotional prosody modulates visual mental imagery

Abstract

AbstractPerceptual stimuli’s emotional properties are vital for human evolution and adaptation. While visual imagery is predominantly regarded as a weak form of perception, the influence of cross-modal emotional features on imagery is still unknown. The present study aims to investigate how emotional prosody modulates imagery quality (i.e., accuracy and clarity) and neural mechanisms using a combination of behavioral tasks and functional magnetic resonance imaging (fMRI). At the behavioral level, our results showed that frustrated conditions induced significant prosody effects on visual mental imagery quality measures, and the effects were particularly pronounced in individuals with lower imagery use tendency. At the neural level, compared with the neutral condition, the emotional prosody conditions (both happy and frustrated) showed stronger activation in various regions including the middle occipital gyrus, supporting the critical role of primary visual system in imagery. Moreover, compared to the frustrated prosody condition, the happy prosody showed stronger activation in the precuneus and anterior cingulate cortex, which are core components of the default mode network. A machine learning prediction analysis with a random forest model identified a significant brain-behavioral correlation between prosody-linked neural activity and individual imagery use tendency. A subsequent Shapley Additive exPlanations (SHAP) analysis further highlighted the primary visual and default mode regions as top contributors to this prediction. Taken together, these results provide new insights for the understanding of how emotional prosody modulates visual mental imagery, considering individual differences, and provide compelling evidence for incorporating emotion as important shaping factor in more general model for imagery.