Sight-over-sound effect depends on interaction between evaluators’ musical experience and auditory-visual integration: An examination using Japanese brass band competition recordings

Abstract

The sight-over-sound effect, in which visual information dominates auditory cues in musical evaluations, challenges the common belief that sound is the primary factor in music evaluation. However, the replicability of the sight-over-sound effect remains controversial across different musical genres and contexts. Here, we investigated the sight-over-sound effect using recordings from Japanese brass band competitions with carefully controlled musical pieces and camera angles. Adult participants (age: 34.81 ± 11.71 years [mean ± standard deviation]) were divided into three groups based on their brass band and musical experience. Of the 301 participants, 171 were brass band musicians (age: 33.08 ± 11.57 years), 78 were non-brass band musicians (age: 35.39 ± 13.03 years), and 52 were non-musicians (age: 39.15 ± 8.73 years). The sight-over-sound effect was observed in non-brass band musicians (Kruskal–Wallis test: p < 0.001) but was absent in brass band musicians (p = 0.48) and non-musicians (p = 0.37). These findings indicate that the sight-over-sound effect depends on musical experience; specifically, our results indicate that auditory training in brass band musicians may mitigate the visual influence. Furthermore, the absence of the sight-over-sound effect in non-musicians suggests that without specific musical training, evaluators may not consistently prioritize visual information. The present findings fill a critical gap in our understanding of multisensory integration, especially regarding how different levels of musical expertise shape evaluative processes across sensory modalities. Our results underscore the need for educational and evaluative practices to consciously balance the influences of visual and auditory cues, particularly in situations in which visual dominance may overshadow auditory quality.