Congruency of color–sound crossmodal correspondence interacts with color and sound discrimination depending on color category

Abstract

People occasionally associate color (e.g., hue) with sound (e.g., pitch). Previous studies have reported color–sound associations, which are examples of crossmodal correspondences. However, the association between both semantic and perceptual factors with color/sound discrimination in crossmodal correspondence remains unclear. To clarify this, three psychological experiments were conducted, where Stroop tasks were used to assess automatic process on the association. We focused on the crossmodal correspondence between color (Experiment 1)/color word (Experiment 2) and sound. Participants discriminated the color/word or the sound presented simultaneously. The results showed the color–sound bidirectional enhancement/interference of the response by certain associations of the crossmodal correspondence (blue-drop and yellow-shiny) in both experiments. These results suggest that these Stroop effects were caused by the semantic factor (color category) and the perceptual factor (color appearance) was not necessary for the current results. In Experiment 3, response modulation by color labeling was investigated to clarify the influence of subjective labeling. Participants labeled a presented ambiguous color, which was a hue specification between two specific colors, by listening to the sound. The results revealed that the Stroop effect was caused only when the presented color was classified as the color related to the presented sound. This showed that subjective labeling played a role in the regulation of the effect of crossmodal correspondences. These findings should contribute to the explanation of crossmodal correspondences through semantic mediation.