Cross-Modal Correspondence The influence of brightness on pitch perception

Abstract

Cross-modal correspondence is a perception mechanism based on stimulus compatibility. In this study we explore the cross-modal correspondence between brightness and pitch. Previous research has suggested that we are more likely to choose bright colors in congruence with higher pitch and dark colors in congruence with lower pitch. In accordance with previous research our hypothesis was that brighter visual stimuli corresponds to higher pitch. We conducted two experiments to test this hypothesis. 12 observers participated in the experiments, five in Experiment 1 and seven in Experiment 2. The observers were presented with a visual stimulus combined with an auditory one. The visual stimulus was a screen showing different brightness values and the auditory stimulus were different pure tone values. This was followed by a fixation cross before showing a second stimulus combination. The task was to indicate whether the first or second tone had a higher frequency. In Experiment 1 the maximum likelihood conjoint measurement (MLCM) was used to measure how two physical dimensions (brightness and pitch) can affect perception (pitch perception). The results from Experiment 1 did not support the hypothesis but did indicate that extreme brightness values would potentially have more influence on pitch perception. In Experiment 2 the brightness values were only two, black and white, and psychometric functions were used to analyze the data. Results from Experiment 2 did support the hypothesis where most observers perceived a tone to be higher if paired with a white visual stimulus. Our results suggest that extreme values of stimulus intensity are more suitable for the evaluation of cross-modal correspondence on brightness and pitch.