Cross-Modal Correspondence Between Speech Sound and Visual Shape Influencing Perceptual Representation of Shape: the Role of Articulation and Pitch
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kwak, Yuna | - |
dc.contributor.author | Nam, Hosung | - |
dc.contributor.author | Kim, Hyun-Woong | - |
dc.contributor.author | Kim, Chai-Youn | - |
dc.date.accessioned | 2021-08-30T21:33:37Z | - |
dc.date.available | 2021-08-30T21:33:37Z | - |
dc.date.created | 2021-06-19 | - |
dc.date.issued | 2020-06 | - |
dc.identifier.issn | 2213-4794 | - |
dc.identifier.uri | https://scholar.korea.ac.kr/handle/2021.sw.korea/55102 | - |
dc.description.abstract | Cross-modal correspondence is the tendency to systematically map stimulus features across sensory modalities. The current study explored cross-modal correspondence between speech sound and shape (Experiment 1), and whether such association can influence shape representation (Experiment 2). For the purpose of closely examining the role of the two factors - articulation and pitch - combined in speech acoustics, we generated two sets of 25 vowel stimuli - pitch-varying and pitch-constant sets. Both sets were generated by manipulating articulation - frontness and height of the tongue body's positions - but differed in terms of whether pitch varied among the sounds within the same set. In Experiment 1, participants made a forced choice between a round and a spiky shape to indicate the shape better associated with each sound. Results showed that shape choice was modulated according to both articulation and pitch, and we therefore concluded that both factors play significant roles in sound-shape correspondence. In Experiment 2, participants reported their subjective experience of shape accompanied by vowel sounds by adjusting an ambiguous shape in the response display. We found that sound-shape correspondence exerts an effect on shape representation by modulating audiovisual interaction, but only in the case of pitch-varying sounds. Therefore, pitch information within vowel acoustics plays the leading role in sound-shape correspondence influencing shape representation. Taken together, our results suggest the importance of teasing apart the roles of articulation and pitch for understanding sound-shape correspondence. | - |
dc.language | English | - |
dc.language.iso | en | - |
dc.publisher | BRILL | - |
dc.subject | AUDITORY PITCH | - |
dc.subject | SPEEDED CLASSIFICATION | - |
dc.subject | CHANGING PITCH | - |
dc.subject | INTEGRATION | - |
dc.subject | ASSOCIATIONS | - |
dc.subject | SIMILARITY | - |
dc.subject | SYMBOLISM | - |
dc.subject | CORTEX | - |
dc.subject | BOUBA | - |
dc.subject | KIKI | - |
dc.title | Cross-Modal Correspondence Between Speech Sound and Visual Shape Influencing Perceptual Representation of Shape: the Role of Articulation and Pitch | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Nam, Hosung | - |
dc.contributor.affiliatedAuthor | Kim, Chai-Youn | - |
dc.identifier.doi | 10.1163/22134808-20191330 | - |
dc.identifier.scopusid | 2-s2.0-85091514260 | - |
dc.identifier.wosid | 000548166800001 | - |
dc.identifier.bibliographicCitation | MULTISENSORY RESEARCH, v.33, no.6, pp.569 - 598 | - |
dc.relation.isPartOf | MULTISENSORY RESEARCH | - |
dc.citation.title | MULTISENSORY RESEARCH | - |
dc.citation.volume | 33 | - |
dc.citation.number | 6 | - |
dc.citation.startPage | 569 | - |
dc.citation.endPage | 598 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | ssci | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Biophysics | - |
dc.relation.journalResearchArea | Psychology | - |
dc.relation.journalWebOfScienceCategory | Biophysics | - |
dc.relation.journalWebOfScienceCategory | Psychology | - |
dc.relation.journalWebOfScienceCategory | Psychology, Experimental | - |
dc.subject.keywordPlus | AUDITORY PITCH | - |
dc.subject.keywordPlus | SPEEDED CLASSIFICATION | - |
dc.subject.keywordPlus | CHANGING PITCH | - |
dc.subject.keywordPlus | INTEGRATION | - |
dc.subject.keywordPlus | ASSOCIATIONS | - |
dc.subject.keywordPlus | SIMILARITY | - |
dc.subject.keywordPlus | SYMBOLISM | - |
dc.subject.keywordPlus | CORTEX | - |
dc.subject.keywordPlus | BOUBA | - |
dc.subject.keywordPlus | KIKI | - |
dc.subject.keywordAuthor | Cross-modal correspondence | - |
dc.subject.keywordAuthor | bouba-kiki effect | - |
dc.subject.keywordAuthor | audiovisual interaction | - |
dc.subject.keywordAuthor | speech sound | - |
dc.subject.keywordAuthor | vowel sound | - |
dc.subject.keywordAuthor | articulation | - |
dc.subject.keywordAuthor | pitch | - |
dc.subject.keywordAuthor | visual shape | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
(02841) 서울특별시 성북구 안암로 14502-3290-1114
COPYRIGHT © 2021 Korea University. All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.