Communicating pitch in clear speech: articulation, acoustics, intelligibility, neuro-processing, and computational modeling

点击翻译按钮获取中文摘要

Speech communication involves multiple styles as a function of different speaking environments and communicative needs. In noisy environments, or interacting with hearing-impaired or non-native perceivers, speakers often alter their speech productions using a clarified speech style, resulting in visual articulatory and acoustic changes. Such modifications may enhance speech intelligibility as perceivers make use of clear-speech cues from speaker face and voice. However, excessively exaggerated clear speech resulting in overlap of different sound categories, or attention to incorrect speech cues, may inhibit intelligibility.This research intends to investigate the role of clear speech in communicating pitch-related information: lexical tone. The objectives are to identify how speakers modify their tone production while still maintaining tone category distinctions, and how perceivers utilize tonal enhancement and categorical cues from different forms of input in challenging listening conditions. These questions will be addressed in a series of inter-related studies examining articulation, acoustics, intelligibility, and neuro-processing of clear-speech tones. Audio-video recordings will be made with native Mandarin Chinese speakers producing Mandarin tones in clear and conversational speech. Videos of speakers' facial articulatory movements in clear and conversational tone productions will be examined using innovative computer image processing techniques. Acoustic correlates of tone productions will be analyzed through computerized acoustic measurements. Auditory-visual intelligibility data will be collected in perceptually challenging conditions: cafeteria background noise and a non-native language setting. Neural processing of clear-speech tones will be examined through electro-physiological studies. Finally, articulatory, acoustic, perceptual, and neural data will be related in statistical models to determine which articulatory and acoustic clear-speech cues contribute to enhanced intelligibility and neural sensitivity of lexical tones. This research has significant implications for the fields of speech science, communication, neuroscience, computer speech technology and speech engineering. Findings of how perceivers balance tonal enhancement cues and categorical tone distinctions provide insight into how the interaction of sensory-motor and cognitive processing is orchestrated by both lower- and higher-level brain mechanisms. On a practical front, this research will inform how speech enhancement principles can be effectively applied in the development of speech technologies in different communicative contexts, e.g., speech learning and teaching, speech therapy, and human-computer interface.