Audiovisual Description and Recognition of Dysarthric Speech

点击翻译按钮获取中文摘要

DESCRIPTION (provided by applicant): The primary aim of this proposal is to describe and automatically recognize the audible and visible correlates of phonological contrasts as produced by persons with spastic dysarthria. In order to meet the primary aim, this investigation will: - enroll a total of 16 subjects with dysarthria and 16 control subjects, - record each subject's production of phonetically balanced and pragmatically useful speech material using our AVICAR array of eight microphones and four video cameras, - measure the acoustic and visible correlates of consonant place of articulation, including formant locus, frication spectrum, lip aperture area, and jaw height, - develop automatic audio-only and audiovisual isolated word recognition algorithms, and - record each subject's participation in an objective comparison of audiovisual speech recognition, audio-only speech recognition, and typing as text input methods for human computer interface. LAY LANGUAGE SUMMARY: Subjects whose neuromotor deficit precludes or hinders their use of a keyboard may nevertheless retain some control over speech articulators. Our preliminary data demonstrate that subjects with 19-30% intelligibility (as rated by human listeners) may nevertheless achieve 90-100% recognition accuracy in an automatic isolated digit recognition task. We have found that the use of video in automatic speech recognition improves word recognition accuracy for talkers without dysarthria; we propose to extend our work to seek the same gains for talkers with dysarthria.