Audiovisual Description and Recognition of Dysarthric Speech

描述（由申请人提供）：本提案的主要目的是描述和自动识别由痉挛性构音障碍患者产生的语音对比的可听和可见相关物。为了实现主要目标，本研究将：- 招募总共16名患有构音障碍的受试者和16名对照受试者，-使用我们的八个麦克风和四个摄像机的AVICAR阵列记录每个受试者的语音平衡和实用有用的语音材料的产生，-测量发音的辅音位置的声学和可见相关性，包括共振峰轨迹、摩擦频谱，唇孔面积和下颌高度，-开发自动的仅音频和视听孤立词识别算法，以及-记录每个受试者参与视听语音识别、仅音频语音识别和打字作为人机接口的文本输入方法的客观比较。非专业语言摘要：受试者的神经运动缺陷排除或阻碍他们使用键盘，但可能保留一些控制语音发音。我们的初步数据表明，受试者与19-30%的可懂度（由人类听众评定），但可以达到90-100%的识别准确率在自动孤立的数字识别任务。我们已经发现，在自动语音识别中使用视频提高了没有构音障碍的说话者的单词识别准确性;我们建议扩展我们的工作，为构音障碍的说话者寻求同样的收益。

DESCRIPTION (provided by applicant): The primary aim of this proposal is to describe and automatically recognize the audible and visible correlates of phonological contrasts as produced by persons with spastic dysarthria. In order to meet the primary aim, this investigation will: - enroll a total of 16 subjects with dysarthria and 16 control subjects, - record each subject's production of phonetically balanced and pragmatically useful speech material using our AVICAR array of eight microphones and four video cameras, - measure the acoustic and visible correlates of consonant place of articulation, including formant locus, frication spectrum, lip aperture area, and jaw height, - develop automatic audio-only and audiovisual isolated word recognition algorithms, and - record each subject's participation in an objective comparison of audiovisual speech recognition, audio-only speech recognition, and typing as text input methods for human computer interface. LAY LANGUAGE SUMMARY: Subjects whose neuromotor deficit precludes or hinders their use of a keyboard may nevertheless retain some control over speech articulators. Our preliminary data demonstrate that subjects with 19-30% intelligibility (as rated by human listeners) may nevertheless achieve 90-100% recognition accuracy in an automatic isolated digit recognition task. We have found that the use of video in automatic speech recognition improves word recognition accuracy for talkers without dysarthria; we propose to extend our work to seek the same gains for talkers with dysarthria.