Audiovisual Description and Recognition of Dysarthric Speech

描述(由申请人提供)：本建议的主要目的是描述和自动识别痉挛构音障碍患者产生的语音对比的听觉和视觉相关性。为了达到主要目标，这项调查将：-登记16名构音障碍受试者和16名对照受试者；-使用我们的AVICAR由8个麦克风和4个摄像机组成的阵列记录每个受试者产生的语音平衡和语用有用的语音材料；-测量发音辅音位置的声学和视觉相关性，包括共振峰轨迹、摩擦光谱、唇孔面积和下巴高度；-开发自动纯音频和视听孤立词识别算法；以及-记录每个受试者参与视听语音识别、纯音频语音识别和打字作为人机界面文本输入方法的客观比较的情况。外行语言总结：如果受试者的神经运动缺陷妨碍或阻碍了他们使用键盘，他们仍然可以保持对语音发音的一些控制。我们的初步数据表明，在自动孤立数字识别任务中，具有19%-30%可理解度的受试者(根据人类听者的评分)仍然可以达到90%-100%的识别准确率。我们发现，在自动语音识别中使用视频可以提高没有构音障碍的说话者的单词识别准确率；我们建议将我们的工作扩展到为有构音障碍的说话者寻求同样的收益。

DESCRIPTION (provided by applicant): The primary aim of this proposal is to describe and automatically recognize the audible and visible correlates of phonological contrasts as produced by persons with spastic dysarthria. In order to meet the primary aim, this investigation will: - enroll a total of 16 subjects with dysarthria and 16 control subjects, - record each subject's production of phonetically balanced and pragmatically useful speech material using our AVICAR array of eight microphones and four video cameras, - measure the acoustic and visible correlates of consonant place of articulation, including formant locus, frication spectrum, lip aperture area, and jaw height, - develop automatic audio-only and audiovisual isolated word recognition algorithms, and - record each subject's participation in an objective comparison of audiovisual speech recognition, audio-only speech recognition, and typing as text input methods for human computer interface. LAY LANGUAGE SUMMARY: Subjects whose neuromotor deficit precludes or hinders their use of a keyboard may nevertheless retain some control over speech articulators. Our preliminary data demonstrate that subjects with 19-30% intelligibility (as rated by human listeners) may nevertheless achieve 90-100% recognition accuracy in an automatic isolated digit recognition task. We have found that the use of video in automatic speech recognition improves word recognition accuracy for talkers without dysarthria; we propose to extend our work to seek the same gains for talkers with dysarthria.