Audiovisual Description and Recognition of Dysarthric Speech

描述（由申请人提供）：本提案的主要目的是描述和自动识别痉挛构音障碍患者产生的语音对比的可听和可见相关性。为了达到主要目的，本调查将：-总共招募16名患有音障碍的受试者和16名对照受试者，-使用我们的AVICAR 8个麦克风和4个摄像机阵列记录每个受试者的语音平衡和实用有用的语音材料，-测量辅音发音位置的声学和可见相关性，包括形成峰轨迹，摩擦谱，唇孔面积和下巴高度，-开发自动音频和视听孤立词识别算法。并记录每个受试者参与视听语音识别、纯音频语音识别和打字作为人机界面文本输入法的客观比较。语言概述：神经运动缺陷排除或阻碍了他们使用键盘的受试者仍然可以保留对语音发音的一些控制。我们的初步数据表明，具有19-30%可理解度（由人类听众评定）的受试者在自动隔离数字识别任务中可能达到90-100%的识别准确率。我们发现，在自动语音识别中使用视频可以提高没有构音障碍的说话者的单词识别准确性；我们建议扩展我们的工作，为患有构音障碍的说话者寻求同样的收益。

DESCRIPTION (provided by applicant): The primary aim of this proposal is to describe and automatically recognize the audible and visible correlates of phonological contrasts as produced by persons with spastic dysarthria. In order to meet the primary aim, this investigation will: - enroll a total of 16 subjects with dysarthria and 16 control subjects, - record each subject's production of phonetically balanced and pragmatically useful speech material using our AVICAR array of eight microphones and four video cameras, - measure the acoustic and visible correlates of consonant place of articulation, including formant locus, frication spectrum, lip aperture area, and jaw height, - develop automatic audio-only and audiovisual isolated word recognition algorithms, and - record each subject's participation in an objective comparison of audiovisual speech recognition, audio-only speech recognition, and typing as text input methods for human computer interface. LAY LANGUAGE SUMMARY: Subjects whose neuromotor deficit precludes or hinders their use of a keyboard may nevertheless retain some control over speech articulators. Our preliminary data demonstrate that subjects with 19-30% intelligibility (as rated by human listeners) may nevertheless achieve 90-100% recognition accuracy in an automatic isolated digit recognition task. We have found that the use of video in automatic speech recognition improves word recognition accuracy for talkers without dysarthria; we propose to extend our work to seek the same gains for talkers with dysarthria.