Computer-Based Speech Training for the Hearing Impaired

9977260zahorian本研究的目的是研究和优化提取与说话人无关的基于声学的语音参数的方法，这些参数表示语音的语音内容，并将这些参数转换为视觉显示，作为听力障碍人士的发音训练辅助工具。拟议的项目是先前由美国国家科学基金会资助的研究的延续和扩展。这项工作的一个主要组成部分将是收集和注释来自听力正常和听力受损听众的语音样本的大型数据库，用于拟议的工作，并促进该领域的其他研究工作。将进行大量的实验来优化语音参数到显示参数的转换，使音素和音素的独特特征，无论是在孤立的、音节上下文中产生的，还是作为连续语音的一部分，都可以根据其显示特征很容易地进行区分和识别。将开发和改进自动语音识别算法，以创建视觉显示表示。声学特征将从峰值包络谱、基频和短时能量中提取出来。几个识别策略，所有使用神经网络作为一个关键的构建块，将被调查。视觉显示将被开发和测试，用于不同复杂程度的几种语音训练练习——从孤立生产的手机的独特特征到连续语音的语音显示。将开发一种显示模式，提供元音和辅音发音特征与显示模式之间的连续关系，并将显示发音中任何需要纠正的程度和方向。此外，离散的显示，如条形图或游戏，将表明一个产品接近“正常”的生产。该研究的重点是提高自动语音识别的水平，使算法给出的反馈与经验丰富的专家听众所提供的反馈一样准确和一致。将进行受控评估实验，以确定在各种显示器给出的反馈的基础上学习“新”语音发音模式的程度。此外，本署亦会为听障儿童进行发音训练，以评估视觉显示在改善发音方面的作用，并协助确定改善发音的方法。

9977260ZahorianThe objectives of the proposed research are to investigate and optimize methods for extracting speaker-independent acoustically-based speech parameters that signal the phonetic content of speech and to convert these parameters to a visual display for use as an articulation training aid for the hearing impaired. The proposed project is a continuation and extension of research previously funded by the NSF. A major component of the effort will be to collect and annotate a large database of speech samples from both normally-hearing and hearing-impaired listeners, for use with the proposed work and also to facilitate other research efforts in this field. Extensive experiments will be conducted to optimize the conversion of speech parameters to display parameters so that phonemes and distinctive features of phonemes, produced either in isolation, syllabic contexts, or as part of continuous speech, can readily be discriminated and identified based on their display characteristics. Automatic speech recognition algorithms will be developed and refined to create a visual display representation. Acoustic features will be extracted from the peak envelope spectrum, fundamental frequency, and short-time energy. Several recognition strategies, all using neural networks as a key building block, will be investigated. Visual displays will be developed and tested for several speech training exercises at various levels of complexity-ranging from distinctive features for phones produced in isolation to phonetic displays for continuous speech. One display mode will be developed which provides a continuous relationship between articulatory features for vowel and consonant production and displayed patterns, and will show both degree and directions of any needed corrections in articulation. Additionally, discrete displays, such as bargraphs or games, will indicate the closeness of a production to a "normal" production. The emphasis of the research is to improve automatic phonetic recognition to the level that the feedback given by the algorithm is as accurate and consistent as that which would be possible by an expert experienced listener. Controlled evaluation experiments will be conducted to determine the extent to which "new" speech articulation patterns can be learned on the basis of the feedback given from the various displays. Speech articulation training exercises will also be conducted with hearing-impaired children to assess the usefulness of the visual display in improving speech production and to help determine methods to improve it.