Computer-Based Speech Training for the Hearing Impaired

9977260 Zahorian提出的研究的目的是调查和优化方法，用于提取扬声器独立的声学为基础的语音参数，语音的语音内容的信号，并将这些参数转换为视觉显示，用作发音训练辅助听力受损。 拟议的项目是以前由NSF资助的研究的延续和延伸。 这项工作的一个主要组成部分将是收集和注释一个大型数据库的语音样本从正常听力和听力受损的听众，用于拟议的工作，也促进其他研究工作在这一领域。 将进行广泛的实验，以优化语音参数的转换，以显示参数，使音素和音素的区别性特征，无论是孤立的，音节的上下文中，或作为连续语音的一部分，可以很容易地被区分和识别的基础上，他们的显示特性。 将开发和改进自动语音识别算法，以创建可视化显示表示。 声学特征将从峰值包络谱、基频和短时能量中提取。 几个识别策略，所有使用神经网络作为一个关键的积木，将进行调查。 视觉显示将被开发和测试的几个语音训练练习，在不同的复杂程度，从独特的功能，孤立生产的电话语音显示连续讲话。 将开发一种显示模式，该模式提供了元音和辅音产生的发音特征与显示模式之间的连续关系，并将显示发音中任何所需校正的程度和方向。 此外，离散显示，如条形图或游戏，将指示生产的接近程度，以“正常”的生产。 研究的重点是提高自动语音识别的水平，算法给出的反馈是准确和一致的，这将是可能的专家经验丰富的听众。 将进行受控评估实验，以确定在何种程度上“新的”语音清晰度模式可以从各种显示器给出的反馈的基础上学习。 此外，亦会为听障儿童进行言语清晰度训练，以评估视觉显示器在改善言语表达方面的效用，并协助确定改善方法。

9977260ZahorianThe objectives of the proposed research are to investigate and optimize methods for extracting speaker-independent acoustically-based speech parameters that signal the phonetic content of speech and to convert these parameters to a visual display for use as an articulation training aid for the hearing impaired. The proposed project is a continuation and extension of research previously funded by the NSF. A major component of the effort will be to collect and annotate a large database of speech samples from both normally-hearing and hearing-impaired listeners, for use with the proposed work and also to facilitate other research efforts in this field. Extensive experiments will be conducted to optimize the conversion of speech parameters to display parameters so that phonemes and distinctive features of phonemes, produced either in isolation, syllabic contexts, or as part of continuous speech, can readily be discriminated and identified based on their display characteristics. Automatic speech recognition algorithms will be developed and refined to create a visual display representation. Acoustic features will be extracted from the peak envelope spectrum, fundamental frequency, and short-time energy. Several recognition strategies, all using neural networks as a key building block, will be investigated. Visual displays will be developed and tested for several speech training exercises at various levels of complexity-ranging from distinctive features for phones produced in isolation to phonetic displays for continuous speech. One display mode will be developed which provides a continuous relationship between articulatory features for vowel and consonant production and displayed patterns, and will show both degree and directions of any needed corrections in articulation. Additionally, discrete displays, such as bargraphs or games, will indicate the closeness of a production to a "normal" production. The emphasis of the research is to improve automatic phonetic recognition to the level that the feedback given by the algorithm is as accurate and consistent as that which would be possible by an expert experienced listener. Controlled evaluation experiments will be conducted to determine the extent to which "new" speech articulation patterns can be learned on the basis of the feedback given from the various displays. Speech articulation training exercises will also be conducted with hearing-impaired children to assess the usefulness of the visual display in improving speech production and to help determine methods to improve it.