RI: Small: Ensemble Modeling of Speech Signals for Automatic Speech Recognition

该项目旨在开发语音信号集成建模的新框架，以解决长期存在的对自发语音的鲁棒性和准确性识别的挑战。为此，采用基于随机森林的语音聚类方法，通过随机抽取支持语音变化的变量来构建语音集成模型；利用数据采样平衡集内数据充分性和集间数据多样性，丰富了集成模型的多样性；利用功能判别训练进一步优化集成模型的效率和准确性。这些方法的实验评估是在一个标准的语音识别任务上进行的，以促进语音研究界对其有效性的直接评估。由于集成模型在解码搜索的局部步骤中提高了似然分数，因此集成建模方法比目前的多系统集成方法具有更高的精度性能和更低的计算成本。本项目所倡导的方法为研究语音声学建模中的许多问题开辟了一个新的范例，它为结构化数据的集成建模提供了一种新的方法，因此它具有显著影响语音识别和其他机器学习应用领域的潜力。研究成果通过期刊出版物、会议报告和网站传播。本课题所提出的方法在语音识别和结构化数据分类等领域具有广泛的应用前景，特别是用于提高远程医疗自动字幕系统的准确率。

This project is aimed at developing a new framework of ensemble modeling of speech signals to address the long standing challenge of robust and accurate recognition of spontaneous speech. Toward this goal, random forests based allophonic clustering is used to construct ensemble models of allophones by random sampling the variables underpinning the allophonic variations; data sampling is used to enrich the diversity of the ensemble models by balancing within-set data sufficiency and between-sets data diversity; functional discriminative training is used to further optimize the efficiency and accuracy of the ensemble models. Experimental evaluations of these methods are performed on a standard speech recognition task to facilitate direct assessments of their efficacy by the speech research community. The ensemble modeling approach promises higher accuracy performance and lower computation costs than the current multiple system integration approach, owing to the improved likelihood scores contributed by the ensemble models in local steps of decoding search. The approach as advocated in this project opens up a new paradigm for investigating the many issues in speech acoustic modeling, it offers a new way for ensemble modeling of structured data generally, and therefore it has the potential of significantly impacting the fields of speech recognition and other machine learning applications. The research findings are disseminated via journal publication, conference presentation, and a website. The methods of this project have broad applications in speech recognition and structured data classification, and particularly they are employed to improve the accuracy performance of a telemedicine automatic captioning system.