CRII: RI: Towards Human-Level Assessment of Speech Quality and Intelligibility in Real-World Environments

CRII：RI：实现现实环境中语音质量和清晰度的人类水平评估

• 批准号: 1755844
• 负责人: Donald Williamson
• 金额: $ 17.5万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-06-01 至 2021-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1755844&HistoricalAwards=false
• 关键词: CRII; RI; Towards; Human; Level

Separating speech from background noise is crucial for many speech-based applications, including hearing prostheses, robotics, and multimedia communication. Many speech separation algorithms perform reasonably well when they are tested in simulated environments, but this level of performance does not always carry over to real environments that are more nuanced. For example, a common complaint of many hearing aid users is that their hearing aid is not effective in noisy environments such as restaurants. Current computational measures do not enable practical or convenient speech assessment in everyday environments, and this is a major hurdle for improving real-world separation performance. In addition, the end-user has largely been left out of the development and evaluation process, which is not ideal since an approach's usefulness is ultimately determined by people. The objective of this project is to develop computational evaluation algorithms to better assess speech quality and intelligibility in real environments. A key area of research focuses on developing novel, data-driven assessment algorithms that use deep learning to predict human assessment scores, which enables testing in real environments. Considering the recent success that deep learning has had in speech processing, this new assessment approach is promising and offers substantial differences from prior approaches. The relationship between spectral-temporal speech attributes and human assessment scores are determined as a result of this project. Quantifying this relationship ensures that assessment algorithms are accurate and have strong agreement with human evaluations. An effective integration of human assessment in speech separation algorithm development should result in improved separation algorithms, which ultimately benefits users and applications. This is expected since accurate assessment enables researchers to more easily identify and correct weaknesses based on real-world environmental factors. The research activities lay the foundation for the emerging research area of improving realism in speech processing applications and offer key insights on human perception to the larger scientific community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从背景噪声中分离语音对于许多基于语音的应用至关重要，包括听力假体，机器人和多媒体通信。许多语音分离算法在模拟环境中进行测试时表现相当不错，但这种性能水平并不总是适用于更微妙的真实的环境。例如，许多助听器用户的共同抱怨是他们的助听器在诸如餐馆的嘈杂环境中无效。目前的计算措施不能在日常环境中实现实用或方便的语音评估，这是提高现实世界分离性能的主要障碍。此外，最终用户在很大程度上被排除在开发和评价过程之外，这是不理想的，因为一种方法的有用性最终是由人决定的。这个项目的目标是开发计算评估算法，以更好地评估语音质量和可懂度在真实的环境。一个关键的研究领域专注于开发新的数据驱动的评估算法，这些算法使用深度学习来预测人类评估分数，从而能够在真实的环境中进行测试。 考虑到深度学习最近在语音处理方面取得的成功，这种新的评估方法很有前途，与以前的方法有很大的不同。作为该项目的结果，确定的频谱-时间语音属性和人类评估分数之间的关系。量化这种关系可以确保评估算法是准确的，并且与人类评估有很强的一致性。在语音分离算法开发中有效地集成人的评估应该导致改进的分离算法，这最终有利于用户和应用。这是预期的，因为准确的评估使研究人员能够更容易地根据现实世界的环境因素识别和纠正弱点。该研究活动为提高语音处理应用的真实性这一新兴研究领域奠定了基础，并为更大的科学界提供了关于人类感知的关键见解。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An End-To-End Non-Intrusive Model for Subjective and Objective Real-World Speech Assessment Using a Multi-Task Framework

• DOI: 10.1109/icassp39728.2021.9414182
• 发表时间: 2021-06
• 期刊: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Zhuohuang Zhang;P. Vyas;Xuan Dong;D. Williamson

A Classification-Aided Framework for Non-Intrusive Speech Quality Assessment

• DOI: 10.1109/waspaa.2019.8937192
• 发表时间: 2019-10
• 期刊: 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)
• 作者: Xuan Dong;D. Williamson

Towards real-world objective speech quality and intelligibility assessment using speech-enhancement residuals and convolutional long short-term memory networks

• DOI: 10.1121/10.0002702
• 发表时间: 2020-11-01
• 期刊: JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA
• 影响因子: 2.4
• 作者: Dong, Xuan;Williamson, Donald S.
• 通讯作者: Williamson, Donald S.