CRII: RI: Towards Human-Level Assessment of Speech Quality and Intelligibility in Real-World Environments

将语音从背景噪声中分离出来对于许多基于语音的应用至关重要，包括助听器、机器人和多媒体通信。许多语音分离算法在模拟环境中测试时表现得相当好，但这种水平的性能并不总是延续到更微妙的真实环境中。例如，许多助听器使用者的一个常见抱怨是，他们的助听器在嘈杂的环境（如餐馆）中效果不佳。目前的计算方法不能在日常环境中实现实用或方便的语音评估，这是提高现实世界分离性能的主要障碍。此外，最终用户在很大程度上被排除在开发和评估过程之外，这是不理想的，因为方法的有用性最终是由人决定的。该项目的目标是开发计算评估算法，以更好地评估真实环境中的语音质量和可理解性。一个关键的研究领域集中在开发新颖的、数据驱动的评估算法，该算法使用深度学习来预测人类的评估分数，从而能够在真实环境中进行测试。考虑到深度学习最近在语音处理方面取得的成功，这种新的评估方法很有前途，与以前的方法有很大的不同。作为这个项目的结果，确定了频谱-时间语音属性与人类评估分数之间的关系。量化这种关系可以确保评估算法是准确的，并且与人类评估有很强的一致性。在语音分离算法的开发中有效地整合人的评估将导致分离算法的改进，最终使用户和应用受益。这是预期的，因为准确的评估使研究人员能够更容易地识别和纠正基于现实世界环境因素的弱点。这些研究活动为提高语音处理应用中的真实感这一新兴研究领域奠定了基础，并为更大的科学界提供了关于人类感知的关键见解。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Separating speech from background noise is crucial for many speech-based applications, including hearing prostheses, robotics, and multimedia communication. Many speech separation algorithms perform reasonably well when they are tested in simulated environments, but this level of performance does not always carry over to real environments that are more nuanced. For example, a common complaint of many hearing aid users is that their hearing aid is not effective in noisy environments such as restaurants. Current computational measures do not enable practical or convenient speech assessment in everyday environments, and this is a major hurdle for improving real-world separation performance. In addition, the end-user has largely been left out of the development and evaluation process, which is not ideal since an approach's usefulness is ultimately determined by people. The objective of this project is to develop computational evaluation algorithms to better assess speech quality and intelligibility in real environments. A key area of research focuses on developing novel, data-driven assessment algorithms that use deep learning to predict human assessment scores, which enables testing in real environments. Considering the recent success that deep learning has had in speech processing, this new assessment approach is promising and offers substantial differences from prior approaches. The relationship between spectral-temporal speech attributes and human assessment scores are determined as a result of this project. Quantifying this relationship ensures that assessment algorithms are accurate and have strong agreement with human evaluations. An effective integration of human assessment in speech separation algorithm development should result in improved separation algorithms, which ultimately benefits users and applications. This is expected since accurate assessment enables researchers to more easily identify and correct weaknesses based on real-world environmental factors. The research activities lay the foundation for the emerging research area of improving realism in speech processing applications and offer key insights on human perception to the larger scientific community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.