EAGER: Exploring the Role of Acoustic-Prosodic, Lexical, and Demographic Factors in Trustworthy Speech Perception for Conversational Agents

这个EArly探索性研究基金探索了声学韵律，词汇和人口因素在感知会话代理人的可信合成语音中的作用。随着机器学习和语音技术的进步，会话代理越来越能够参与类似人类的对话。然而，信任对于有效的沟通和协作至关重要，理解可信语音的信号对于成功的互动至关重要。 虽然跨学科的研究人员试图发现可信语音的信号，主要是在人类语音中，但目前对可信人类语音的理解与可以在会话代理中实现和使用的内容之间仍然存在差距。本计画将进行一系列创新性及探索性的知觉研究，以系统地探讨可信赖合成语音的韵律、词汇及人口统计特性。 为了在需要脆弱性和信任的环境中评估信任感知，将使用情感支持对话等现实世界的应用程序。 通过揭示声学韵律，词汇和人口因素的具体影响，本研究将推进我们对信任如何形成和维持人机交互的理解。 这项工作的结果将有助于有价值的见解，以提高感知的可信度的会话代理。 反过来，这将使人们能够更多地采用变革性技术，这些技术将在重要的应用领域造福社会，包括老年人和家庭护理环境中的辅助机器人伴侣，心理评估和治疗，以及医院中的辅助医疗护理。 本研究的主要目的是识别合成语音的信任感知的声学韵律，词汇和人口统计学因素。该项目将使用大规模的众包感知研究系统地测试合成语音的这些因素对人类信任的影响。高度控制的参数将被操纵来测试声学韵律特征的影响，包括音高，强度和语速，以及词汇特征，如对话行为，礼貌和复杂性。 此外，本研究还将考察说话者和听话者在信任感知上的个体差异。本研究通过探讨个人因素以及韵律、词汇和人口统计学因素组合的交互作用，全面了解它们对用户信任的影响。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly Grant for Exploratory Research explores the role of acoustic-prosodic, lexical, and demographic factors in the perception of trustworthy synthesized speech for conversational agents. With advances in machine learning and speech technologies, conversational agents are becoming increasingly capable of engaging in human-like conversations. However, trust is crucial for effective communication and collaboration, and understanding the signals of trustworthy speech is essential for successful interactions. While researchers across disciplines have sought to discover the signals of trustworthy speech, mostly in human speech, there remains a gap between what is currently understood about trustworthy human speech and what can be implemented and used in conversational agents. This project will implement a series of innovative and exploratory perception studies designed to systematically investigate the prosodic, lexical, and demographic properties of trustworthy synthesized speech. To evaluate trust perception in contexts that require vulnerability and trust, real-world applications such as emotional support dialogues will be used. By uncovering the specific influences of acoustic-prosodic, lexical, and demographic factors, this research will advance our understanding of how trust is formed and maintained in human-machine interactions. The findings of this work will contribute valuable insights to improve the perceived trustworthiness of conversational agents. This, in turn, will enable the increased adoption of transformative technologies that will benefit society in important application areas, including assistive robot companions in homecare settings for the elderly and homebound, psychological assessment and treatment, and assistive medical care in hospitals. The main objective of this research is to identify acoustic-prosodic, lexical, and demographic factors in trust perception of synthetic speech. The project will systematically test the effects of these factors of synthesized speech on human trust using a large-scale crowdsourced perception study. Highly controlled parameters will be manipulated to test the effects of acoustic-prosodic features including pitch, intensity, and speaking rate, as well as lexical features such as dialogue act, politeness, and complexity. In addition, the study will examine individual differences in trust perception across speaker and listener traits. By exploring individual factors as well as interactive effects of combinations of prosodic, lexical and demographic factors, this research will provide a comprehensive understanding of their influence on user trust. The findings will inform the design, development, and deployment of conversational agents, leading to the creation of more trustworthy and engaging human-machine interactions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.