EAGER: Exploring the Role of Acoustic-Prosodic, Lexical, and Demographic Factors in Trustworthy Speech Perception for Conversational Agents

这项探索性研究的早期资助探索了声学韵律、词汇和人口统计学因素在对话代理可信赖合成语音感知中的作用。随着机器学习和语音技术的进步，会话代理越来越有能力参与类似人类的对话。然而，信任对于有效的沟通和合作是至关重要的，理解可信言语的信号对于成功的互动是必不可少的。虽然跨学科的研究人员一直在寻求发现可信赖语音的信号，主要是在人类语音中，但目前对可信赖人类语音的理解与可在会话代理中实现和使用的内容之间仍然存在差距。该项目将实施一系列创新和探索性的感知研究，旨在系统地调查可信合成语音的韵律、词汇和人口统计学特征。为了在需要脆弱性和信任的环境中评估信任感知，将使用现实世界的应用，如情感支持对话。通过揭示声韵律、词汇和人口因素的具体影响，本研究将促进我们对人机交互中信任是如何形成和维持的理解。本研究结果将为提高会话代理的感知可信度提供有价值的见解。反过来，这将使更多地采用变革性技术，这些技术将在重要的应用领域造福社会，包括为老年人和居家人士提供家庭护理环境中的辅助机器人伴侣、心理评估和治疗，以及医院的辅助医疗护理。本研究的主要目的是识别合成语音信任感知中的声学、韵律、词汇和人口因素。该项目将通过大规模众包感知研究系统地测试合成语音的这些因素对人类信任的影响。高度控制的参数将被操纵来测试声学-韵律特征的影响，包括音调、强度和说话速度，以及词汇特征，如对话行为、礼貌和复杂性。此外，本研究还将考察说话者和倾听者在信任感知方面的个体差异。通过探索个体因素以及韵律、词汇和人口因素组合的交互作用，本研究将全面了解它们对用户信任的影响。这些发现将为对话代理的设计、开发和部署提供信息，从而创造出更值得信赖、更吸引人的人机交互。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly Grant for Exploratory Research explores the role of acoustic-prosodic, lexical, and demographic factors in the perception of trustworthy synthesized speech for conversational agents. With advances in machine learning and speech technologies, conversational agents are becoming increasingly capable of engaging in human-like conversations. However, trust is crucial for effective communication and collaboration, and understanding the signals of trustworthy speech is essential for successful interactions. While researchers across disciplines have sought to discover the signals of trustworthy speech, mostly in human speech, there remains a gap between what is currently understood about trustworthy human speech and what can be implemented and used in conversational agents. This project will implement a series of innovative and exploratory perception studies designed to systematically investigate the prosodic, lexical, and demographic properties of trustworthy synthesized speech. To evaluate trust perception in contexts that require vulnerability and trust, real-world applications such as emotional support dialogues will be used. By uncovering the specific influences of acoustic-prosodic, lexical, and demographic factors, this research will advance our understanding of how trust is formed and maintained in human-machine interactions. The findings of this work will contribute valuable insights to improve the perceived trustworthiness of conversational agents. This, in turn, will enable the increased adoption of transformative technologies that will benefit society in important application areas, including assistive robot companions in homecare settings for the elderly and homebound, psychological assessment and treatment, and assistive medical care in hospitals. The main objective of this research is to identify acoustic-prosodic, lexical, and demographic factors in trust perception of synthetic speech. The project will systematically test the effects of these factors of synthesized speech on human trust using a large-scale crowdsourced perception study. Highly controlled parameters will be manipulated to test the effects of acoustic-prosodic features including pitch, intensity, and speaking rate, as well as lexical features such as dialogue act, politeness, and complexity. In addition, the study will examine individual differences in trust perception across speaker and listener traits. By exploring individual factors as well as interactive effects of combinations of prosodic, lexical and demographic factors, this research will provide a comprehensive understanding of their influence on user trust. The findings will inform the design, development, and deployment of conversational agents, leading to the creation of more trustworthy and engaging human-machine interactions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.