EAGER: Exploring the Role of Acoustic-Prosodic, Lexical, and Demographic Factors in Trustworthy Speech Perception for Conversational Agents

这一早期的探索性研究拨款探索了声学韵律、词汇和人口统计因素在会话主体感知值得信赖的合成语音中的作用。随着机器学习和语音技术的进步，会话代理参与类似人类的对话的能力越来越强。然而，信任对于有效的沟通和协作至关重要，而理解可信言语的信号对于成功的互动是必不可少的。虽然各学科的研究人员都在试图发现可信言语的信号，主要是在人类言语中，但目前人们对可信人类言语的理解与可以在会话代理中实现和使用的东西之间仍然存在差距。该项目将实施一系列创新和探索性的知觉研究，旨在系统地研究可信合成语音的韵律、词汇和人口统计学特性。为了评估在需要脆弱性和信任的环境中的信任感知，将使用真实世界的应用程序，如情感支持对话。通过揭示声学韵律、词汇和人口统计学因素的具体影响，这项研究将促进我们对人机交互中信任是如何形成和维持的理解。这项工作的发现将有助于提高会话代理的可信性。这反过来又将使更多地采用变革性技术，这些技术将在重要的应用领域造福社会，包括老年人和居家护理环境中的辅助机器人同伴、心理评估和治疗以及医院的辅助医疗。本研究的主要目的是识别声学韵律、词汇和人口统计学因素对合成语音的信任感知。该项目将通过一项大规模的众包感知研究，系统地测试合成语音的这些因素对人类信任的影响。高度受控的参数将被用来测试包括音调、强度和语速在内的声学韵律特征以及对话行为、礼貌和复杂性等词汇特征的影响。此外，这项研究还将考察说话人和听话人在信任知觉上的个体差异。通过探索韵律、词汇和人口统计因素组合的个体因素和交互效应，本研究将全面了解它们对用户信任的影响。这些发现将为对话代理的设计、开发和部署提供信息，导致创建更值得信赖和参与的人机交互。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly Grant for Exploratory Research explores the role of acoustic-prosodic, lexical, and demographic factors in the perception of trustworthy synthesized speech for conversational agents. With advances in machine learning and speech technologies, conversational agents are becoming increasingly capable of engaging in human-like conversations. However, trust is crucial for effective communication and collaboration, and understanding the signals of trustworthy speech is essential for successful interactions. While researchers across disciplines have sought to discover the signals of trustworthy speech, mostly in human speech, there remains a gap between what is currently understood about trustworthy human speech and what can be implemented and used in conversational agents. This project will implement a series of innovative and exploratory perception studies designed to systematically investigate the prosodic, lexical, and demographic properties of trustworthy synthesized speech. To evaluate trust perception in contexts that require vulnerability and trust, real-world applications such as emotional support dialogues will be used. By uncovering the specific influences of acoustic-prosodic, lexical, and demographic factors, this research will advance our understanding of how trust is formed and maintained in human-machine interactions. The findings of this work will contribute valuable insights to improve the perceived trustworthiness of conversational agents. This, in turn, will enable the increased adoption of transformative technologies that will benefit society in important application areas, including assistive robot companions in homecare settings for the elderly and homebound, psychological assessment and treatment, and assistive medical care in hospitals. The main objective of this research is to identify acoustic-prosodic, lexical, and demographic factors in trust perception of synthetic speech. The project will systematically test the effects of these factors of synthesized speech on human trust using a large-scale crowdsourced perception study. Highly controlled parameters will be manipulated to test the effects of acoustic-prosodic features including pitch, intensity, and speaking rate, as well as lexical features such as dialogue act, politeness, and complexity. In addition, the study will examine individual differences in trust perception across speaker and listener traits. By exploring individual factors as well as interactive effects of combinations of prosodic, lexical and demographic factors, this research will provide a comprehensive understanding of their influence on user trust. The findings will inform the design, development, and deployment of conversational agents, leading to the creation of more trustworthy and engaging human-machine interactions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.