Device-DS as a window into mechanisms of speech production: An investigation of adults and children

Device-DS 作为语音产生机制的窗口：对成人和儿童的调查

• 批准号: 1911855
• 负责人: Michelle Cohn
• 金额: $ 13.8万
• 依托单位: Cohn, Michelle Dana
• 依托单位国家: 美国
• 项目类别: Fellowship Award
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-01 至 2022-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1911855&HistoricalAwards=false
• 关键词: Device; DS; window; into; mechanisms

This award was provided as part of NSF's Social, Behavioral and Economic Sciences Postdoctoral Research Fellowships (SPRF) program. The goal of the SPRF program is to prepare promising, early career doctoral-level scientists for scientific careers in academia, industry or private sector, and government. SPRF awards involve two years of training under the sponsorship of established scientists and encourage Postdoctoral Fellows to perform independent research. NSF seeks to promote the participation of scientists from all segments of the scientific community, including those from underrepresented groups, in its research programs and activities; the postdoctoral period is considered to be an important level of professional development in attaining this goal. Each Postdoctoral Fellow must address important scientific questions that advance their respective disciplinary fields. Under the sponsorship of Dr. Zhou Yu, Dr. Georgia Zellou, and Dr. Katharine Graf Estes at the University of California, Davis, this postdoctoral fellowship award supports an early career scientist exploring how individuals adjust their speech when talking to voice-activated artificially intelligent (voice-AI) devices, such as Amazon's Alexa or Apple's Siri, which are becoming ubiquitous features of everyday life. This project investigates the ways in which adults and children engage with voice-AI; currently little is known about how people adjust their speech when talking to these smart devices. Studying human-device interaction is important to examine the changing relationship between humans and technology; as digital devices become more prevalent, it is critical to understand their impact on our communicative and social behavior. This project explores the ways in which adults and children adapt their speech when talking to voice-activated digital assistants (e.g., Amazon's Alexa), compared to adult human interlocutors. This line of work provides a way to test differing theoretical predictions as to the extent that speech-register adjustments are driven by functional motives (e.g., intelligibility) and social factors (e.g., gender). For instance, this research explores whether the same functional motivations that apply when correcting comprehension errors to human interlocutors apply in device-directed speech (DS), such as in manipulating the phonological nature of errors, to carefully control the level of intelligibility-related pressures in communication. At the same time, this project explores how social factors may impact speech adaptation strategies, such as by interlocutor type, speaker age, or device gender. This project additionally involves important methodological innovations in programming and running experiments directly through a digital device platform. Overall, this project aims to fill a gap in our knowledge in the acoustic-phonetic adjustments humans make when talking to voice-AI devices, and can ultimately reveal the underlying mechanisms in speech production by different speakers (e.g., based on age, gender, device experience), contributing to basic science research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项是美国国家科学基金会社会、行为和经济科学博士后研究奖学金（SPRF）计划的一部分。SPRF计划的目标是为学术界、工业界或私营部门和政府的科学事业准备有前途的早期职业博士级科学家。SPRF奖励包括在知名科学家的赞助下进行为期两年的培训，并鼓励博士后进行独立研究。美国国家科学基金会寻求促进科学界各阶层的科学家，包括那些未被充分代表的群体的科学家，参与其研究项目和活动；博士后阶段被认为是实现这一目标的一个重要的专业发展阶段。每个博士后必须解决各自学科领域的重要科学问题。在加州大学戴维斯分校的周宇博士、乔治亚·泽罗博士和凯瑟琳·格拉夫·埃斯蒂斯博士的赞助下，这项博士后奖学金支持一位早期职业科学家探索个人如何在与语音激活的人工智能（voice-AI）设备（如亚马逊的Alexa或苹果的Siri）交谈时调整自己的语言，这些设备正在成为日常生活中无处不在的功能。该项目调查了成人和儿童使用语音人工智能的方式；目前，人们在与这些智能设备交谈时如何调整自己的语言还知之甚少。研究人与设备的交互对于研究人与技术之间不断变化的关系非常重要；随着数字设备变得越来越普遍，理解它们对我们的交流和社会行为的影响至关重要。该项目探索了成人和儿童在与声控数字助理（例如亚马逊的Alexa）交谈时，与成人对话者相比，如何调整他们的语言。这方面的工作提供了一种方法来测试不同的理论预测，即语音寄存器调整是由功能动机（例如，可理解性）和社会因素（例如，性别）驱动的程度。例如，本研究探讨了在纠正人类对话者的理解错误时适用的相同功能动机是否适用于设备导向语音（DS），例如操纵错误的语音性质，以仔细控制沟通中与可理解性相关的压力水平。同时，本项目探讨了社会因素如何影响语音适应策略，如对话者类型、说话者年龄或设备性别。该项目还涉及编程和直接通过数字设备平台运行实验的重要方法创新。总体而言，该项目旨在填补我们在人类与语音人工智能设备交谈时语音调整方面的知识空白，并最终揭示不同说话者（例如，基于年龄、性别、设备经验）语音产生的潜在机制，为基础科学研究做出贡献。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

The influence of conversational role on phonetic alignment toward voice-AI and human interlocutors

对话角色对语音人工智能和人类对话者语音对齐的影响

• DOI: 10.1080/23273798.2021.1931372
• 发表时间: 2021
• 期刊: Cognition and Neuroscience
• 作者: Zellou, Georgia;Cohn, Michelle;Kline, Tyler
• 通讯作者: Kline, Tyler

Differences in Gradient Emotion Perception: Human vs. Alexa Voices

梯度情感感知的差异：人类与 Alexa 语音

• DOI: 10.21437/interspeech.2020-1938
• 发表时间: 2020
• 期刊: Proceedings of Interspeech
• 作者: Cohn, Michelle;Raveh, Eran;Predeck, Kristin;Gessinger, Iona;Möbius, Bernd;Zellou, Georgia
• 通讯作者: Zellou, Georgia

Intelligibility of face-masked speech depends on speaking style: Comparing casual, clear, and emotional speech

• DOI: 10.1016/j.cognition.2020.104570
• 发表时间: 2021-01-12
• 期刊: COGNITION
• 影响因子: 3.4
• 作者: Cohn, Michelle;Pycha, Anne;Zellou, Georgia
• 通讯作者: Zellou, Georgia

Prosodic alignment toward emotionally expressive speech: Comparing human and Alexa model talkers

• DOI: 10.1016/j.specom.2021.10.003
• 发表时间: 2021-10-30
• 期刊: SPEECH COMMUNICATION
• 影响因子: 3.2
• 作者: Cohn, Michelle;Predeck, Kristin;Zellou, Georgia
• 通讯作者: Zellou, Georgia

Acoustic-phonetic properties of Siri- and human-directed speech

• DOI: 10.1016/j.wocn.2021.101123
• 发表时间: 2021-12-20
• 期刊: JOURNAL OF PHONETICS
• 影响因子: 1.9
• 作者: Cohn, Michelle;Segedin, Bruno Ferenc;Zellou, Georgia
• 通讯作者: Zellou, Georgia