EAGER: A Corpus of Aligned Speech and ANS Sensor Data

EAGER：对齐语音和 ANS 传感器数据的语料库

• 批准号: 1449202
• 负责人: Elizabeth Shriberg
• 金额: $ 5万
• 依托单位: SRI International
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-01 至 2015-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1449202&HistoricalAwards=false
• 关键词: EAGER; Corpus; Aligned; Speech; ANS

Despite a sizeable literature on emotional speech and speech under stress, little is understood about how features in continuous speech vary with subtle and real-world-relevant changes in physiological state within any particular speaker. This EArly Grant for Exploratory Research relates speech features to direct measures of physiological activation, rather than to categorical hand-annotated labels of emotion or state. The study collects and analyzes a corpus of speech and autonomic nervous system (ANS) sensor data to discover what changes occur in speech features when a person is exposed to different activation-relevant emotional, cognitive, stress-related conditions. The broader significance and impact is discovery of cues in speech that can be used to estimate changes in a speaker's physiological activation level when no sensors are available. Applications include health care (monitoring physical, mental, cognitive states), education and learning (monitoring engagement), social interaction (monitoring activation level), and law enforcement/intelligence (monitoring behavioral changes of high interest individuals).In Phase 1 (Corpus Collection), the project creates a 40-subject corpus of time-aligned speech and physiological signals. Activation is measured using state-of-the-art methods to extract cardiovascular (ECG), blood pressure, respiration rate, and skin conductance signals. Each subject participates in five conditions: (1) neutral baseline; (2) emotional (description of emotionally salient pictures); (3) stressed (speaking task incentivized for accuracy and completion time); (4) cognitive load (speaking task with a visual distractor, incentivized for task completion and distractor task accuracy); and (5) computer-directed speech (task requiring perfect recognition from a speech recognizer). In Phase 2 (Analysis), sensor output is post-processed to calibrate the signals and look for changes. These changes are then compared to a range of automatically extracted features (based on acoustics, prosody, discourse patterns, and disfluency patterns) from the time-aligned speech. Analyses and machine learning experiments then examine which speech feature changes correlate with changes in sensor output, both within and across speakers. Results shed light on how information from natural continuous speech can be used to estimate changes in a speaker?s physiological activation level in ongoing, subtle and everyday contexts.

尽管有相当多的关于情绪演讲和压力下演讲的文献，但人们对连续演讲中的特征如何随着任何特定演讲者体内生理状态的微妙和现实世界相关的变化而变化知之甚少。这项EArly探索性研究资助将语音特征与生理激活的直接测量联系起来，而不是与情感或状态的分类手写标注标签联系起来。该研究收集并分析了语音和自主神经系统（ANS）传感器数据的语料库，以发现当一个人暴露于不同的激活相关的情绪，认知，压力相关条件时，语音特征会发生什么变化。更广泛的意义和影响是发现语音中的线索，可以用来估计扬声器的生理激活水平的变化时，没有传感器可用。应用包括医疗保健（监测身体、心理和认知状态）、教育和学习（监测参与度）、社会互动（监测激活水平）和执法/情报（监测高兴趣个体的行为变化）。在第一阶段（语料库收集），该项目创建了一个包含40个主题的时间对齐语音和生理信号的语料库。使用最先进的方法来测量激活，以提取心血管（ECG）、血压、呼吸率和皮肤电导信号。每个受试者参与五个条件：（1）中性基线;（2）情绪（描述情绪突出的图片）;（3）强调（4）认知负荷（具有视觉干扰物的说话任务，激励任务完成和干扰物任务准确性）;以及（5）计算机指导的语音（需要来自语音识别器的完美识别的任务）。在阶段2（分析）中，传感器输出经过后处理，以校准信号并查找变化。然后将这些变化与从时间对齐的语音中自动提取的一系列特征（基于声学，韵律，话语模式和不流利模式）进行比较。然后，分析和机器学习实验会检查哪些语音特征变化与传感器输出的变化相关，无论是在扬声器内部还是在扬声器之间。结果揭示了如何从自然连续语音的信息可以用来估计在扬声器的变化？的生理激活水平在持续的，微妙的和日常的情况下。