EAGER: A Corpus of Aligned Speech and ANS Sensor Data

尽管有大量关于情绪言语和压力下言语的文献，但对于连续言语的特征如何随着任何特定说话者生理状态的微妙和现实相关变化而变化，人们知之甚少。这项探索性研究的早期拨款将语音特征与生理激活的直接测量联系起来，而不是与情绪或状态的分类手工注释标签联系起来。该研究收集并分析了语料库和自主神经系统（ANS）传感器数据，以发现当一个人暴露在不同的激活相关的情绪、认知、压力相关的条件下，语言特征会发生什么变化。更广泛的意义和影响是发现语言中的线索，可以用来估计说话者在没有传感器可用时生理激活水平的变化。应用包括医疗保健（监测身体、心理和认知状态）、教育和学习（监测参与度）、社会互动（监测激活水平）和执法/情报（监测高兴趣个体的行为变化）。在第一阶段（语料库收集），该项目创建了一个40个主题的语料库，其中包含与时间一致的语音和生理信号。激活测量使用最先进的方法提取心血管（ECG），血压，呼吸率和皮肤电导信号。每位受试者参与五个条件：(1)中性基线；(2)情绪性（描述情绪性突出的画面）；(3)强调（以准确性和完成时间为激励的口语任务）；(4)认知负荷（具有视觉干扰物的说话任务，对任务完成和干扰物任务准确性的激励）；(5)计算机定向语音（需要语音识别器完美识别的任务）。在阶段2（分析）中，对传感器输出进行后处理以校准信号并寻找变化。然后将这些变化与从时间排列的语音中自动提取的一系列特征（基于声学、韵律、话语模式和不流畅模式）进行比较。然后，分析和机器学习实验检查哪些语音特征变化与扬声器内部和扬声器之间的传感器输出变化相关。研究结果揭示了如何利用自然连续语音中的信息来估计说话者的变化。S生理激活水平在持续的，微妙的和日常环境。

Despite a sizeable literature on emotional speech and speech under stress, little is understood about how features in continuous speech vary with subtle and real-world-relevant changes in physiological state within any particular speaker. This EArly Grant for Exploratory Research relates speech features to direct measures of physiological activation, rather than to categorical hand-annotated labels of emotion or state. The study collects and analyzes a corpus of speech and autonomic nervous system (ANS) sensor data to discover what changes occur in speech features when a person is exposed to different activation-relevant emotional, cognitive, stress-related conditions. The broader significance and impact is discovery of cues in speech that can be used to estimate changes in a speaker's physiological activation level when no sensors are available. Applications include health care (monitoring physical, mental, cognitive states), education and learning (monitoring engagement), social interaction (monitoring activation level), and law enforcement/intelligence (monitoring behavioral changes of high interest individuals).In Phase 1 (Corpus Collection), the project creates a 40-subject corpus of time-aligned speech and physiological signals. Activation is measured using state-of-the-art methods to extract cardiovascular (ECG), blood pressure, respiration rate, and skin conductance signals. Each subject participates in five conditions: (1) neutral baseline; (2) emotional (description of emotionally salient pictures); (3) stressed (speaking task incentivized for accuracy and completion time); (4) cognitive load (speaking task with a visual distractor, incentivized for task completion and distractor task accuracy); and (5) computer-directed speech (task requiring perfect recognition from a speech recognizer). In Phase 2 (Analysis), sensor output is post-processed to calibrate the signals and look for changes. These changes are then compared to a range of automatically extracted features (based on acoustics, prosody, discourse patterns, and disfluency patterns) from the time-aligned speech. Analyses and machine learning experiments then examine which speech feature changes correlate with changes in sensor output, both within and across speakers. Results shed light on how information from natural continuous speech can be used to estimate changes in a speaker?s physiological activation level in ongoing, subtle and everyday contexts.