CISE-MSI: RCBP-RF: HCC: Understanding Human Emotions Associated with Attention Activities in Temporal Domain Utilizing Multimodal Data

CISE-MSI：RCBP-RF：HCC：利用多模态数据了解与时域注意力活动相关的人类情绪

• 批准号: 2219634
• 负责人: Soo-Yeon Ji
• 金额: $ 39.99万
• 依托单位: Bowie State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2219634&HistoricalAwards=false
• 关键词: CISE; MSI; RCBP; RF; HCC

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Emotions impact human behavior and have considerable effect on cognitive function, as they are closely linked to learning, decision-making, and memory. Since the importance of recognizing emotions has been steadily growing, identifying them has been studied in domains such as robotics, audience understanding, and therapy. In clinical settings, especially with respect to neurodevelopmental disorders, emotion analysis has been studied in an effort to help individuals with attention-deficit/hyperactivity disorder (ADHD). Adults with ADHD have difficulty attending to important details, planning task completion, modulating responses, and complex learning. Understanding emotions and how they change over time in young adults with ADHD is not yet fully understood and is the focus of this research. Project outcomes will include. an interdisciplinary solution for understanding and adjusting emotional behavior to enhance human performance. Project outcomes will enable monitoring of emotion changes for various applications such as attention training, interventions in social-emotional adjustments, and classroom-based learning, and ultimately will contribute to the development of real-time emotion monitoring utilizing wearable sensor devices. Additional broad impact will derive from building research capacity at a minority serving institute, by preparing student researchers to collaborate with faculty on cutting edge research.This research involves designing a model that can predict emotions in the near future by examining the emotions associated with attention activities over time and analyzing multimodal data such as eye-tracking, electroencephalogram (EEG), skin temperature, galvanic skin response, and oxygen saturation. The project will investigate stream-based multimodal data acquisition using a scientific workflow system that leverages both metadata standards and visual programming. Next, it will collect data from young adults with ADHD to generate multimodal metrics during an emotionality analysis task and extract emotion-attention features associated with attention activities. Subsequently, the research will focus on generating a model to forecast emotions to enhance students' learning in data science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。情绪影响人类行为，对认知功能有相当大的影响，因为它们与学习，决策和记忆密切相关。由于识别情绪的重要性一直在稳步增长，识别它们已经在机器人，观众理解和治疗等领域进行了研究。 在临床环境中，特别是在神经发育障碍方面，已经研究了情绪分析，以帮助患有注意力缺陷/多动障碍（ADHD）的个体。患有ADHD的成年人很难注意到重要的细节，计划任务完成，调节反应和复杂的学习。 了解情绪以及它们如何随着时间的推移在患有ADHD的年轻人中变化尚未完全理解，这是这项研究的重点。 项目成果将包括。 一个跨学科的解决方案，用于理解和调整情绪行为，以提高人类的表现。 项目成果将使监测情绪变化的各种应用，如注意力训练，干预社会情绪调整，并以课堂为基础的学习，并最终将有助于利用可穿戴传感器设备的实时情绪监测的发展。 通过培养学生研究人员与教师合作进行前沿研究，在少数民族服务机构建立研究能力，将产生更广泛的影响。这项研究涉及设计一个模型，通过检查与注意力活动相关的情绪，并分析多模态数据，如眼动跟踪，脑电图（EEG），皮肤温度，皮肤电反应，和氧饱和度。 该项目将研究基于流的多模态数据采集使用科学的工作流程系统，利用元数据标准和可视化编程。接下来，它将收集患有ADHD的年轻人的数据，以在情绪分析任务期间生成多模态指标，并提取与注意力活动相关的情绪注意力特征。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Identifying Patterns for Neurological Disabilities by Integrating Discrete Wavelet Transform and Visualization

• DOI: 10.3390/app14010273
• 发表时间: 2023-12
• 期刊: Applied Sciences
• 作者: Soo-Yeon Ji;S. Jayarathna;A. M. Perrotti;Katrina Kardiasmenos;Dong H. Jeong