Collaborative Research: EAGER: Deep Learning-based Multimodal Analysis of Sleep

合作研究：EAGER：基于深度学习的睡眠多模态分析

• 批准号: 2334665
• 负责人: Ping Chen
• 金额: $ 12.5万
• 依托单位: University of Massachusetts Boston
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2334665&HistoricalAwards=false
• 关键词: Collaborative; Research; EAGER; Deep; Learning

Sleep is one of the most fundamental behaviors for animals and humans, and understanding group sleep will provide key insights into neuroscience and social behavior and interactions. To overcome limitations that stem from single modality animal behavior platforms, the project will develop a multimodal machine learning method to simultaneously monitor and process Electroencephalogram (EEG) data and animal behavior data to systematically study group behavior, especially sleep, and to annotate animal social movements/behavior. The outcomes from the project will potentially provide a powerful toolkit based on deep learning to make sense of complex animal behavior and EEG activity pattern for mechanistic exploration. Subproblems from this project will be developed into course materials and will be capstone projects or directed study for undergraduate students.The project will process multiple data modalities and group activities involving multiple entities from multiple data sources through a multi-modal machine learning framework enabling the extraction and aggregation of the most pertinent information. A “dictionary” of movements at the semantic level will be developed for learning and processing of long video and EEG data, which is a significant challenge for current state-of-the-art self-attention transformer models. Additionally, to incorporate group interactions, transformer models for dialogue modeling will be developed. The proposed simultaneous EEG and behavior study will provide biological underpinnings of group sleep, leading to insights into brain electrical signaling and behavioral outputs - how the brain marshals its signaling units to generate behaviors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

睡眠是动物和人类最基本的行为之一，了解群体的睡眠将为神经科学以及社会行为和互动提供关键的见解。为了克服源于单个模态行为平台的局限性，该项目将开发一种多模式的机器学习方法，以同时监视和处理脑电图（EEG）数据和动物行为数据，以系统地研究小组行为，尤其是睡眠，并注释动物社会运动/行为。该项目的结果可能会基于深度学习提供强大的工具包，以使动物行为和脑电图活动模式有意义，以进行机械探索。该项目的子问题将被发展为课程材料，并将是本科生的顶峰项目或定向研究。该项目将通过多种模式的机器学习框架来处理多个数据源的多个数据模式和小组活动，从而实现最相关信息的提取和聚合。在语义层面上的“词典”将开发用于学习和处理长视频和脑电图数据的“词典”，这对于当前最新的自我发明变压器模型是一个重大挑战。此外，为了结合群体交互，将开发用于对话建模的变压器模型。拟议的同时脑电图和行为研究将提供群体睡眠的生物学基础，从而深入了解大脑电信号传导和行为输出 - 大脑如何将其信号单位串通以产生行为。该奖项反映了NSF的法规任务，并认为通过基金会的知识优点和更广泛的criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia the Issportion supperiation。