Collaborative Research: CCRI: New: An Open Data Infrastructure for Bodily Expressed Emotion Understanding

合作研究：CCRI：新：用于理解身体表达情绪的开放数据基础设施

• 批准号: 2234197
• 负责人: Rachelle Tsachor
• 金额: $ 17.66万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-15 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2234197&HistoricalAwards=false
• 关键词: Collaborative; Research; CCRI; New; Open

The project goal is to unlock the wealth of information about human expression that is already found in videos on the internet. The multidisciplinary project team will collect videos of human movement available online and use experts in movement analysis and non-experts to pinpoint at characteristics of the human movement that can be used to drive algorithms that will attempt to classify the emotion expressed by the human mover. These characteristics will form labels on the data that include context, demographics, technical concepts from movement analysis, and emotion. This work will take an unprecedented, multidisciplinary approach in creating a data infrastructure for computational modeling of bodily expression of emotion. To ensure the infrastructure's compatibility with human-robot interaction research, the team will conduct a public-facing feasibility study. The team will also employ advisory boards and continue to engage with active researchers in multiple sub-disciplines of the computer and information science and engineering research community in the designing, creation, testing, and dissemination of the data infrastructure, and organizing annual user community workshops and benchmarking challenges. The data infrastructure is expected to promote technological innovations and breakthroughs in data-driven modeling of human bodily expression of emotion and affect, a highly complex problem with applications in healthcare, e.g., caregiving robots and diagnostic tools for mental health, manufacturing, e.g., socially-aware autonomous forklifts and safety monitoring systems, security, e.g., monitoring, and consumer electronics, e.g., improved interactions with a home robot.Bodily movement expresses important information, including conveying emotion, which is crucial for future human-machine interactions. As in other areas of artificial intelligence (AI), such as image recognition, a large-scale data-driven approach holds promise for revealing new insights into the complex, subtle, and contextual nature of human bodily expression. However, research on computational recognition of bodily expression, an area of affective computing, AI, and human-robot interaction, is struggling to mature as researchers must replicate many of the same work-intensive steps, creating divergent efforts and expense. This NSF project aims to create a large-scale, high-quality, multifaceted, annotated, open, and extensible data infrastructure for computational understanding of human bodily expressions in a variety of settings. It will leverage the team's expertise in AI, computer vision, affective computing, expressive robotics, emotion recognition, psychology, movement analysis, statistics and data mining, data ethics, and the arts to create (1) a data-sharing infrastructure tailored to the needs of research into subjective experience, emotion, and bodily movement, (2) a crowdsourced annotated video dataset, and (3) a collection of tools and software for rigorous reliability validation, reproducibility and transparency assessment, and content-based search and retrieval. The data infrastructure is expected to serve applications in fields such as robotics, psychology, performing arts, animation, and entertainment. The project also develops human expertise in this emerging field by supporting graduate and undergraduate students, including students from underrepresented groups, providing experience in conducting infrastructure development, integrating knowledge from multiple disciplines. These students will interact regularly with the team’s international partners. Public events that create broad public engagement in the work will focus on numerous applications to human-robot interaction. The infrastructure will stimulate focused research projects and agendas in affective computing, AI, including artificial emotional intelligence and human-AI interaction, computer vision, social/assistive robotics, virtual agents, psychiatric telemedicine, human-centered design, machine/deep learning, ethics in computing, and related communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目标是解开已经在互联网视频中发现的关于人类表达的丰富信息。多学科项目团队将收集在线上可用的人类运动视频，并使用运动分析专家和非专家来确定人类运动的特征，这些特征可用于驱动算法，这些算法将尝试对人类运动者所表达的情感进行分类。这些特征将在数据上形成标签，包括上下文、人口统计、来自运动分析的技术概念和情感。这项工作将采用前所未有的多学科方法，为身体情感表达的计算建模创建数据基础设施。为了确保基础设施与人机交互研究的兼容性，该团队将进行面向公众的可行性研究。该团队还将聘请顾问委员会，并继续与计算机和信息科学与工程研究社区的多个子学科的活跃研究人员合作，设计、创建、测试和传播数据基础设施，并组织年度用户社区研讨会和基准挑战。数据基础设施预计将促进人类情感和情感表达的数据驱动建模方面的技术创新和突破，这是一个高度复杂的问题，在医疗保健领域应用，例如护理机器人和心理健康诊断工具，制造业，例如社会意识自主叉车和安全监控系统，安全，例如监控，以及消费电子产品，例如与家庭机器人的改进交互。身体动作表达了重要的信息，包括传递情感，这对未来的人机交互至关重要。与图像识别等人工智能（AI）的其他领域一样，大规模数据驱动的方法有望揭示人类身体表达的复杂、微妙和语境本质的新见解。然而，对身体表情的计算识别的研究，情感计算、人工智能和人机交互的一个领域，正在努力走向成熟，因为研究人员必须复制许多相同的工作密集型步骤，产生不同的努力和费用。这个NSF项目旨在创建一个大规模的、高质量的、多方面的、有注释的、开放的和可扩展的数据基础设施，用于在各种环境下对人类身体表情的计算理解。它将利用团队在人工智能、计算机视觉、情感计算、表达机器人、情感识别、心理学、运动分析、统计和数据挖掘、数据伦理和艺术方面的专业知识，创建(1)一个根据主观体验、情感和身体运动研究需求量身定制的数据共享基础设施，(2)一个众包注释视频数据集，以及(3)一系列用于严格可靠性验证的工具和软件。再现性和透明度评估，以及基于内容的搜索和检索。该数据基础设施有望服务于机器人、心理学、表演艺术、动画和娱乐等领域的应用。该项目还通过支持研究生和本科生，包括来自代表性不足群体的学生，提供进行基础设施开发的经验，整合多学科知识，开发这一新兴领域的人力专业知识。这些学生将定期与团队的国际合作伙伴进行互动。在工作中创造广泛公众参与的公共事件将集中在人机交互的众多应用上。该基础设施将刺激情感计算、人工智能（包括人工情感智能和人机交互）、计算机视觉、社交/辅助机器人、虚拟代理、精神病学远程医疗、以人为本的设计、机器/深度学习、计算伦理以及相关社区的重点研究项目和议程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。