Collaborative Research: CCRI: New: An Open Data Infrastructure for Bodily Expressed Emotion Understanding

该项目的目标是解锁已经在互联网视频中找到的关于人类表情的丰富信息。多学科项目团队将收集在线提供的人类运动视频，并使用运动分析专家和非专家来精确定位人类运动的特征，这些特征可用于驱动算法，该算法将尝试对人类运动者表达的情感进行分类。这些特征将在数据上形成标签，包括背景、人口统计、移动分析的技术概念和情感。这项工作将采取史无前例的多学科方法，为身体情感表达的计算建模创建数据基础设施。为了确保基础设施与人-机器人交互研究的兼容性，该团队将进行一项面向公众的可行性研究。该小组还将雇用咨询委员会，并继续与计算机和信息科学和工程研究界多个分学科的活跃研究人员接触，设计、创建、测试和传播数据基础设施，并组织年度用户社区讲习班和基准挑战。数据基础设施有望推动人类身体情感和情感表达的数据驱动建模方面的技术创新和突破，这是医疗保健应用中的一个高度复杂的问题，例如护理机器人和心理健康诊断工具，制造例如社会意识自动叉车和安全监控系统，安全，例如监控，以及消费电子产品，例如改善与家庭机器人的交互。身体运动表达重要信息，包括传达情感，这对未来的人机交互至关重要。就像在图像识别等人工智能(AI)的其他领域一样，大规模数据驱动的方法有望揭示人类身体表达的复杂、微妙和上下文性质的新见解。然而，身体表情的计算识别研究正在努力成熟，因为研究人员必须重复许多相同的密集工作步骤，从而造成不同的努力和费用。肢体表情计算识别是情感计算、人工智能和人机交互的一个领域。该NSF项目旨在创建一个大规模的、高质量的、多方面的、带注释的、开放的和可扩展的数据基础设施，用于在各种环境中对人体表情进行计算理解。它将利用该团队在人工智能、计算机视觉、情感计算、表现力机器人、情感识别、心理学、运动分析、统计和数据挖掘、数据伦理和艺术方面的专业知识来创建(1)根据主观经验、情感和身体运动研究的需求量身定做的数据共享基础设施，(2)众包注释视频数据集，以及(3)用于严格可靠性验证、重复性和透明度评估以及基于内容的搜索和检索的工具和软件集合。数据基础设施预计将服务于机器人、心理学、表演艺术、动画和娱乐等领域的应用。该项目还通过支持研究生和本科生，包括来自代表性不足的群体的学生，提供进行基础设施开发的经验，整合来自多个学科的知识，从而发展这一新兴领域的人力专业知识。这些学生将定期与团队的国际合作伙伴互动。在这项工作中创造广泛公众参与的公共活动将集中在人与机器人交互的众多应用上。该基础设施将促进情感计算、人工智能、包括人工情感智能和人-人工智能交互、计算机视觉、社交/辅助机器人、虚拟代理、精神病学远程医疗、以人为中心的设计、机器/深度学习、计算伦理和相关社区的重点研究项目和议程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The project goal is to unlock the wealth of information about human expression that is already found in videos on the internet. The multidisciplinary project team will collect videos of human movement available online and use experts in movement analysis and non-experts to pinpoint at characteristics of the human movement that can be used to drive algorithms that will attempt to classify the emotion expressed by the human mover. These characteristics will form labels on the data that include context, demographics, technical concepts from movement analysis, and emotion. This work will take an unprecedented, multidisciplinary approach in creating a data infrastructure for computational modeling of bodily expression of emotion. To ensure the infrastructure's compatibility with human-robot interaction research, the team will conduct a public-facing feasibility study. The team will also employ advisory boards and continue to engage with active researchers in multiple sub-disciplines of the computer and information science and engineering research community in the designing, creation, testing, and dissemination of the data infrastructure, and organizing annual user community workshops and benchmarking challenges. The data infrastructure is expected to promote technological innovations and breakthroughs in data-driven modeling of human bodily expression of emotion and affect, a highly complex problem with applications in healthcare, e.g., caregiving robots and diagnostic tools for mental health, manufacturing, e.g., socially-aware autonomous forklifts and safety monitoring systems, security, e.g., monitoring, and consumer electronics, e.g., improved interactions with a home robot.Bodily movement expresses important information, including conveying emotion, which is crucial for future human-machine interactions. As in other areas of artificial intelligence (AI), such as image recognition, a large-scale data-driven approach holds promise for revealing new insights into the complex, subtle, and contextual nature of human bodily expression. However, research on computational recognition of bodily expression, an area of affective computing, AI, and human-robot interaction, is struggling to mature as researchers must replicate many of the same work-intensive steps, creating divergent efforts and expense. This NSF project aims to create a large-scale, high-quality, multifaceted, annotated, open, and extensible data infrastructure for computational understanding of human bodily expressions in a variety of settings. It will leverage the team's expertise in AI, computer vision, affective computing, expressive robotics, emotion recognition, psychology, movement analysis, statistics and data mining, data ethics, and the arts to create (1) a data-sharing infrastructure tailored to the needs of research into subjective experience, emotion, and bodily movement, (2) a crowdsourced annotated video dataset, and (3) a collection of tools and software for rigorous reliability validation, reproducibility and transparency assessment, and content-based search and retrieval. The data infrastructure is expected to serve applications in fields such as robotics, psychology, performing arts, animation, and entertainment. The project also develops human expertise in this emerging field by supporting graduate and undergraduate students, including students from underrepresented groups, providing experience in conducting infrastructure development, integrating knowledge from multiple disciplines. These students will interact regularly with the team’s international partners. Public events that create broad public engagement in the work will focus on numerous applications to human-robot interaction. The infrastructure will stimulate focused research projects and agendas in affective computing, AI, including artificial emotional intelligence and human-AI interaction, computer vision, social/assistive robotics, virtual agents, psychiatric telemedicine, human-centered design, machine/deep learning, ethics in computing, and related communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.