RI: Small: Understanding Subtle Non-Social Facial Expressivity to Boost Learning and Computer Interaction

面部表情在人类的日常交流中扮演着重要的角色。计算机对这些复杂而微妙的表达的理解将导致具有主动计算机的高能力的交互式网络人类系统，这些系统对人类交互做出更适当的反应。该项目汇集了一个跨学科的研究团队，以解决与非社会场景中自发微表情识别相关的关键挑战。该项目专注于在学习技能（如在线学习）的同时从人类产生生物反馈，并以连续的彩色和深度视频流进行记录和分析。它将开发用于人机协同的计算机算法，并测试当训练应用程序通过近实时的表达信息生物反馈增强时，这些信息如何提供上级学习。这代表着在训练机器识别和分类面部微表情方面迈出了重要的一步，并最大限度地提高了网络人类系统的协同作用，从而提高了生活体验的质量。它将为普通人提供一个可以检测和预测人们的兴趣甚至健康的计算环境，对技能学习，教育和信息检索产生重大影响。该项目开发了一种理解复杂和微妙的面部微表情和生物反馈的方法，可以充分利用网络和人类系统之间的协同作用。它解决了在具有挑战性的现实环境中与智能的计算理解和建模相关的关键挑战。它使用基于面部微表情的评估和干预，以最大限度地发挥网络和人类系统在技能学习方面的协同作用。首先，它考虑了深度学习和闭环视频分析，以在强化学习框架中优化技能学习。其次，它开发了新的表示的面部微表情的颜色和深度的视频流，并使用它们的人独立的情感识别，以及个人特定的情感识别时，学习任务是适应。第三，它不仅利用彩色相机，还利用集成的深度相机进行精确测量，这还没有用于微表情。重点是确定微表情的实时分类可以在多大程度上提供更适当的交互性，从而促进人类在真实的应用中的学习。结果将通过一个网站广泛传播，该网站将定期发布数据库和软件工具，并在主要的专业会议上提供教程、研讨会和演示。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

Facial expressions play a significant role in everyday communication among humans. Computer understanding of these complex and subtle expressions will lead to highly capable interactive cyber-human systems with proactive computers that make more appropriate responses to human interactions. This project brings together an interdisciplinary team of investigators to address key challenges associated with spontaneous microexpression recognition in non-social scenarios. The project concentrates on generating bio-feedback from humans while learning skills, such as online learning, and being recorded and analyzed in continuous color and depth video streams. It will develop computer algorithms for human-machine synergy and test how this information can provide for superior learning when training applications are augmented with expression-informed bio-feedback in near real-time. This represents a significant step forward in training machines to recognize and classify facial microexpressions and maximizing the synergy of cyber-human systems that will improve the quality of life experiences. It will provide a computing environment within the reach of common people in which the interests or even the health of people can be detected and predicted, with significant impacts on skill learning, education and information retrieval.The project develops an approach to the understanding of complex and subtle facial microexpressions and bio-feedback where the synergy between cyber and human systems can be fully exploited. It addresses key challenges associated with computational understanding and modeling of intelligence in challenging, realistic contexts. It uses assessment and intervention based on facial microexpressions to maximize synergy of cyber and human systems for skill learning. First, it considers deep learning and closed-loop video analysis for optimized skill learning in a reinforcement learning framework. Second, it develops novel representation of facial microexpressions from color and depth video streams and use them for person independent emotion recognition as well as person-specific emotions recognition when a learning task is adapted. Third, it exploits not only the color camera but also the integrated depth camera for precise measurements, which has not been used for microexpressions. The focus is to determine the extent to which real-time classification of microexpressions can provide for more appropriate interactivity that will facilitate human learning in real applications. The results will be broadly disseminated through a website that will have regular releases of databases and software tools by offering tutorials, workshops and demos at major professional meetings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.