RI: Small: Understanding Subtle Non-Social Facial Expressivity to Boost Learning and Computer Interaction

面部表情在人类的日常交流中起着重要的作用。计算机对这些复杂而微妙的表达的理解将导致具有主动性的计算机对人类互动做出更适当反应的高性能交互式网络-人系统。该项目汇集了一个跨学科的研究团队，以解决与非社会场景中自发微表情识别相关的关键挑战。该项目专注于在学习技能（如在线学习）时产生人类的生物反馈，并以连续的彩色和深度视频流进行记录和分析。它将开发人机协同的计算机算法，并测试当训练应用被近乎实时的表情信息生物反馈增强时，这些信息如何提供卓越的学习。这代表了在训练机器识别和分类面部微表情方面迈出的重要一步，并最大限度地提高了网络人类系统的协同作用，从而提高了生活体验的质量。它将提供一个普通人触手可及的计算环境，在这个环境中可以检测和预测人们的兴趣甚至健康状况，这将对技能学习、教育和信息检索产生重大影响。该项目开发了一种理解复杂和微妙的面部微表情和生物反馈的方法，可以充分利用网络和人类系统之间的协同作用。它解决了在具有挑战性的现实环境中与计算理解和智能建模相关的关键挑战。它使用基于面部微表情的评估和干预，以最大限度地发挥网络和人类系统在技能学习方面的协同作用。首先，它考虑了深度学习和闭环视频分析在强化学习框架中优化技能学习。其次，它从颜色和深度视频流中开发了新的面部微表情表示，并将其用于独立于人的情绪识别以及适应学习任务时的特定于人的情绪识别。第三，它不仅利用了彩色相机，还利用了集成深度相机进行精确测量，这在微表情上是没有的。重点是确定微表情的实时分类在多大程度上可以提供更适当的交互性，从而促进人类在实际应用中的学习。结果将通过一个网站广泛传播，该网站将定期发布数据库和软件工具，在主要专业会议上提供教程、讲习班和演示。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Facial expressions play a significant role in everyday communication among humans. Computer understanding of these complex and subtle expressions will lead to highly capable interactive cyber-human systems with proactive computers that make more appropriate responses to human interactions. This project brings together an interdisciplinary team of investigators to address key challenges associated with spontaneous microexpression recognition in non-social scenarios. The project concentrates on generating bio-feedback from humans while learning skills, such as online learning, and being recorded and analyzed in continuous color and depth video streams. It will develop computer algorithms for human-machine synergy and test how this information can provide for superior learning when training applications are augmented with expression-informed bio-feedback in near real-time. This represents a significant step forward in training machines to recognize and classify facial microexpressions and maximizing the synergy of cyber-human systems that will improve the quality of life experiences. It will provide a computing environment within the reach of common people in which the interests or even the health of people can be detected and predicted, with significant impacts on skill learning, education and information retrieval.The project develops an approach to the understanding of complex and subtle facial microexpressions and bio-feedback where the synergy between cyber and human systems can be fully exploited. It addresses key challenges associated with computational understanding and modeling of intelligence in challenging, realistic contexts. It uses assessment and intervention based on facial microexpressions to maximize synergy of cyber and human systems for skill learning. First, it considers deep learning and closed-loop video analysis for optimized skill learning in a reinforcement learning framework. Second, it develops novel representation of facial microexpressions from color and depth video streams and use them for person independent emotion recognition as well as person-specific emotions recognition when a learning task is adapted. Third, it exploits not only the color camera but also the integrated depth camera for precise measurements, which has not been used for microexpressions. The focus is to determine the extent to which real-time classification of microexpressions can provide for more appropriate interactivity that will facilitate human learning in real applications. The results will be broadly disseminated through a website that will have regular releases of databases and software tools by offering tutorials, workshops and demos at major professional meetings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.