SBIR Phase II: Emotionally Immersive Tele-Learning

SBIR第二期项目旨在将新型机器视觉和社交网络功能整合到在线教育和网络研讨会中。长期以来，研究人员一直认为，在任何学习环境中，尤其是在线环境中，参与都是成功的关键因素，但目前的在线教学系统缺乏教师衡量学生参与程度的手段，因为这些学生是不可见的。因此，为了改进当前的在线教学模式，有必要找到方法与教师沟通他们未见过的在线学生的参与程度。该项目的成功成果将使讲师能够从面部表情、凝视和其他身体动作中获得实时反馈，当这些反馈在虚拟教室中平均时，就会提供与信息传递的接收相关的反馈。该项目支持美国国家科学基金会的教育使命，旨在回答有关教师如何为学生提供有效的认知和动机支持的问题。该项目旨在通过整合认知和动机框架，促进导师和在线学生之间更丰富的互动，从而提高学生的成功水平，使他们能够在21世纪的劳动力大军中更有效地参与竞争。该项目贡献了四个重要的创新：一个用于注视方向和面部表情的机器视觉识别系统，该系统可以收集参与者的数据，以提高信号对噪声的影响；用于检测手势和姿势动作的机器视觉识别系统；在使用辅助硬件的框架内启用第三方教学应用程序，这些应用程序可以通过教育应用程序商店出售；并整合社交网络功能，复制课前和课后的社交活动，包括结对分享、分组讨论、团队教学和支持在线教学协助。支持上述创新所需的研究目标和范围包括：改进机器视觉分类器数据和功能；优化用户界面，集成用户校准流程；将系统的功能扩展到同步和异步模式；开发网络教学过程的神经心理学和认知模型设计和整合内容/媒体互操作性和社交网络功能；通过数字白板和平板电脑为第三方开发人员和应用程序商店功能创建开放的应用程序编程接口；开发辅助软件模块，帮助学生管理与其教育工作相关的数据，诊断学习和成就模式，并根据该数据提供专家建议。

This SBIR Phase II project aims to incorporate novel machine vision and social networking functionality into technologies used in online education and webinars. Researchers have long identified engagement as the key ingredient for success in any learning environment and particularly the online environment, but current online teaching systems lack the means by which instructors can gauge their students' level of engagement because these students are not visible. Therefore in order to improve current online teaching modalities, it is necessary to find ways to communicate to the instructor the level of engagement of their unseen online students. The successful outcome of the project will allow the lecturer to receive real-time feedback from facial expressions, gaze and other body kinesics, which when averaged across the virtual classroom, provides feedback related to the reception of information delivery. The project supports NSF's mission in education, which seeks to answer questions about how teachers can provide effective cognitive and motivational support for students. This project aims to promote richer interactions between tutor and online students through integrated cognitive and motivational scaffolding, leading to higher levels of student success enabling them to compete more effectively as skilled artisans in the 21st century workforce. This project contributes four significant innovations: a machine-vision recognition system for gaze direction and facial expressions of engagement, which aggregates data across participants to improve signal to noise; a machine-vision recognition system for detecting hand gestures and postural kinesics; enabling third party pedagogical applications within a framework using ancillary hardware, and which can be sold through an educational application store; and to integrate social network functionality that replicates pre- and post-lecture socialization including pair sharing, breakout groups, team teaching, and support for online teaching assistance. The goals and scope of research required to support the above innovations include: improving machine-vision classifier data and functionality; optimization of user interface and integrated user calibration process; extending the system's functionality to both synchronous and asynchronous modalities; developing neuro-psychological and cognitive models of the online pedagogical process; design and integration of content/media interoperability and social networking capabilities; creation of open application programming interfaces for third-party developers and application store functionality with digital whiteboards and tablets; development of ancillary software modules to help students manage data related to their educational efforts, diagnose study and achievement patterns, and provide expert advice based on that data.