Cyberlearning: Sensei: High-Fidelity, Non-Invasive Classroom Sensing for Professional Development

网络学习：Sensei：用于专业发展的高保真、非侵入式课堂感知

• 批准号: 1822813
• 负责人: Amy Ogan
• 金额: $ 75万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1822813&HistoricalAwards=false
• 关键词: Cyberlearning; Sensei; Fidelity; Non; Invasive

For years, research has shown that moving away from large lectures and increasing student engagement and participation in classrooms significantly improves learning. Unfortunately, professors lack quality professional development opportunities to improve their instruction, and typically receive no training on how to teach. This project is addressing the issue of college professors' professional development through a cyberlearning innovation called Sensei. Sensei has novel capabilities using sensors to capture, isolate, and analyze voice and video that will provide near real time data on classroom interactions such as the percent time students talk vs professors, the percent time students talk to students, student engagement through facial expression analysis, turn taking between students and professors, etc all of which involve multimodal analysis of voice and video. The second component of this research is the development of suggested actions to improve the professor's performance as a teacher.More precisely, Sensei draws on technical and socio-technical advances in sensing arrays, computer vision, intelligent environments, and personal informatics, as well as frameworks of professional development in higher education. In this project the researchers will 1) develop the technologies needed to automatically sense and display feedback to instructors, 2) deploy this system in-vivo to college instructors over semesters of use in a series of design-based research studies, and interpret the results to 3) iterate on our framework for the routine incorporation of classroom data into professional development. This research is enabled by a cyber innovation in which computing is expanded by the capabilities of state of the art multimodal sensing approaches to achieve non-invasive sensing at classroom-scale. This cyber innovation drives a learning innovation of delivering near-real-time data on teaching practices in a combined reflection and training system by delivering rapid and frequent feedback and instruction on good strategies in manageable instructional units, that support a focus on student-centered beliefs. In turn, the learning innovation advances understanding of how instructors learn in technology-rich learning environments by exploring mechanisms in a framework of professional development that would not be possible without this new cyberlearning genre. In particular, through a series of design-based research studies with instructors teaching STEM college courses, the researchers explore ways in which Sensei a) can trigger critical self reflection, b) how this self-reflection changes based on the features of the data viewed, c) how datadriven goal-setting can foster self-efficacy in teaching, and d) how these effects vary over time. All of the code will be developed as open source and, if successful, Sensei could be generalized to include K-12 teachers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

多年来，研究表明，远离大型讲座，增加学生在课堂上的参与度和参与度，可以显著提高学习效果。不幸的是，教授们缺乏高质量的职业发展机会来改进他们的教学，而且通常没有接受过如何教学的培训。该项目通过一种名为Sensei的网络学习创新来解决大学教授的专业发展问题。Sensei具有使用传感器捕获、分离和分析语音和视频的新颖功能，这些功能将提供有关课堂互动的近乎实时的数据，例如学生与教授交谈的百分比、学生与学生交谈的百分比、通过面部表情分析的学生参与度、学生和教授之间的轮换等，所有这些都涉及语音和视频的多模式分析。这项研究的第二个组成部分是制定建议的行动，以提高教授作为教师的表现。更准确地说，Sensei借鉴了传感阵列、计算机视觉、智能环境和个人信息学方面的技术和社会技术进步，以及高等教育中的专业发展框架。在这个项目中，研究人员将1)开发向教师自动感知和显示反馈所需的技术；2)在一系列基于设计的研究的学期中，将该系统在体内部署到大学教师身上，并解释结果；3)重复我们将课堂数据纳入专业发展的常规框架。这项研究是由一项网络创新实现的，其中计算通过最先进的多模式传感方法的能力进行了扩展，以实现课堂规模的非侵入性传感。这一网络创新推动了一项学习创新，通过在可管理的教学单元中提供快速而频繁的反馈和良好策略的指导，在综合反思和培训系统中提供近实时的教学实践数据，从而支持对以学生为中心的信念的关注。反过来，学习创新通过探索专业发展框架中的机制来促进对教师如何在技术丰富的学习环境中学习的理解，如果没有这种新的网络学习流派，这种机制是不可能的。特别是，通过与教授STEM大学课程的教师进行的一系列基于设计的研究，研究人员探索了Sensei a)触发批判性自我反思的方法，b)这种自我反思如何基于所查看的数据的特征，c)数据驱动的目标设置如何在教学中培养自我效能感，以及d)这些影响如何随着时间的推移而变化。所有的代码都将作为开源开发，如果成功，SENSEI可能会被推广到包括K-12教师。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Classroom Digital Twins with Instrumentation-Free Gaze Tracking

• DOI: 10.1145/3411764.3445711
• 发表时间: 2021-05
• 期刊: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
• 作者: Karan Ahuja;Deval Shah;Sujeath Pareddy;Françeska Xhakaj;A. Ogan;Yuvraj Agarwal;Chris Harrison

Investigating Teacher Data Needs In Terms of Teacher Immediacy and Nonverbal Behaviors

从教师即时性和非语言行为方面调查教师数据需求

• 发表时间: 2021
• 期刊: Proceedings of the 15th International Conference of the Learning Sciences - ICLS 2021
• 作者: Xhakaj, Franceska;Ogan, Amy;Lee, Na Young;Ulberg, Erik;Luo, Amy;Lee, Seoyoung;Hu, Katrina
• 通讯作者: Hu, Katrina