Designing and Evaluating a Naturalistic Platform for Collaborative Learning About Spatial Reasonings

空间推理和计算思维是未来科学、技术、工程和数学（STEM）工作者的基本知识。该项目的研究人员将设计和研究一个智能多模式界面，以支持中小学生在基于Minecraft的协作数字平台上学习这些认知技能。新的界面将包括音频和语音、眼球追踪和有形的物理对象。这些人机交互模式的结合可以为更广泛的学习者提供学习支持，并在更自然和富有成效的环境中促进同伴之间的合作。不仅可以通过键盘命令，还可以通过语音，物理块和眼睛注视与计算机进行交互，学习者将有机会在非正式，灵活和可访问的环境中练习和提高他们的空间推理和计算思维技能。更广泛地说，这个项目研究了多模式界面的未来。该项目探索的下一代多模态语音界面的一些关键组件包括解释日常人类语言的能力，以及在协作过程中使用语音或手势表达想法的能力。除了开发新技术外，该项目还研究了如何以促进空间推理和计算思维的方式构建不同的数字学习体验。与伊利诺斯州埃文斯顿公立学区合作，将通过实验室研究和课后学习俱乐部研究来测试与空间推理和计算思维相关的几个假设。该项目由人类技术前沿工作网络学习项目资助，该项目旨在资助学习技术的探索性和协同研究，以帮助学习者在人类技术前沿工作中脱颖而出。该研究项目将调查四个重要的研究问题。(1)多模式界面以何种方式使或促进当前非参与者（尤其是年轻学生、代表性不足群体和女孩）更多地参与课后游戏学习俱乐部？(2)多模态界面如何改变当前学习者的现有行为模式；具体来说，多模态接口以何种方式引入、支持和破坏协作学习和构建实践？(3)这些参与和游戏实践的变化是否与空间思维、计算思维的发展和对计算机职业的追求相关？(4)需要什么样的设计原则、技术和/或算法发展来支持通过多模态接口的协同工作？研究人员将采用迭代设计过程。设计修改将通过混合方法分析视频数据、调查、学习评估、用户活动日志，以及关于学习者如何参与多模式、协作解决问题的多模式感官数据。这项研究有可能为代表性不足的年轻学习者发现通往STEM职业的学习途径。在三年内，约有400名学生将参与并体验拟议的多模式学习系统。由此产生的软件和设计文件将向公众开放源代码，以便接触更多的机构，扩大STEM学习的参与。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Spatial reasoning and computational thinking are essential knowledge for future workers in Science, Technology, Engineering and Mathematics (STEM). Researchers in this project will design and study an intelligent multimodal interface to support elementary and middle school students to learn these cognitive skills on a collaborative digital platform based on Minecraft. The new interface will include audio and speech, gaze with eye tracking, and tangible physical objects. The combination of these human-computer interaction modalities may support learning for a broader diversity of learners and facilitate collaboration between peers in a more naturalistic and productive setting. Enabling interacting with computers not only through keyboard commands but also through speech, physical blocks and eye gaze, learners will have the opportunity to practice and improve their spatial reasoning and computational thinking skills in an informal, flexible and accessible context. More broadly, this project investigates the future of multimodal interfaces. Some of the key components of the next-generation multimodal, voice-enabled interface that this project explores include the ability to interpret everyday human language and to express an idea using speech or gesture during collaboration. In addition to developing new technologies, this project examines how to structure different digital learning experiences in ways that promote spatial reasoning and computational thinking. Several hypotheses related to spatial reasoning and computational thinking will be tested through laboratory studies and afterschool learning club studies in partnership with the Evanston Public School District in Illinois. This project is funded by the Cyberlearning for Work at the Human-Technology Frontier program, which seeks to fund exploratory and synergistic research in learning technologies to prepare learners to excel in work at the human-technology frontier.The research project will investigate four important research questions. (1) In what ways does a multimodal interface enable or promote increased participation in the afterschool game-based learning club among current non-participants, especially younger students, people from underrepresented groups and girls? (2) How might a multimodal interface change existing patterns of behavior among current learners; specifically, in what ways do multimodal interfaces introduce, support, and disrupt collaborative learning and building practices? (3) Do these changes in participation and in-game practices correlate with the development of spatial thinking, computational thinking and the pursuit of computing careers? (4) What design principles and technological and/or algorithmic developments are needed to support collaborative work through multimodal interfaces? Researchers will employ an iterative design process. Design modifications will be informed by mixed-method analyses of video data, surveys, learning assessments, user activity logs, and multimodal sensory data on how learners engage in multimodal, collaborative problem solving . This research has the potential to discover learning pathways to STEM careers for underrepresented young learners. Around 400 students will participate and experience the proposed multimodal learning system during the three years. The resulting software and design documents will be open source for the public in order to reach a much larger number of institutes to broadening participation of STEM learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.