i-learn: empowering engineering learners using visualizations in mixed reality and machine learning ecosystems

在工程教育领域，关于技术学习的重要研究集中在学习者与技术的互动、技术支持的学习材料或最终的学习成果上。关于技术学习对个体认知功能（感知、注意力和认知技能）影响的研究相对较少。对于学习者如何或为何在学习过程中取得不同的结果，仍然存在着知识上的差距。i-learn研究项目通过考虑学习者在执行解决问题的任务时的个人特征，同时与先进技术支持的环境-机器学习（ML）和混合现实（MX）技术进行交互，来弥合分歧。i-learn研究项目的理论和实证方法将提供关于如何通过这些技术的进步增强学习者的认知功能的基础知识，缩小在理解工程教育中个人感知和认知能力多样性方面的差距。用ML技术定制MX来预测个人解决问题的角度利用机会来研究学习者与这些技术的交互是否会提高任务绩效。i-learn创建了基于MX的环境，其中包含ML模型提供的功能，不仅是交互功能，而且是感知动作的通信媒介。通过关注学习认知的组成部分（例如，工作记忆和持续注意力）以及心理模拟和情境意识，本项目（1）使用ML预测建模来探索个体差异对学习者在问题解决任务建模中表现的影响，（2）研究学习者在需要空间和认知能力的问题解决任务中的僵局时刻，及（3）发展及评估一个互动及适应性的MX学习平台，供不同认知及注意力的学生使用时的成效。该项目将提高对如何设计技术以加强技术支持的学习的基本理解。它促进了具身认知，开创了使用ML的个性化学习，并推进了对运动活动对与视觉空间能力相关的操作的影响的见解。研究成果将为适应性学习技术的设计和工程教育教学的定制提供信息。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

In the field of engineering education, significant research on learning with technologies has focused on either the learner’s interaction with the technology, on technology-enabled learning material, or on final learning outcomes. Research on learning with technologies involving the influence of individual cognitive function (perceptual, attentional, and cognitive skills) is relatively unexplored. There remains a gap in knowledge on how or why learners arrive at different results in the learning process. The i-learn research project bridges that divide by considering the learner’s individual characteristics as they execute problem-solving tasks while interacting with the advanced technology-enabled environments – a machine-learning (ML) and mixed-reality (MX) technology. The i-learn research project’s theoretical and empirical methods will provide fundamental knowledge about how learners’ cognitive function can be augmented through the advancements of these technologies, closing the gap in understanding the diversity of individuals’ perceptual and cognitive abilities in engineering education. The perspective of tailoring MX with ML technologies to predict individual problem-solving harnesses opportunities to research whether learners’ interactions with these technologies leads to improved task performance. i-learn creates MX-based environments with features informed by ML models not merely as interactive features but as a medium of communication for perceptive-action. By focusing on components of learning cognition (e.g., working memory and sustained attention) as well as mental simulation and situational awareness, this project (1) uses ML predictive modelling to explore the effects of individual differences on learners’ performance in problem-solving tasks modelling, (2) studies learners’ moments of impasse in problem-solving tasks that demand spatial and cognitive ability, and (3) develops and assesses the effectiveness of an interactive and adaptive MX platform for learning when used by students with different cognitive and attentional abilities. This project will improve the fundamental understanding of how technologies should be designed to enhance technology-enabled learning. It promotes embodied cognition, pioneers individualized learning using ML, and advances insights on the effect of motor activity on operations linked to visuo-spatial abilities. Research outcomes will inform the design of adaptive learning technologies and the customization of instruction in engineering education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.