In-Situ Labeling for Augmented Reality Language Learning
In-Situ Labeling for Augmented Reality Language Learning
复制标题
DOI：
10.1109/vr.2019.8798358
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Brandon Huynh;J. Orlosky;Tobias Höllerer
中科院分区：
文献类型：
--
作者：
Brandon Huynh;J. Orlosky;Tobias Höllerer
增强现实是一种有前途的学习应用交互范例。它有潜力通过将教育内容与学习者日常环境中的空间线索和语义相关对象相结合来提高学习成果。这种界面的影响可以与基因座法相媲美，这是一种众所周知的记忆冠军和多语言者使用的记忆增强技术。然而，由于多种原因，以这种方式使用增强现实仍然不切实际。可扩展的对象识别和对象的一致标记是一个重大挑战，因此 AR 场景中与任意（未建模）物理对象的交互尚未得到很好的探索。为了帮助应对这些挑战，我们提出了一个增强现实中的原位对象标记和选择框架，特别关注语言学习应用程序。我们的框架使用广义对象识别模型来实时识别世界中的对象，集成眼动追踪以促进界面内的选择和交互，并结合动态适应学生成长的个性化学习模型。我们展示了该系统目前的开发进展，包括初步测试和基准测试。我们探讨了在实践中使用此类系统的挑战，并讨论了我们对 AR 语言学习应用的未来愿景。
Augmented Reality is a promising interaction paradigm for learning applications. It has the potential to improve learning outcomes by merging educational content with spatial cues and semantically relevant objects within a learner's everyday environment. The impact of such an interface could be comparable to the method of loci, a well known memory enhancement technique used by memory champions and polyglots. However, using Augmented Reality in this manner is still impractical for a number of reasons. Scalable object recognition and consistent labeling of objects is a significant challenge, and interaction with arbitrary (unmodeled) physical objects in AR scenes has consequently not been well explored. To help address these challenges, we present a framework for in-situ object labeling and selection in Augmented Reality, with a particular focus on language learning applications. Our framework uses a generalized object recognition model to identify objects in the world in real time, integrates eye tracking to facilitate selection and interaction within the interface, and incorporates a personalized learning model that dynamically adapts to student's growth. We show our current progress in the development of this system, including preliminary tests and benchmarks. We explore challenges with using such a system in practice, and discuss our vision for the future of AR language learning applications.