Collaborative Research: Coupling Conversational Interactivity with Multimedia in Support of Physics Learning

已经使用了各种界面来使学生能够与学习技术交互。除了点击，文本和各种媒体也是常用的。然而，这些技术很少尝试提供类似于两个人之间的对话的界面。在卡内基梅隆人机交互研究所工作的基础上，我们将这种对话界面应用于加强物理教学，并将其纳入物理教学网络咨询(Path)，这是一个已被证明在物理教师教育中有效的环境。该项目正在扩展这一独特的学习环境；在该环境中正式研究学生的学习；并通过这项研究，建立专门针对这种新的计算机与学生交互模式的设计原则。CMU的合成访谈(SI)通过一个人的视频和高度可靠的自然语言界面相结合，为用户提供对话互动的感觉。在这个项目中，SI系统将与视频、文本、动画、插图和模拟等其他媒体紧密集成。结果会给学生一个人的印象，一个人直接与他们交谈，并用高度互动的插图媒体解释复杂的概念。我们的假设是，SI与其他交互式多媒体的结合将创建一个系统，将提高学生对物理主题的理解。这一假设将在该项目的研究部分得到严格检验。人们可以想象，当SI软件与屏幕上的其他演示相结合时，学习可以进行的各种方式。屏幕上的人可以参考其他对象，指导学生在继续任何其他努力之前与这些对象交互，并直接帮助学生进行他/她的学习。因此，学习环境受到系统的严格控制。控制也可能相当宽松。屏幕上的人可以推荐其他学习演示文稿，但不要求他们采取行动，或者简单地指出他们是可用的。目前，还没有用于这种类型的交互，特别是包括对话模式的软件。因此，没有研究可以帮助学习技术的开发人员了解哪种类型的耦合最有利于改进学习。这项研究将探讨哪种类型的整合最有效地为学生学习物理概念提供支架，鼓励学生反思自己的学习，并促进知识向物理学习的转移。

A variety of interfaces have been used to enable students to interact with learning technologies. In addition to point and click, text and various media are in common use. However, these technologies seldom attempt to provide an interface that is similar to a conversation between two people. Building on work at the Carnegie Mellon Human-Computer Interaction Institute, we have applied such a conversational interface to the enhancement of physics teaching and incorporated it into the Physics Teaching Web Advisory (Pathway), an environment which has proven effective in physics teacher education. This project is extending this unique learning environment; formally studying student learning within the environment; and establishing, through this research, design principles which are specifically focused on this new mode of computer-student interaction. CMU's Synthetic Interview (SI) provides the user with the feeling of a conversational interaction through a combination of a video of a person and a highly reliable natural language interface. During this project the SI system will be tightly integrated with other media such as video, text, animations, illustrations, and simulations. The result will give the students the impression of a person who is talking directly to them and explaining complex concepts with highly interactive illustrative media. Our hypothesis is that the combination of the SI with other interactive multimedia will create a system that will improve student understanding of topics in physics. This hypothesis will be rigorously tested in the research component of this project. One can imagine a variety of ways in which learning can proceed when the SI software is coupled to other presentations on the screen. The on-screen person can refer to other objects, direct a student to interact with these objects before continuing with any other effort, and help the student directly with his/her learning. Thus, the learning environment is tightly controlled by the system. The control could also be rather loose. The on-screen person can recommend but not require actions with other learning presentations or simply point out that they are available. At present, software for this type of interaction, particularly including the conversational mode, is not available. Thus, no research is available to help developers of learning technologies know what type of coupling is best for improvement of learning. This research will investigate what types of integrations are most effective in providing scaffolding for student learning of physics concepts, for encouraging students to reflect on their learning, and for enhancing transfer of knowledge to the learning of physics.