EAGER: Comprehension Assessment via Spoken Dialog

My Science Tutor(MyST)是一个面向小学生的智能虚拟导师，在过去的10年里已经开发出来，在8个科学领域有超过13,000个口语对话。它的目标是评估学生对概念的理解，而不是事实，这对学生和未来STEM劳动力的准备非常重要。这个处于早期阶段、渴望探索的项目试图确定对新数据语料库的分析是否可以促进myst的开发和使用，以便教师、课程开发人员和研究人员可以更容易地为新的科学主题开发自动化评估。该方法将应用深度学习技术的最新进展，在与虚拟导师的口头对话中评估学生对科学的概念理解。该项目的动机是最近提供了适用于培训和测试拟议系统的范例语料库。这项研究的成功结果将产生一种新颖、稳健和可移植的方法，用于从学生的回答中提取语义表征，并将这些表征与参考陈述进行比较，以确定是否正确地表达了概念关系。这种方法有可能消除发展以对话为基础的评估的主要障碍：语法发展或专题培训。这种新颖且未经测试的方法风险很高，但如果成功，将会有很高的回报，因为它允许教程开发人员只需要为正在讨论的每个概念提供一条示例语句，而不必明确指定通过开发语法来表达它的所有允许的方式。这反过来可以消除口语对话系统广泛发展的障碍，并使用很少或没有训练数据来开发新主题的自动评估。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

My Science Tutor (MyST) is an intelligent virtual tutor for elementary school students that has been developed over the last 10 years, with over 13,000 spoken dialog sessions in 8 areas of science. Its goal is to assess student understanding of concepts rather than facts, which is very important to prepare students and the future workforce in STEM. This early-stage, exploratory EAGER project seeks to determine whether the analysis of a new corpora of data could advance the development and use of MyST so that teachers, curriculum developers and researchers could more easily develop automated assessments for new science topics. The approach will apply recent advances in deep learning techniques to assess students' conceptual understanding of science during spoken dialogs with the virtual tutor. The project is motivated by the recent availability of a corpus of examples suitable for training and testing the proposed system. Successful outcomes of the proposed research will result in a novel, robust and portable method for extracting semantic representations from student responses and comparing these to reference statements to determine if conceptual relationships are correctly expressed. The approach has the potential to remove the primary impediments to developing dialog-based assessments: grammar development or topic-specific training. This novel and untested approach is high-risk, but if successful, would have high-payoff, by allowing tutorial developers to only need to provide one example statement for each concept being discussed rather than having to explicitly specify all allowable ways that it could be expressed through development of grammars. This in turn could remove barriers to widespread development of spoken dialog systems and develop automated assessments for new topics using little or no training data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.