EAGER: Comprehension Assessment via Spoken Dialog

点击翻译按钮获取中文摘要

My Science Tutor (MyST) is an intelligent virtual tutor for elementary school students that has been developed over the last 10 years, with over 13,000 spoken dialog sessions in 8 areas of science. Its goal is to assess student understanding of concepts rather than facts, which is very important to prepare students and the future workforce in STEM. This early-stage, exploratory EAGER project seeks to determine whether the analysis of a new corpora of data could advance the development and use of MyST so that teachers, curriculum developers and researchers could more easily develop automated assessments for new science topics. The approach will apply recent advances in deep learning techniques to assess students' conceptual understanding of science during spoken dialogs with the virtual tutor. The project is motivated by the recent availability of a corpus of examples suitable for training and testing the proposed system. Successful outcomes of the proposed research will result in a novel, robust and portable method for extracting semantic representations from student responses and comparing these to reference statements to determine if conceptual relationships are correctly expressed. The approach has the potential to remove the primary impediments to developing dialog-based assessments: grammar development or topic-specific training. This novel and untested approach is high-risk, but if successful, would have high-payoff, by allowing tutorial developers to only need to provide one example statement for each concept being discussed rather than having to explicitly specify all allowable ways that it could be expressed through development of grammars. This in turn could remove barriers to widespread development of spoken dialog systems and develop automated assessments for new topics using little or no training data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.