Teaching the Teacher: Tutoring SimStudent Leads to More Effective Cognitive Tutor Authoring
Teaching the Teacher: Tutoring SimStudent Leads to More Effective Cognitive Tutor Authoring
复制标题
DOI：
10.1007/s40593-014-0020-1
复制
发表时间：
2015-03-01
影响因子：
4.9
通讯作者：
Koedinger, Kenneth R.
中科院分区：
文献类型：
--
作者：
Matsuda, Noboru;Cohen, William W.;Koedinger, Kenneth R.
SimStudent是一个机器学习代理，最初是为了帮助新手作者创建认知导师而开发的，无需繁重的编程。SimStudent集成到一套名为认知导师创作工具（Cognitive Tutor authororing tools， CTAT）的现有软件工具中，通过指导SimStudent如何解决问题，帮助作者为认知导师创建专家模型。用SimStudent创建专家模型有两种不同的方法。在authororing by Tutoring的背景下，作者通过向SimStudent提出问题，对SimStudent执行的步骤提供反馈，并在SimStudent无法正确执行步骤时演示步骤作为对SimStudent提示请求的响应，从而交互式地指导SimStudent。在通过演示进行创作的上下文中，作者演示了解决方案的步骤，SimStudent试图通过概括这些已完成的示例来归纳潜在的领域原则。我们进行了评估研究，以调查哪种创作策略更有利于创作，并发现了两个关键结果。首先，与Authoring by Demonstration生成的专家模型相比，Authoring by Tutoring生成的专家模型在保持相同的完整性水平的同时，具有更好的准确性和更高的准确性。准确性更高的原因是，由辅导生成的专家模型受益于为SimStudent不正确的生产应用程序提供的负反馈。其次，家教写作比示范写作需要更少的时间。这种提高的创作效率部分是因为(a)在通过示范创作时，作者需要测试专家模型的质量，而专家模型的形成性评估是通过观察SimStudent在通过辅导创作时的表现自然完成的，(b)在辅导过程中需要演示的步骤数量随着学习的进展而减少。
SimStudent is a machine-learning agent initially developed to help novice authors to create cognitive tutors without heavy programming. Integrated into an existing suite of software tools called Cognitive Tutor Authoring Tools (CTAT), SimStudent helps authors to create an expert model for a cognitive tutor by tutoring SimStudent on how to solve problems. There are two different ways to author an expert model with SimStudent. In the context of Authoring by Tutoring, the author interactively tutors SimStudent by posing problems to SimStudent, providing feedback on the steps performed by SimStudent, and also demonstrating steps as a response to SimStudent's hint requests when SimStudent cannot perform steps correctly. In the context of Authoring by Demonstration, the author demonstrates solution steps, and SimStudent attempts to induce underlying domain principles by generalizing those worked-out examples. We conducted evaluation studies to investigate which authoring strategy better facilitates authoring and found two key results. First, the expert model generated with Authoring by Tutoring is better and has higher accuracy while maintaining the same level of completeness than the one generated with Authoring by Demonstration. The reason for this better accuracy is that the expert model generated by tutoring benefits from negative feedback provided for SimStudent's incorrect production applications. Second, authoring by Tutoring requires less time than Authoring by Demonstration. This enhanced authoring efficiency is partially because (a) when Authoring by Demonstration, the author needs to test the quality of the expert model, whereas the formative assessment of the expert model is done naturally by observing SimStudent's performance when Authoring by Tutoring, and (b) the number of steps that need to be demonstrated during tutoring decreases as learning progresses.