Knowledge Annotation for Intelligent Textbooks
Knowledge Annotation for Intelligent Textbooks
复制标题
DOI：
10.1007/s10758-021-09544-z
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Mengdi Wang;Hung Chau;Khushboo Thaker;Peter Brusilovsky;Daqing He
中科院分区：
文献类型：
--
作者：
Mengdi Wang;Hung Chau;Khushboo Thaker;Peter Brusilovsky;Daqing He
随着电子教科书的日益普及，人们越来越有兴趣开发新一代“智能教科书”，这种教科书能够根据读者的学习目标和现有知识来指导读者。智能教科书通过整合机器可操作的知识来扩展普通教科书，最流行的整合知识类型是教科书中提到的相关概念的列表。利用这些概念，可以执行多种智能操作，例如内容链接、内容推荐或学生建模。然而，现有的自动关键短语提取方法，即使是有监督的方法，也无法提供足够的准确度来在这项任务中发挥实际作用。专家的手动注释已被证明是为训练监督模型生成高质量标记数据的首选方法。然而，教育领域的大多数研究人员仍然认为概念注释过程是一项临时活动，而不是仔细执行的任务，这可能会导致低质量的注释数据。以《信息检索导论》教材中的概念标注为案例，提出一种获取可靠概念标注的知识工程方法。正如我们收集的数据所示，随着我们的过程，注释者间的一致性逐渐增加，并且我们生成的概念注释在文档链接和学生建模任务中带来了更好的结果。我们工作的贡献包括经过验证的知识工程程序、技术概念注释的密码本以及目标教科书的一组概念注释，这些可以作为进一步智能教科书研究的黄金标准。
With the increased popularity of electronic textbooks, there is a growing interest in developing a new generation of “intelligent textbooks,” which have the ability to guide readers according to their learning goals and current knowledge. Intelligent textbooks extend regular textbooks by integrating machine-manipulable knowledge, and the most popular type of integrated knowledge is a list of relevant concepts mentioned in the textbooks. With these concepts, multiple intelligent operations, such as content linking, content recommendation, or student modeling, can be performed. However, existing automatic keyphrase extraction methods, even supervised ones, cannot deliver sufficient accuracy to be practically useful in this task. Manual annotation by experts has been demonstrated to be a preferred approach for producing high-quality labeled data for training supervised models. However, most researchers in the education domain still consider the concept annotation process as an ad-hoc activity rather than a carefully executed task, which can result in low-quality annotated data. Using the annotation of concepts for theIntroduction to Information Retrievaltextbook as a case study, this paper presents a knowledge engineering method to obtain reliable concept annotations. As demonstrated by the data we collected, the inter-annotator agreement gradually increased along with our procedure, and the concept annotations we produced led to better results in document linking and student modeling tasks. The contributions of our work include a validated knowledge engineering procedure, a codebook for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as a gold standard in further intelligent textbook research.