A Proposal for a Hybrid Syllabus Search Tool that Combines Keyword Search and Content Based Classification
A Proposal for a Hybrid Syllabus Search Tool that Combines Keyword Search and Content Based Classification
复制标题
结合关键词搜索和基于内容的分类的混合教学大纲搜索工具的提案
DOI：
10.1109/educon46332.2021.9454140
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Yamaguchi Kazunori
中科院分区：
文献类型：
--
作者：
Sekiya Takayuki;Tatejima Tomohiro;Matsuda Yoshitatsu;Yamaguchi Kazunori
教学大纲是分析教育活动的重要线索之一。我们先前的研究报告了大约47所大学的计算机科学课程大纲可以揭示计算机科学课程的有趣结构。然而，课程大纲是手工收集的。因此，要大幅度增加教学大纲的数量是很困难的，需要对大量的课程教学大纲进行半自动抓取来进行进一步的分析。我们一直在研究使用通用的网络爬虫，根据从学校网站下载的大量网页内容收集教学大纲信息。我们利用线性支持向量机（linear support vector machine，简称linear SVM）在一定程度上自动发现教学大纲页面的结构。我们使用目标CS领域本科教育部门的首页作为每个大学的爬行起始页。为了查找这样一个部门的页面，我们有时使用谷歌搜索。谷歌API期望提供一种有效的方法来收集教学大纲信息，同时节省计算时间、存储和其他资源。在本研究中，我们提出了一种结合谷歌API作为通用关键词搜索引擎和线性支持向量机作为基于内容的分类模型的混合方法。我们开发了一个系统来支持教学大纲的收集过程。该系统由爬虫、分类器和数据库三个子系统组成。爬虫是谷歌API和通用网络爬虫的结合。我们可以使用谷歌API在大学网站上搜索与教学大纲相关的网页，其中包含与教学大纲相关的搜索关键词和网站的域名。分类器根据线性支持向量机的置信度分数对大量网页中与CS教学大纲相关的页面进行排序。我们使用在以前的研究中收集的教学大纲页面来训练线性支持向量机的决策模型。使用谷歌API和线性支持向量机获得的页面，我们可以找到来自更多大学的CS教学大纲页面列表，而不是单独使用每种方法。结合谷歌API结果的前9名和线性支持向量机决策模型的前2名，我们获得了58所大学中96.6%以上的CS教学大纲页面。
A syllabus is one of the most important clues in the analysis of the educational activities. Our previous works reported that the course syllabi of computer science (CS) curricula from about 47 universities can disclose the interesting structures in the CS curricula. However, the course syllabi were collected manually. Therefore, it was difficult to increase the number of syllabi largely, and semi-automatic crawling of massive course syllabi is needed for further analysis. We have been studying to collect syllabus information based on the contents of a large number of web pages downloaded from the university's website by using a general-purpose web crawler. We discovered the structures of the syllabus pages to some extent automatically by using the linear support vector machine (linear SVM). We used the top page of the target department educating bachelor's degree in CS field as a start page of crawling for each university. To look for such a department's page, we sometimes used Google search. Google Custom Search API1,(Google API) is expected to provide an efficient way to gather syllabus information while saving computation time, storage, and other resources. In this study, we propose a hybrid method which combines Google API as a general keyword search engine and linear SVM as content-based classification models. We developed a system to support the syllabus collection process. The system consists of three subsystems: Crawler, Classifier, and Database. Crawler is the combination of Google API and general-purpose web crawler. We can search syllabus-related web pages from university websites using Google API with syllabus-related search keywords and domain names of the websites. Classifier ranks pages related to CS syllabus from a large number of web pages according to the confidence scores of the linear SVM. We trained the decision model of linear SVM using the syllabus pages we collected in the former studies. Using the pages obtained from Google API and linear SVM, we can find a list of CS syllabus pages from more universities than using each method alone. Combining the top nine of Google API results and the top two of linear SVM's decision model, we obtained the CS syllabus pages from more than 96.6% of the 58 universities.