The Development of a Search Engine for Academic Papers in Web

点击翻译按钮获取中文摘要

Open access scientific papers available on the Web could be searched through several search engines. For example, Google scholar has higher coverage of literature, although it does not necessarily guarantee free access to full text. We have developed and evaluated the "Aletheia" search engine for full text academic papers. The system obtains PDF files on a broad range of topics and automatically detects academic papers using classifiers based on text and structure features. We have built PDF database collection containing 3 million Japanese PDF files, five types of Weka classifiers(AdaBoost, Decision Tree(C4. 5), Naive Bayes, Random Forest, and Support Vector Machine) were separately trained for 20, 000 test collection using 10-fold cross-validation to automatically detect academic papers. The features were generated using hand-built rules and consisted by the three types of features : structure, URL, and content.