Measuring effectiveness of ontology debugging systems
Measuring effectiveness of ontology debugging systems
复制标题
测量本体调试系统的有效性
DOI：
10.1016/j.knosys.2014.07.023
复制
发表时间：
2014-11-01
影响因子：
8.8
通讯作者：
Zhu, Man
中科院分区：
文献类型：
--
作者：
Ji, Qiu;Gao, Zhiqiang;Zhu, Man
本体调试旨在为用户提供 OWL 本体蕴含的理由。迄今为止，已经提出了多种本体调试算法，并且有多种本体调试系统可供使用。已经有一些以效率作为主要评价指标来评估这些系统的工作。然而，现有系统可能无法在时限内找到必然的所有理由，并且可能返回不正确的理由。因此，通过考虑理由的正确性和一组已发现理由的完整性来衡量其有效性是有帮助的。在本文中，我们首先对现有的本体调试方法和系统进行了调查。然后，我们基于大量不同的本体来评估现有本体调试系统的有效性和效率。为了评估本体调试系统的有效性，我们首先提出了一种构建参考理由集并定义系统的正确性和完整性程度的方法。然后我们构建了一个包含 80 个大小和表达能力显着不同的本体的数据集。基于提出的评估措施和构建的数据集，我们进行了全面的实验。结果表明现有本体调试系统在正确性、完整性和效率方面的优缺点。根据结果，我们为用户选择合适的本体调试系统、开发者设计本体调试算法和构建本体调试系统提供了一些建议。 (C) 2014 Elsevier B.V. 保留所有权利。
Ontology debugging aims to provide users with justifications for an entailment in OWL ontologies. So far, many ontology debugging algorithms have been proposed and several ontology debugging systems are available. There has been some work on evaluating these systems with the efficiency as the main evaluation measure. However, existing systems may fail to find all justifications for an entailment within a time limit and may return incorrect justifications. Therefore, measuring their effectiveness by considering the correctness of justifications and the completeness of a found set of justifications is helpful. In this paper, we first give a survey of existing ontology debugging approaches and systems. We then evaluate both the effectiveness and the efficiency of existing ontology debugging systems based on a large collection of diverse ontologies. To assess the effectiveness of an ontology debugging system, we first propose a method to construct the reference justification sets and define the degrees of correctness and completeness of the system. Then we construct a dataset containing 80 ontologies with significantly different sizes and expressivities. Based on the proposed evaluation measures and the constructed dataset, we do comprehensive experiments. The results show the advantages and disadvantages of existing ontology debugging systems in terms of correctness, completeness and efficiency. Based on the results, we provide several suggestions for users to choose an appropriate ontology debugging system and for developers to design an ontology debugging algorithm and build an ontology debugging system. (C) 2014 Elsevier B.V. All rights reserved.