A transparent and transportable methodology for evaluating Data Linkage software
A transparent and transportable methodology for evaluating Data Linkage software
复制标题
DOI：
10.1016/j.jbi.2011.10.006
复制
发表时间：
2012-02-01
影响因子：
4.5
通讯作者：
Boyd, James
中科院分区：
文献类型：
--
作者：
Ferrante, Anna;Boyd, James
近年来，数据链接活动有了大幅增长。这反映了对链接数据或非链接数据的需求和供应的增长。随着深度学习“服务”利用率的提高，人们越来越需要关于深度学习软件程序和软件包的适用性和性能的公正信息。大量的软件包的评估是罕见的，因为时间和资源的负担放在评估者和需要一个合适的“金标准”的评估dataset.In本文中，我们提出了一个评估方法，克服了一些这些困难。我们的方法涉及到代表性的合成数据的生成和使用;使用预定义的链接策略执行一系列链接;以及使用标准的链接质量指标来评估性能。该方法既透明又便于移植，产生真正可比较的结果。科廷大学的数据链接中心（CDL）在十个DL软件包的评估中使用的方法。它也被用来评估更大的链接系统（不仅仅是包）。该方法提供了一个独特的机会，可以衡量不同业务环境中联系的质量。(C)2011 Elsevier Inc.版权所有© 2016
There has been substantial growth in Data Linkage (DL) activities in recent years. This reflects growth in both the demand for, and the supply of, linked or linkable data. Increased utilisation of DL "services" has brought with it increased need for impartial information about the suitability and performance capabilities of DL software programs and packages.Although evaluations of DL software exist; most have been restricted to the comparison of two or three packages. Evaluations of a large number of packages are rare because of the time and resource burden placed on the evaluators and the need for a suitable "gold standard" evaluation dataset.In this paper we present an evaluation methodology that overcomes a number of these difficulties. Our approach involves the generation and use of representative synthetic data; the execution of a series of linkages using a pre-defined linkage strategy; and the use of standard linkage quality metrics to assess performance. The methodology is both transparent and transportable, producing genuinely comparable results. The methodology was used by the Centre for Data Linkage (CDL) at Curtin University in an evaluation of ten DL software packages. It is also being used to evaluate larger linkage systems (not just packages). The methodology provides a unique opportunity to benchmark the quality of linkages in different operational environments. (C) 2011 Elsevier Inc. All rights reserved,