The effect of data cleaning on record linkage quality.
The effect of data cleaning on record linkage quality.
复制标题
DOI：
10.1186/1472-6947-13-64
复制
发表时间：
2013-06-05
影响因子：
3.5
通讯作者：
Semmens JB
中科院分区：
文献类型：
--
作者：
Randall SM;Ferrante AM;Boyd JH;Semmens JB
在记录链接领域，采用了大量的数据清理和标准化技术来确保链接的最高质量。虽然这些设施在记录链接软件包中很常见，并且定期部署在记录链接单元中，但很少有工作发表来证明数据清理对链接质量的影响。对综合生成的数据集和先前链接到高标准的大型管理数据集应用了一系列清理技术。使用成对 F 测量来研究这些变化对连锁质量的影响以确定质量。数据清洗对整体链接质量影响不大，大量清洗会导致质量下降。进一步的检查表明，链接质量的下降是由于清理技术通常会减少变异性——尽管现在正确的记录更有可能匹配，但不正确的记录也更有可能匹配，并且这些不正确的匹配超过了正确的匹配，从而降低了整体质量。数据清理技术对链接质量的影响很小。在数据清理过程中应小心。
Within the field of record linkage, numerous data cleaning and standardisation techniques are employed to ensure the highest quality of links. While these facilities are common in record linkage software packages and are regularly deployed across record linkage units, little work has been published demonstrating the impact of data cleaning on linkage quality. A range of cleaning techniques was applied to both a synthetically generated dataset and a large administrative dataset previously linked to a high standard. The effect of these changes on linkage quality was investigated using pairwise F-measure to determine quality. Data cleaning made little difference to the overall linkage quality, with heavy cleaning leading to a decrease in quality. Further examination showed that decreases in linkage quality were due to cleaning techniques typically reducing the variability – although correct records were now more likely to match, incorrect records were also more likely to match, and these incorrect matches outweighed the correct matches, reducing quality overall. Data cleaning techniques have minimal effect on linkage quality. Care should be taken during the data cleaning process.
登录
查看更多内容
DOI：
10.1016/0010-4809(81)90004-5
发表时间：
1981-01-01
期刊：
COMPUTERS AND BIOMEDICAL RESEARCH
影响因子：
--
作者：
HOWE, GR;LINDSAY, J
通讯作者：
LINDSAY, J
影响因子：
4.5
作者：
Ferrante, Anna;Boyd, James
通讯作者：
Boyd, James
影响因子：
2
作者：
Gomatam, S;Carter, R;Mitchell, G
通讯作者：
Mitchell, G
影响因子：
3.7
作者：
JARO, MA
通讯作者：
JARO, MA
影响因子：
3
作者：
Campbell, Kevin M;Deck, Dennis;Krupski, Antoinette
通讯作者：
Krupski, Antoinette