A transparent and transferable framework for tracking quality information in large datasets.
A transparent and transferable framework for tracking quality information in large datasets.
复制标题
DOI：
10.1371/journal.pone.0112249
复制
发表时间：
2014
期刊：
影响因子：
3.7
通讯作者：
Taylor JR
中科院分区：
文献类型：
--
作者：
Smith DE;Metzger S;Taylor JR
评估数据有效性的能力对于任何调查都是必不可少的，过去对数据质量的人工“观察”评估占主导地位。然而，随着所收集数据的规模不断增加，评估其质量所需的努力也在增加。这一挑战对于自动化数据收集的网络尤其重要，并导致许多质量保证和质量控制分析的自动化。不幸的是，对于大型数据集，对结果数据质量标志的解释可能变得非常具有挑战性。我们已经开发了一个框架来总结数据质量信息，并方便用户的解释。我们的框架包括首先汇编数据质量信息，然后通过两个单独的机制来呈现它;质量报告和质量摘要。质量报告显示特定质量分析的结果，因为它们与单个观察有关，而质量摘要则对每个质量分析进行空间或时间聚合，并提供结果摘要。在质量摘要中包括最终质量标志，其进一步压缩数据质量信息以评估数据产品是否有效。该框架具有额外的灵活性，允许将关于数据质量的信息“关注”到许多数据类型中。此外，如果传感器或系统出现故障，该框架可以帮助跟踪和解决问题。
The ability to evaluate the validity of data is essential to any investigation, and manual “eyes on” assessments of data quality have dominated in the past. Yet, as the size of collected data continues to increase, so does the effort required to assess their quality. This challenge is of particular concern for networks that automate their data collection, and has resulted in the automation of many quality assurance and quality control analyses. Unfortunately, the interpretation of the resulting data quality flags can become quite challenging with large data sets. We have developed a framework to summarize data quality information and facilitate interpretation by the user. Our framework consists of first compiling data quality information and then presenting it through 2 separate mechanisms; a quality report and a quality summary. The quality report presents the results of specific quality analyses as they relate to individual observations, while the quality summary takes a spatial or temporal aggregate of each quality analysis and provides a summary of the results. Included in the quality summary is a final quality flag, which further condenses data quality information to assess whether a data product is valid or not. This framework has the added flexibility to allow “eyes on” information on data quality to be incorporated for many data types. Furthermore, this framework can aid problem tracking and resolution, should sensor or system malfunctions arise.
影响因子：
6.2
作者：
Göckede, M;Rebmann, C;Foken, T
通讯作者：
Foken, T
影响因子：
6.2
作者：
Mauder, Matthias;Cuntz, Matthias;Steinbrecher, Rainer
通讯作者：
Steinbrecher, Rainer
影响因子：
4.9
作者：
Taylor, J. R.;Loescher, H. L.
通讯作者：
Loescher, H. L.