Automated quality control methods for sensor data: a novel observatory approach
Automated quality control methods for sensor data: a novel observatory approach
复制标题
DOI：
10.5194/bg-10-4957-2013
复制
发表时间：
2013-01-01
期刊：
影响因子：
4.9
通讯作者：
Loescher, H. L.
中科院分区：
文献类型：
--
作者：
Taylor, J. R.;Loescher, H. L.
国家和国际陆基传感器网络和观测站正在迅速出现。因此，需要一种标准化的方法来控制数据质量，以及传感器网络之间的数据互操作性。美国国家生态观测站网络(NEON)已经开始建设他们的第一个陆地观测点，预计到2017年将有60个观测点分布在全美各地。这将导致14000多个自动化传感器每年记录超过100TB的数据。然后，这些数据被用来创建其他数据集和后续的“更高级别”数据产品。为应对这一挑战，已制定了一项全面的数据质量保证计划，并确定了第一套数据质量控制措施。这种数据驱动的方法侧重于定义一套似是而非测试参数阈值的自动化方法。具体地说，这些可信测试通过使用一套二进制检查来仔细检查每种测量类型的数据范围和方差。建立了这些测试的统计基础，并探讨了计算测试参数阈值的方法。虽然这些测试在其他地方已经被使用，但我们通过计算它们的相关测试参数阈值来应用它们在一种新的方法中。最后，以霓虹灯样机现场的初步数据为例，说明了如何实现自动化质量控制。
National and international networks and observatories of terrestrial-based sensors are emerging rapidly. As such, there is demand for a standardized approach to data quality control, as well as interoperability of data among sensor networks. The National Ecological Observatory Network (NEON) has begun constructing their first terrestrial observing sites, with 60 locations expected to be distributed across the US by 2017. This will result in over 14 000 automated sensors recording more than > 100 Tb of data per year. These data are then used to create other datasets and subsequent "higher-level" data products. In anticipation of this challenge, an overall data quality assurance plan has been developed and the first suite of data quality control measures defined. This data-driven approach focuses on automated methods for defining a suite of plausibility test parameter thresholds. Specifically, these plausibility tests scrutinize the data range and variance of each measurement type by employing a suite of binary checks. The statistical basis for each of these tests is developed, and the methods for calculating test parameter thresholds are explored here. While these tests have been used elsewhere, we apply them in a novel approach by calculating their relevant test parameter thresholds. Finally, implementing automated quality control is demonstrated with preliminary data from a NEON prototype site.