BD Spokes: PLANNING: MIDWEST: Cyberinfrastructure to Enhance Data Quality and Support Reproducible Results in Sensor Originated Big Data

今天，许多生物物理和农业科学的科学家利用传感器收集大量数据，包括天气、植物病原体、能源分布和水消耗，并应用数据管理技术来获得额外的知识。使用这类数据的一个关键要求是确保数据没有错误、准确、完整和最新。低质量的数据可能是由各种实际问题引起的，包括设备故障、软件故障或不利的环境条件。然而，对于具有大量生成数据的非常大的传感器系统，手动验证数据质量是不可行的，并且验证延迟可能会导致由于时间条件的变化而无法重新生成的数据丢失。此外，在科学研究方面，如今至关重要的是，研究过程必须是可重复的，以便验证研究结果和发现科学欺诈行为。对于包括传感器的实验来说，研究的重复性尤其具有挑战性，因为如果可能的话，重新创造获取数据的相同物理条件往往是困难的。该项目旨在创建和培育一个多学科社区，重点关注传感器实验的数据质量和研究结果的重现性。将举办两期讲习班，以确定数据质量和研究重现性的基本概念和要求，并确定开发适当的网络基础设施的相关要求。最初的方法将在普渡大学开发的CRIS系统中作为可扩展的CI进行测试，并将在综合试点系统中演示进展情况，并在普渡大学AARE研究和教育设施中进行测试。该项目将导致理解CI必须满足的要求，以便支持传感器也是CI的一部分时的数据质量和研究结果的重复性。这些要求将导致若干领域的科学进步，包括：大数据的数据质量；传感器数据的数据质量评估；传感器实验的管理技术；研究结果的可重复性的扩展元数据；科学工作流系统。该项目还将在使用传感器和提高数据质量方面推动信息在农业应用领域的使用。我们的项目将对任何数据密集型应用程序产生更广泛的影响，特别是以大数据相互关联为特征的应用程序，因为数据相互关系为数据质量提供了有趣的机会。该项目将有助于讨论研究结果的再现性，因为它将展示有助于再现性的传播与信息工具。

Today, scientists within numerous biophysical and agricultural sciences utilize sensors to collect a multitude of data including weather, plant pathogens, energy distribution and water consumption, and apply data management techniques to derive additional knowledge. A critical requirement for the use of such data is to make sure that data do not have errors, are accurate, complete and up to date. Low quality data may be caused by a variety of real world issues including faulty equipment, software faults, or adverse environmental conditions. However manually validating data quality is not viable for very large sensor systems with high volumes of generated data, and delays in validation may result in losing data which is impossible to regenerate due to changing temporal conditions. Also in scientific research it is today critical that research processes be reproducible in order to validate research results and detect scientific frauds. Research reproducibility is particularly challenging for experiments that include sensors because re-creating the same physical conditions in which data has been captured may often be difficult if possible at all. This project aims at creating and fostering a multi-disciplinary community focusing on data quality and research results reproducibility for sensor-based experiments. Two workshops will be organized to define the foundational concepts and requirements for data quality and research reproducibility, and to identify related requirements for the development of suitable cyberinfrastructures (CI). Initial approaches will be tested within the CRIS system, developed at Purdue University, as a scalable CI, and advances will be demonstrated in an integrated pilot system and tested at the Purdue University ACRE research and education facility.The project will result in an understanding of the requirements that a CI must address in order to support data quality and research results reproducibility also when sensors are part of the CI. These requirements will lead to scientific advances in several areas including: data quality for big data; data quality assessment for sensor-originated data; management techniques for sensor-based experiments; extended metadata for research results reproducibility; scientific workflows systems. The project will also result in advances in the use of CI in the area of agriculture applications with respect to the use of sensors and to the quality of data. Our project will have a broader impact on any data-intensive application, especially applications characterized by big interrelated data, as data inter-relationships offer interesting opportunities for data quality. The project will contribute to the discussion on research results reproducibility in that it will show CI tools that can help reproducibility.