Assessing the Quality of Research Data Infrastructure Software

科学正在迅速发展，包括自动驾驶汽车、高通量科学仪器、高保真数值模型和传感器网络等技术，所有这些技术产生的数据频率、种类和数量都在不断增加。科学家们对共享这些数据感兴趣（或被迫这样做），这需要研究数据基础设施（RDI：为促进数据共享和消费而组织的数字基础设施，以支持研究工作）。RDI软件通常是自产的：由没有接受过软件工程方面正式培训的人创建和部署，或者由主要任务不是软件开发的组织创建和部署。这些开发者通常也是用户；当他们或他们的同事确定需要时，他们会添加功能。我们对软件工程作为一个领域和实践的理解并不能普遍地转化为这个软件。该软件被越来越多的用户使用，但是没有人系统地评估它的可维护性、寿命、技术债务、社区弹性或其他长期健康指标，也不知道评估这些指标的现有方法是否对RDI软件有效。随着RDI的重要性和复杂性的增长，我们必须对其可靠性和准确性充满信心。我们知道非典型的开发过程可以产生高质量的结果，但是我们也知道RDI软件正在被新的特性所扩展，比如大规模的性能、自动数据清理和可视化。这个项目将对RDI软件的可维护性、技术债务、可持续性和最佳实践的使用应用软件质量度量。对于这些由专业开发人员创建的软件的度量，已经有了广泛的研究。我们将确定最相关的度量和工具，并将其应用于基于海洋科学RDI的案例研究。我们将进行人种学研究，将参数所传达的信息与用户社区和开发者的实际体验进行比较。基于结果，我们将推荐工作良好的度量标准，并产生与现实相符的结果。只有当开发人员意识到使用软件质量度量的好处时，软件质量度量才会对RDI软件的质量产生实际影响。我们将研究如何感知这些度量并将其集成到RDI软件的开发过程中，并与现有的开源、商业和科学软件的研究进行比较。好的科学需要好的数据。只有当我们拥有可维护、可靠和可用的研究数据基础设施时，才能实现可查找、可访问、可互操作和可重用的数据。虽然RDI目前有些支离破碎，但长期愿景是建立一个全球网络，将跨学科和边界的研究数据联系起来。一个必要的前提是确保我们的软件能够完成这项任务。这个项目将对这一需求做出重大贡献，同时也推进了实证软件工程研究。

Science is rapidly evolving, incorporating technology like autonomous vehicles, high-throughput scientific instruments, high-fidelity numerical models, and sensor networks, all generating data with increasing frequency, variety, and volume. Scientists are interested in sharing this data (or compelled to), which requires research data infrastructure (RDI: digital infrastructure organized to promote data sharing and consumption in support of research efforts). RDI software is often homegrown: created and deployed by people who have not received formal training in software engineering, or at organizations with primary mandates other than software development. These developers are often also users; they are adding features as they or their colleagues identify the need. Our understanding of software engineering as a field and practice does not universally translate to this software. This software is used by a growing set of users, but no one has systematically assessed its maintainability, longevity, technical debt, community resilience, or other indicators of long-term health, nor is it known if existing approaches to assessing these metrics are effective for RDI software. As RDI grows in importance and complexity, we must be confident in its reliability and accuracy. We know that atypical development processes can yield high-quality results, but we also know that RDI software is being stretched by new expectations for features like performance at scale, automated data cleaning, and visualization. This project will apply software quality metrics on maintainability, technical debt, sustainability, and the use of best practices to RDI software. There has been extensive research on these metrics for software created by professional developers. We will identify the most relevant metric and tools and apply them to case studies based on RDI for ocean science. We'll conduct ethnographic studies to compare what the metrics tell us with the lived experience of user communities and developers. Based on the results, we'll recommend metrics that work well and produce results that match reality. Software quality metrics will only have a practical impact on the quality of RDI software if developers realize benefit from using them. We will study how these metrics are perceived and integrated into development processes for RDI software, in comparison to existing studies on open source, commercial, and scientific software. Good science needs good data. Findable, Accessible, Interoperable, and Reusable data is only achievable if we have a research data infrastructure that is maintainable, reliable, and useable. While RDI is somewhat fractured at present, the long-term vision is a global network that links research data across disciplines and borders. A necessary precursor is ensuring that our software is up to this task. This project will contribute significantly to this need, while also advancing empirical software engineering research.