Collaborative Research: PPoSS: Planning: Performance Scalability, Trust, and Reproducibility: A Community Roadmap to Robust Science in High-throughput Applications

这个项目的重点是计算科学中的一个关键问题。随着所有领域的科学家越来越依赖高通量应用程序（将多个组件组合成异构系统上日益复杂的多模态工作流），这些应用程序的日益复杂性阻碍了科学家产生可靠结果的能力。该项目招募了一个跨学科的社区，共同定义、设计、实施和使用一套可靠的科学解决方案。在这样做的过程中，社区定义了一个路线图，使高吞吐量的应用程序能够承受和克服不利条件，如异构、不可靠的架构在所有规模，包括极端规模、严格的不确定性测试、无法解释的算法（例如，在机器学习中）和黑盒方法。该项目的新颖之处在于它对从硬件和系统到政策和实践的强大科学发现的高通量应用进行了全面、跨学科的研究。通过三个被称为虚拟世界咖啡馆的虚拟小型研讨会，该项目吸引了校园（通过西班牙裔服务机构计算联盟[CAHSI]、学术科学计算联盟[CASC]和南加州地震中心[SCEC]）、国家实验室和工业界的科学家社区。科学家们参与在初始的高吞吐量应用中定义可扩展性、信任和可重复性；确定一组实验实践，以支持这些应用程序工作流的一致成功进展；通过评估实验实践的通用性和可转移性，并通过识别任何缺失的部分，向强大科学的通用硬件和软件解决方案的愿景迈进；并定义下一代工作流程的研究议程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project is focused on a critical issue in computational science. As scientists in all fields increasingly rely on high-throughput applications (which combine multiple components into increasingly complex multi-modal workflows on heterogeneous systems), the increasing complexities of those applications hinder the scientists’ ability to generate robust results. The project recruits a cross-disciplinary community working together to define, design, implement, and use a set of solutions for robust science. In so doing, the community defines a roadmap that enables high-throughput applications to withstand and overcome adverse conditions such as heterogeneous, unreliable architectures at all scales including extreme scale, rigorous testing under uncertainties, unexplainable algorithms (e.g., in machine learning), and black-box methods. The project’s novelties are its comprehensive, cross-disciplinary study of high-throughput applications for robust scientific discovery from hardware and systems all the way to policies and practices.Through three virtual mini-workshops called virtual world cafes, this project engages a community of scientists at campuses (through the Computing Alliance of Hispanic-Serving Institutions [CAHSI], the Coalition for Academic Scientific Computing [CASC], and the Southern California Earthquake Center [SCEC]), at national laboratories, and in industry. The scientists participate in defining scalability, trust, and reproductivity in an initial set of high-throughput applications; identifying a set of experimental practices that support the in-concert successful progress of these applications’ workflows; advancing towards a vision of general hardware and software solutions for robust science by evaluating the generality and transferability of experimental practices and by identifying any missing parts; and defining a research agenda for the next-generation workflows.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.