CAREER: Abstractions and Middleware for D3 Science on NSF Distributed Cyberinfrastructure

技术说明：这个职业奖项目将开发中间件，以支持分布式网络基础设施（DCI）上的分布式动态数据密集型（D3）科学。 现有的NSF资助的CI系统，如极端科学和工程发现环境（XSEDE）和开放科学网格（OSG），使用分布式计算，大大增加了计算能力，可供研究科学家在地球仪;然而，这样的分布式系统面临的能力，以处理大数据量正在产生的今天的限制？的科学仪器和模拟。 为了应对这一挑战，PI将开发和部署可扩展的抽象，以促进高性能计算和大规模数据集的集成。 建立在以前的工作试点工作，这些新的抽象将实现类似的概念？试点数据？和联系原则？亲和力？ 其结果将是一个统一的概念框架，以改进数据和计算资源的匹配，并促进动态工作流程的安排和时间安排。 通过生成生产级中间件，在一系列DCI系统上实现可扩展的大数据科学，这项研究有可能极大地推动多个科学和工程领域的发展。更广泛的重要性：科学研究人员可用的高性能计算资源越来越多地分布在多个位置的多台机器上。 这些资源的整合需要一个结构？中间件，？在其上可以构建和运行各种各样的用户应用、工具和服务。 然而，随着越来越精确和无处不在的科学仪器和模型产生越来越大的数据量，这种分布式网络基础设施（DCI）正面临着前所未有的数据处理挑战，这些挑战超出了现有DCI中间件的能力。 在这个项目中，PI将开发、测试和实施新的中间件解决方案，专门为即将到来的大数据分布式超级计算时代而设计。该项目还将开发新的课程和新的教学和推广材料，向中学和大学生、中学教师和公众介绍新兴的分布式数据密集型科学领域。 与FutureGrid合作，PI将设计简单有效的工具，与历史黑人学院和大学（HBCU）以及其他机构共享这些资源，否则教师可能有相对有限的机会开发高级课程材料。 PI还将与道格拉斯妇女科学计划和罗格斯大学工程与技术女孩学院（TARGET）合作，以增加DCI研究界女学生的参与和支持。

Technical Description: This CAREER Award project will develop middleware to support Distributed Dynamic Data-intensive (D3) science on Distributed Cyberinfrastructure (DCI). Existing NSF-funded CI systems, such as the Extreme Science and Engineering Discovery Environment (XSEDE) and the Open Science Grid (OSG), use distributed computing to substantially increase the computational power available to research scientists around the globe; however, such distributed systems face limitations in their ability to handle the large data-volumes being generated by today?s scientific instruments and simulations. To address this challenge, the PI will develop and deploying extensible abstractions that will facilitate the integration of high-performance computing and large-scale data sets. Building on previous work on pilot-jobs, these new abstractions will implement the analogous concept of ?pilot-data? and the linking principle of ?affinity.? The result will be a unified conceptual framework for improving the matching of data and computing resources and for facilitating dynamic workflow placement and scheduling. This research has the potential to significantly advance multiple areas of science and engineering, by generating production-grade middleware for accomplishing scalable big-data science on a range of DCI systems.Broader Importance: Increasingly, the high-performance computing resources available to scientific researchers are distributed across multiple machines in multiple locations. The integration of these resources requires a fabric of ?middleware,? upon which a wide variety of user applications, tools and services can be built and run. As more accurate and ubiquitous scientific instruments and models produce ever-larger volumes of data, however, this distributed cyberinfrastructure (DCI) is confronting unprecedented data-handling challenges that exceed the capabilities of existing DCI middleware. In this project, the PI will develop, test and implement new middleware solutions, specifically designed for the coming era of big-data distributed supercomputing.The project will also develop new curricula and new teaching and outreach materials for introducing secondary and college students, secondary school teachers, and the general public to the emerging field of distributed data-intensive science. In partnership with FutureGrid, the PI will design simple and effective vehicles for sharing these resources with Historically Black Colleges and Universities (HBCUs) and other institutions where faculty might otherwise have relatively limited opportunity to develop advanced course materials. The PI will also partner with the Douglass Program for Women in Science, and The Academy at Rutgers for Girls in Engineering and Technology (TARGET), to increase engagement of, and support for, female students in the DCI research community.