CAREER: Data-aware Distributed Computing for Enabling Large-scale Collaborative Science

职业生涯：数据感知分布式计算支持大规模协作科学PI：路易斯安那州立大学Tevfik Kosar摘要所有科学领域的应用和实验都变得越来越复杂，对计算和数据的要求也越来越高。一些应用程序生成的数据量高达数PB。共享、传播和分析这些大型数据集成为一个巨大的挑战，尤其是在使用分布式资源的情况下。该学院早期职业发展(CALEAR)项目提出了一种新的分布式计算范例，称为数据感知分布式计算，它将包括一组不同的算法、模型和工具，用于缓解分布式计算系统中的数据瓶颈；并将支持广泛的数据密集型和动态数据驱动的应用程序。作为该项目的一部分，研究和开发将在三个主要组件上进行：i)数据感知调度器，它将提供数据移动任务的计划、调度、资源保留、作业执行和错误恢复等能力；ii)将这些能力集成到分布式计算中的其他层，如工作流规划、资源代理和存储管理；以及iii)通过动态调整底层协议传输参数来进一步优化数据移动任务。研究将被整合到教育的各个层面，其中将包括与K-12学生(其中99%是少数族裔)有关数据密集型计算的科学项目、研讨会和夏令营；针对本科生和研究生的课程开发、指导和国际学生/实习生交流计划；专门针对HBCU社区(包括教职员工)的暑期实习和研讨会。在这个项目中开发的工具和软件将通过开源分发向公众开放。

CAREER: Data-aware Distributed Computing for EnablingLarge-scale Collaborative SciencePI: Tevfik Kosar, Louisiana State UniversityAbstractApplications and experiments in all areas of science are becoming increasingly complex and more demanding in terms of their computational and data requirements. Some applications generate data volumes reaching petabytes. Sharing, disseminating, and analyzing these large data sets becomes a big challenge, especially when distributed resources are used. This Faculty Early Career Development (CAREER) project proposes a new distributed computing paradigm called ?data-aware distributed computing?, which will include a diverse set of algorithms, models, and tools for mitigating the data bottleneck in distributed computing systems; and will support a broad range of data-intensive as well as dynamic data-driven applications. As part of this project, research and development will be performed on three main components: i) a data-aware scheduler which will provide capabilities such as planning, scheduling, resource reservation, job execution, and error recovery for data movement tasks; ii) integration of these capabilities to the other layers in distributed computing such as workflow planning, resource brokering, and storage management; and iii) further optimization of data movement tasks via dynamically tuning of underlying protocol transfer parameters. Research will be integrated to literally all levels of education which will include science projects, seminars and summer camps on data-intensive computing with K-12 students (where 99% is minority); curriculum development, mentoring, and international student/intern exchange programs for undergraduate and graduate students; summer internships and workshops specifically for HBCU community including faculty members. The tools and software developed in this project will be available to public via open-source distribution.