CAREER: Data-aware Distributed Computing for Enabling Large-scale Collaborative Science

职业：支持大规模协作科学的数据感知分布式计算i: Tevfik Kosar，路易斯安那州立大学摘要所有科学领域的应用和实验正变得越来越复杂，在计算和数据需求方面要求越来越高。一些应用程序生成的数据量达到pb级。共享、传播和分析这些大型数据集成为一个巨大的挑战，特别是在使用分布式资源时。这个教师早期职业发展（Career）项目提出了一种新的分布式计算范式，叫做？数据感知分布式计算？，它将包括一套不同的算法、模型和工具，用于缓解分布式计算系统中的数据瓶颈；并且将支持广泛的数据密集型以及动态数据驱动的应用程序。作为该项目的一部分，研究和开发将在三个主要组件上进行：i)数据感知调度器，它将为数据移动任务提供规划、调度、资源预留、作业执行和错误恢复等功能；Ii)将这些功能集成到分布式计算的其他层，如工作流规划、资源代理和存储管理；iii)通过动态调整底层协议传输参数进一步优化数据移动任务。研究将被整合到教育的各个层面，包括科学项目、研讨会和面向K-12学生的数据密集型计算夏令营（其中99%是少数民族）；为本科生和研究生提供课程开发、指导和国际学生/实习生交流项目；专门为HBCU社区包括教职员工提供的暑期实习和研讨会。在这个项目中开发的工具和软件将通过开源分发向公众提供。

CAREER: Data-aware Distributed Computing for EnablingLarge-scale Collaborative SciencePI: Tevfik Kosar, Louisiana State UniversityAbstractApplications and experiments in all areas of science are becoming increasingly complex and more demanding in terms of their computational and data requirements. Some applications generate data volumes reaching petabytes. Sharing, disseminating, and analyzing these large data sets becomes a big challenge, especially when distributed resources are used. This Faculty Early Career Development (CAREER) project proposes a new distributed computing paradigm called ?data-aware distributed computing?, which will include a diverse set of algorithms, models, and tools for mitigating the data bottleneck in distributed computing systems; and will support a broad range of data-intensive as well as dynamic data-driven applications. As part of this project, research and development will be performed on three main components: i) a data-aware scheduler which will provide capabilities such as planning, scheduling, resource reservation, job execution, and error recovery for data movement tasks; ii) integration of these capabilities to the other layers in distributed computing such as workflow planning, resource brokering, and storage management; and iii) further optimization of data movement tasks via dynamically tuning of underlying protocol transfer parameters. Research will be integrated to literally all levels of education which will include science projects, seminars and summer camps on data-intensive computing with K-12 students (where 99% is minority); curriculum development, mentoring, and international student/intern exchange programs for undergraduate and graduate students; summer internships and workshops specifically for HBCU community including faculty members. The tools and software developed in this project will be available to public via open-source distribution.