CAREER: Data Intensive Grid Computing on Active Storage Clusters

集群和网格在执行计算密集型工作负载方面取得了巨大的成功。然而，执行数据密集型工作负载已被证明要困难得多：数据服务是大型计算系统中的一个严重瓶颈。为了解决这个问题，集群和网格计算系统必须将计算与数据存储紧密结合起来。根据这一原则，人们正在开发各种新的系统结构。为了提供可扩展的计算和存储，由活动存储单元构建的集群可以作为归档文件系统和批处理计算系统的组合。建立在活动存储上的分布式数据结构可以以协调的方式指导数据和计算的放置。一种新的集群编程语言允许大型工作负载有效地利用分布式数据结构。该项目的结果将是集群和网格处理数据密集型科学工作负载的能力的急剧增加。这个项目将为操作系统和分布式系统的课堂项目提供一个令人兴奋的环境，并且开发的软件将作为“课件”分发给其他教育者使用。这项工作将对各种各样的科学学科产生影响，但将与天体物理学、生物信息学、生物计量学和分子动力学领域的活跃用户密切合作。本项目创建的软件将以开放源代码的形式传播，供更广泛的社区使用。此外，该软件还将被用作通过圣母大学扩展研究社区向高中生和教师提供实时天体物理学数据的工具。

Clusters and grids have achieved great successes in executing computation intensive workloads. However, executing data intensive workloads has proven to be much more difficult: data services are a serious bottleneck in large computing systems. To address this problem, cluster and grid computing systems must closely integrate computation with data storage. A variety of new system structures are being developed according to this principle. To provide scalable computation and storage, a cluster constructed from active storage units can serve as a combined archival file system and batch computing system. Distributed data structures built on active storage can guide data and computation placement in a coordinated manner. A new cluster programming language allows distributed data structures to be efficiently harnessed by large workloads. The result of this project will be a dramatic increase in the capacity of clusters and grids to process data intensive scientific workloads. This project will serve as an exciting environment for class projects in operating systems and distributed systems classes, and the software developed will be distributed as "courseware" for use by other educators. This work will have an impact on a wide variety of scientific disciplines, but will work particularly closely with active users in astrophysics, bioinformatics, biometrics, and molecular dynamics. The software created by this project will be disseminated in open source form for use by the broader community.In addition, this software will also be used as a vehicle for making live astrophysics data accessible to high school students and teachers through the Notre Dame Extended Research Community.