CAREER: Data Intensive Grid Computing on Active Storage Clusters

集群和网格在执行计算密集型工作负载方面取得了巨大的成功。然而，事实证明，执行数据密集型工作负载要困难得多：数据服务是大型计算系统中的严重瓶颈。为了解决这个问题，集群和网格计算系统必须将计算与数据存储紧密结合起来。根据这一原则，正在开发各种新的系统结构。为了提供可伸缩的计算和存储，由主动存储单元构造的集群可以用作组合的归档文件系统和批处理计算系统。构建在主动存储上的分布式数据结构可以以协调的方式指导数据和计算的放置。一种新的集群编程语言允许分布式数据结构有效地利用大型工作负载。该项目的结果将是集群和网格处理数据密集型科学工作量的能力大幅提高。这个项目将作为一个令人兴奋的环境，类项目在操作系统和分布式系统类，和软件开发将被分发为“课件”供其他教育工作者使用。 这项工作将对各种科学学科产生影响，但将与天体物理学，生物信息学，生物统计学和分子动力学的活跃用户密切合作。该项目开发的软件将以开放源代码的形式传播，供更广泛的社区使用，此外，该软件还将作为一种工具，通过圣母大学扩展研究社区向高中学生和教师提供实时天体物理学数据。

Clusters and grids have achieved great successes in executing computation intensive workloads. However, executing data intensive workloads has proven to be much more difficult: data services are a serious bottleneck in large computing systems. To address this problem, cluster and grid computing systems must closely integrate computation with data storage. A variety of new system structures are being developed according to this principle. To provide scalable computation and storage, a cluster constructed from active storage units can serve as a combined archival file system and batch computing system. Distributed data structures built on active storage can guide data and computation placement in a coordinated manner. A new cluster programming language allows distributed data structures to be efficiently harnessed by large workloads. The result of this project will be a dramatic increase in the capacity of clusters and grids to process data intensive scientific workloads. This project will serve as an exciting environment for class projects in operating systems and distributed systems classes, and the software developed will be distributed as "courseware" for use by other educators. This work will have an impact on a wide variety of scientific disciplines, but will work particularly closely with active users in astrophysics, bioinformatics, biometrics, and molecular dynamics. The software created by this project will be disseminated in open source form for use by the broader community.In addition, this software will also be used as a vehicle for making live astrophysics data accessible to high school students and teachers through the Notre Dame Extended Research Community.