SCI: ETF Grid Infrastructure Group: Providing System Management and Integration for the TeraGrid

可扩展兆级设施（ETF）是NSF大规模网络基础设施发展的下一个阶段，旨在实现高端计算研究。ETF使研究人员能够利用9个资源合作伙伴的综合资源、数据收集、仪器和可视化能力来解决最具挑战性的计算问题。2004年10月1日，ETF结束了为期三年的建设工作，创建了这个称为TeraGrid （TG）的分布式环境，我们现在正在进入生产操作阶段。TeraGrid资源合作伙伴包括：芝加哥大学/阿贡国家实验室、加州大学圣地亚哥分校的圣地亚哥超级计算机中心、德克萨斯大学奥斯汀分校的德克萨斯高级计算中心、UIUC的国家超级计算应用中心、印第安纳大学、普渡大学、橡树岭国家实验室和匹兹堡超级计算中心。2004年10月19日，美国国家科学基金会（NSF）为TeraGrid电网基础设施集团（GIG）提交了一份单独的提案。在UC/ANL的Charlie Catlett的指导下，GIG将负责协调TeraGrid的开发活动，并将分包合同分包给合作伙伴。资源合作伙伴（RP）将分别与NSF签订独立的合作协议，但将与GIG密切合作，以实现TeraGrid的愿景。该提案概述了网格基础设施组（GIG）计划作为TeraGrid团队中的系统管理和集成组参与其中，为资源合作伙伴和科学界提供对该计算科学设施的持续访问。该提案涵盖2004年11月1日至2009年10月31日期间。世界一流的网络、计算和存储基础设施TeraGrid已经建成并部署。这项计划现在面临着进一步吸引科学和工程社区的挑战，以指导这种通用基础设施的剪裁，以更好地支持他们的需求，催化新的发现并扩大计算科学的基础。TeraGrid整合了一些国家最强大的资源，为科学界提供高性能的生产服务。此外，NSF通过NSF中间件计划（NMI）支持通用软件，并通过其信息技术研究（ITR）项目支持社区特定的基础设施。TeraGrid电网基础设施集团（GIG）将在这些基础上扩大从网络基础设施中受益的社区，并加强和深化TeraGrid的独特能力。GIG与16个科学伙伴合作，制定了基础设施优先事项，以简化即使在当今的网络基础设施下仍然困难（或不可行的）的研究模式。例如，TeraGrid的目标是使以下频繁请求但目前困难的任务常规化：以集成和自动的方式跨TeraGrid和其他网格中的较小规模资源驱动具有多个计算和数据访问步骤的复杂工作流。2. 以按需模式利用TeraGrid资源，为从天气到医疗等时间关键事件提供计算决策支持。3. 通过创建资源代理来优化周转、成本和利用率，这些资源代理提供了基于资源可用性信息跨所有TeraGrid资源调度计算和数据管理任务的单一访问点。已经提出了一个五年路线图。但是认识到用户需求会随着科学机遇的发展而不断演变，计划每年在一系列不断扩大的科学伙伴讨论的基础上重新评估这一路线图。TeraGrid将鼓励科学界利用这一资源来解决几乎所有科学学科中最重要的计算问题。基础设施和社区驱动的网格服务桥梁和门户，被称为科学网关，将为大量迄今尚未使用过NSF高性能计算资源的科学家带来更高的生产力。当前和计划中的TeraGrid用户所针对的问题是现代科学中计算最密集的领域之一，并且代表了无法通过较小规模的网格环境或独立的超级计算机中心有效解决的一类问题。利用软件和基础设施合作伙伴，TeraGrid将制定支持国际网络基础设施所需的软件、安全和资源共享政策。TeraGrid、NMI、ITR项目和特定学科的基础设施项目将被整合，从而形成一个连贯的网络基础设施。该网络基础设施将提供通用软件组件，并使用TeraGrid网络作为国家网格资源背板，通过科学网关和与其他网格的合作，接触到数千名科学家。与软件合作伙伴一起，电网基础设施集团打算制定一套政策和软件，这些政策和软件将被其他电网项目广泛使用，着眼于在本十年末之后维持基础设施。GIG将协调九个TeraGrid资源提供商站点的教育推广和培训（EOT）计划，以支持网络基础设施的广泛EOT计划。我们为TeraGrid用户社区的增长设定了量化目标：到2009财年达到5000名用户。为了使所有TeraGrid用户都能使用，GIG解决了异构性和独特的国家资源和高可用性生产服务的政策要求，开发了跨这些异构资源的协调软件环境和强大的验证和确认系统。作为TeraGrid的补充，面向社区的ITR项目，如网格物理网络（GriPhyN）和大气发现链接环境（LEAD），正在解决许多部门规模资源聚集所需的扩展和软件打包能力。类似地，美国能源部通过SciDAC倡议的计算科学项目，以及美国国立卫生研究院通过NIH路线图的计算科学项目，也是网络基础设施景观的重要组成部分。此外，我们在欧洲、亚太地区和其他地方的合作伙伴正在建设科学网格基础设施项目，如英国电子科学计划、欧洲电子科学网格（EGEE）和日本国家研究网格计划（NAREGI）。TeraGrid将与这些和其他网格项目、NSF的核心中心计划以及软件供应商（如NMI网格中心）合作，以催化一个具有跨机构和国际影响的综合NSF网络基础设施计划。

The Extensible Terascale Facility (ETF) is the next stage in the evolution of NSF large-scale cyberinfrastructure for enabling high-end computational research. The ETF enables researchers to address the most challenging computational problems by utilizing the integrated resources, data collections, instruments and visualization capabilities of nine resource partners. On October 1, 2004, the ETF concluded a three-year construction effort to create this distributed environment called the TeraGrid (TG) and we are now entering the production operations phase.The TeraGrid resource partners include: the University of Chicago/Argonne National Laboratory, the San Diego Supercomputer Center at UCSD, the Texas Advanced Computing Center at UT-Austin, the National Center for Supercomputing Applications at UIUC, Indiana University, Purdue University, Oak Ridge National Laboratory, and the Pittsburgh Supercomputing Center.A separate proposal was submitted to NSF on October 19, 2004 for the TeraGrid Grid Infrastructure Group (GIG). Under the direction of Charlie Catlett at UC/ANL, in general, the GIG will be responsible for coordination of development activities for the TeraGrid with subcontracts to the partner sites. The resource partners (RP) will each have independent cooperative agreements with NSF, but will work closely with the GIG to implement the vision of the TeraGrid.This proposal outlines the Grid Infrastructure Group (GIG) plans to participate as a System Management and Integration Group within the TeraGrid team to provide the Resource Partners and the scientific community with ongoing access to this computational science facility. This proposal covers the period November 1, 2004 through October 31, 2009.TeraGrid, a world-class networking, computing, and storage infrastructure has been built and deployed. This initiative now faces the challenge of further engaging the science and engineering community to guide the tailoring of this generic infrastructure to better support their needs, catalyzing new discoveries and broadening the base of computational science. TeraGrid integrates some of the nation's most powerful resources to provide high-capability production services to the scientific community. In addition, NSF supports common software through the NSF Middleware Initiative (NMI) and community-specific infrastructure through its Information Technology Research (ITR) projects. The TeraGrid Grid Infrastructure Group (GIG) will build on these foundations to broaden the community benefiting from cyberinfrastructure and to harden and deepen TeraGrid's unique capabilities. Collaborating with 16 science partners, the GIG has developed infrastructure priorities to simplify research modalities that remain difficult (or infeasible), even with today's cyberinfrastructure. For example, the TeraGrid aims to make routine the following frequently requested, but currently difficult tasks: 1. Drive complex workflows with multiple computational and data access steps across TeraGrid and smaller scale resources in other Grids in an integrated and automatic manner. 2. Harness TeraGrid resources in an on-demand mode, to provide computational decision-support for time-critical events ranging from weather to medical treatment. 3. Optimize turnaround, costs, and utilization by creating resource brokers that present a single point of access to schedule computational and data management tasks across all TeraGrid resources based on resource availability information. A five-year roadmap has been presented. But recognizing that user needs continue to evolve in response to scientific opportunities, it is planned to reevaluate this roadmap annually based on a widening set of science partner discussions. TeraGrid will encourage the scientific community to leverage this resource to tackle the most important computational problems in virtually every scientific discipline. The infrastructure and community-driven grid service bridges and portals, which are called science gateways, will bring increased productivity to a large numbers of scientists who have not heretofore used NSF's high-performance computing resources. The problems targeted by current and planned TeraGrid users are among the most computationally intensive areas for modern science and represent a class of problems that cannot be addressed effectively by either smaller-scale grid environments or stand-alone supercomputer centers. Leveraging software and infrastructure partners, the TeraGrid will develop policy for software, security, and resource sharing necessary to underpin international cyberinfrastructure. TeraGrid, NMI, ITR projects, and discipline-specific infrastructure projects will be integrated, thus, forming a coherent cyberinfrastructure. This cyberinfrastructure will provide common software components and use the TeraGrid network as a national grid resource backplane, reaching thousands of scientists through science gateways and collaboration with other grids. Working with the software partners, the Grid Infrastructure Group intend to develop a set of policies and software that will be widely used by other grid projects, with an eye toward sustaining infrastructure beyond the end of this decade. The GIG will coordinate education outreach and training (EOT) initiatives across the nine TeraGrid resource provider sites to support a broad EOT program for cyberinfrastructure. We have set quantitative objectives for growing the TeraGrid user community by an order of magnitude: to 5,000 users by FY09. To empower all TeraGrid users, the GIG has addressed heterogeneity and the policy requirements for unique national resources and high-availability production services, developing a coordinated software environment across these heterogeneous resources and a powerful verification and validation system. In a complementary approach to TeraGrid, community-oriented ITR projects such as the Grid Physics Network (GriPhyN ) and Linked Environments for Atmospheric Discovery (LEAD are addressing scaling and software packaging capabilities necessary for aggregation of many departmental-scale resources. Similarly, computational science projects at DOE, via the SciDAC initiative, and at NIH, via the NIH Roadmap, are also important components of the cyberinfrastructure landscape. Moreover, our collaborators in Europe, Asia-Pacific, and elsewhere are building scientific grid infrastructure in projects such as, the UK eScience Programme ], Enabling Grids for E-Science in Europe (EGEE), and Japan's National Research Grid Initiative (NAREGI). The TeraGrid will partner with these and other grid projects, NSF's core centers program, and software providers such as the NMI GRIDS Center to catalyze an integrated NSF cyberinfrastructure program with cross-agency and international impact.