CC-NIE Integration: Bringing Distributed High Throughput Computing to the Network with Lark

CC-NIE集成：通过Lark为网络带来分布式高吞吐量计算

• 批准号: 1245864
• 负责人: Brian Bockelman
• 金额: $ 57.33万
• 依托单位: University of Nebraska-Lincoln
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-10-01 至 2015-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1245864&HistoricalAwards=false
• 关键词: CC; NIE; Integration; Bringing; Distributed

The Lark project provides network-aware scheduling for distributed computing resources, allowing workload management schedulers to make network-aware provisioning decisions and providing mechanisms for individual batch jobs to describe and request the networking topology required. Specifically, Lark will integrate the capabilities of the perfSONAR, a network performance monitoring tool, and DYNES, a cyber-instrument for allocation of bandwidth guarantees, into distributed computing grid middleware such as Condor and glideinWMS. By using a multi-site IPv6 test-bed and collaborating with a Nebraska project to provide small starter clusters to local colleges, the project aims to produce production-quality technology that could be deployed into the Open Science Grid (OSG).This work will enable better trade-offs between selection of a site to execute jobs, data location decisions, performance, and security in large distributed computing environments, hence providing more effective use of limited resources. This project addresses a clear and immediate need in the Large Hadron Collider (LHC) computing environment, as network-aware scheduling will increase the amount of analysis the LHC experiment collaborations can perform. By integrating this work into the widely-used Condor high throughput computing software and into the OSG stack, the benefits will become available not only to LHC researchers, but to a number of science and industry projects that rely upon distributed high throughput computing.

Lark项目为分布式计算资源提供了网络感知调度，允许工作负载管理人员做出网络感知的供应决策，并为单个批处理作业提供描述和请求所需网络拓扑的机制。具体来说，Lark将把网络性能监控工具perfSONAR和用于分配带宽保证的网络工具DYNES的功能集成到分布式计算网格中间件（如Condor和glideinWMS）中。通过使用多站点IPv6测试床并与内布拉斯加州的一个项目合作，为当地大学提供小型启动集群，该项目旨在生产可部署到开放科学网格（OSG）中的生产质量技术。这项工作将在大型分布式计算环境中选择执行作业的站点、数据位置决策、性能和安全性之间实现更好的权衡，从而更有效地利用有限的资源。该项目解决了大型强子对撞机（LHC）计算环境中明确而迫切的需求，因为网络感知调度将增加LHC实验协作可以执行的分析量。通过将这项工作集成到广泛使用的Condor高吞吐量计算软件和OSG堆栈中，不仅可以为LHC研究人员带来好处，而且可以为许多依赖分布式高吞吐量计算的科学和工业项目带来好处。