CC-NIE Integration: Bringing Distributed High Throughput Computing to the Network with Lark

Lark项目为分布式计算资源提供网络感知调度，允许工作负载管理调度器做出网络感知配置决策，并为单个批处理作业提供描述和请求所需网络拓扑的机制。具体地说，Lark将把网络性能监控工具perSONAR和带宽保证分配网络工具dynes的功能集成到分布式计算网格中间件如秃鹰和glideinWMS中。通过使用多站点IPv6试验台，并与内布拉斯加州的一个项目合作，为当地大学提供小型入门集群，该项目旨在生产可部署到开放科学网格(OSG)中的高质量生产技术。这项工作将在大型分布式计算环境中选择执行作业的站点、数据位置决策、性能和安全性之间进行更好的权衡，从而更有效地利用有限的资源。这个项目解决了大型强子对撞机(LHC)计算环境中一个明确而紧迫的需求，因为网络感知调度将增加LHC实验合作可以执行的分析量。通过将这项工作集成到广泛使用的Condor高通量计算软件和OSG堆栈中，不仅大型强子对撞机的研究人员可以获得好处，而且许多依赖分布式高通量计算的科学和工业项目也将受益。

The Lark project provides network-aware scheduling for distributed computing resources, allowing workload management schedulers to make network-aware provisioning decisions and providing mechanisms for individual batch jobs to describe and request the networking topology required. Specifically, Lark will integrate the capabilities of the perfSONAR, a network performance monitoring tool, and DYNES, a cyber-instrument for allocation of bandwidth guarantees, into distributed computing grid middleware such as Condor and glideinWMS. By using a multi-site IPv6 test-bed and collaborating with a Nebraska project to provide small starter clusters to local colleges, the project aims to produce production-quality technology that could be deployed into the Open Science Grid (OSG).This work will enable better trade-offs between selection of a site to execute jobs, data location decisions, performance, and security in large distributed computing environments, hence providing more effective use of limited resources. This project addresses a clear and immediate need in the Large Hadron Collider (LHC) computing environment, as network-aware scheduling will increase the amount of analysis the LHC experiment collaborations can perform. By integrating this work into the widely-used Condor high throughput computing software and into the OSG stack, the benefits will become available not only to LHC researchers, but to a number of science and industry projects that rely upon distributed high throughput computing.