Flight-Worthy Condor: Enabling Scientific Discovery

该奖项是根据2009年美国复苏与再投资法案(公法111-5)资助的。“可靠和有效地获得大量持续的计算能力对广泛学科的科学发现具有战略意义。全世界越来越多的科学界依靠秃鹰软件工具来利用共享计算资源的力量。在国内，已有100多所美国大学在校园内安装了这些工具，每月软件下载量超过2000次。从在布鲁克海文和费米国家实验室履行管理高能物理计算资源的战略角色，到在较小的代表性不足的机构实现研究计算，秃鹰技术代表着20年来分布式计算研究的顶峰。这项提案中概述的努力使该软件的研究贡献得以继续，并加强了使学术界和工业界能够利用有效的分布式计算的力量的势头。拟议工作的智力价值在于所实现的分布式机制的新颖性，以及团队在开发、维护和支持学术环境中的近100万行代码时面临的软件工程挑战。作为开源分布式计算能力的领先提供商，该软件作为主要的构建块发挥着重要作用。在许多校园网格以及社区、国家和国际网络基础设施倡议中。软件的规模、复杂性、多样性和质量保证要求也给软件工程和集成领域带来了挑战。多层软件堆栈的日常构建和测试、版本控制、软件跟踪和分布式故障排除都是持续挑战的例子。这一努力的更广泛影响是跨学科的，跨越学术界和工业界。使用秃鹰工具的研究人员能够极大地增加他们的计算吞吐量，从而增加他们研究的问题的规模和复杂性。该软件广泛应用于许多计算密集型学科，包括艺术、生物技术、经济学、化学、医学、高能和核物理以及计算机科学，并已被大学和政府实验室采用。此外，该软件通过管理美国大型强子对撞机实验的大部分计算资源，以及为250名研究引力波的LIGO科学家提供服务，发挥了关键作用。秃鹰技术为新的商业产品提供了动力，例如红帽的MRG和去年循环计算的CycleServer。这些技术还被用于工业中的任务关键型任务；例如，美光科技在全球15,000个CPU上使用秃鹰来验证内存芯片制造，摩根大通在5,000个CPU上使用秃鹰进行实时金融投资组合分析，以及迪士尼的全动画电影《荒野》使用秃鹰来渲染超过7000万帧。由于我们提供的高吞吐量计算能力甚至可以有效地利用运行Windows的台式计算机，它使代表性不足的机构的学生和科学家能够在无法访问传统高端计算设施的情况下进行需要大量计算的研究和教学。这项建议直接资助了五名学生，通过GLOW将为新一代跨学科专业人员提供良好的培训环境。

"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."Dependable and effective access to large amounts of sustained computing power is strategic to scientific discovery in a vast range of disciplines. An ever expanding number of science communities worldwide rely on Condor software tools to harness the power of shared computing resources. Closer to home, over 100 US universities have installed these tools on their campus, and downloads of the software surpass 2,000 per month. From fulfilling a strategic role of managing high-energy physics computing resources at Brookhaven and Fermi National Labs to enabling research computing at smaller under-represented institutions, Condor technology represents the culmination of 20 years of distributed computing research. The efforts outlined in this proposal allow the research contributions of this software to continue, and strengthen the momentum to enable academia and industry to leverage the power of effective distributed computing. The intellectual merit of the proposed effort lies within the novelty of the distributed mechanisms implemented and the software engineering challenges the team faces in developing, maintaining and supporting nearly one million lines of code in an academic setting. As a leading provider of open-source distributed computing capabilities, the software plays an important role as a primary ?building block? in many campus grids as well as community, national and international cyber-infrastructure initiatives. The size, complexity, diversity and quality assurance requirements of the software also lead to challenges in the area of software engineering and integration. Daily builds and testing, version control, software tracking and distributed troubleshooting of multi-layer software stacks are examples of the ongoing challenges. The broader impacts of this effort are interdisciplinary and span both academia and industry. Researchers who use Condor tools are able to greatly increase their computing throughput, and consequently increase the size and complexity of the problems they study. The software is widely used in many compute-intensive disciplines, including Art, Biotechnology, Economics, Chemistry, Medical, High Energy and Nuclear Physics, and Computer Science, and has been adopted by both universities and government labs. Furthermore, the software plays a critical role by managing the majority of the compute resources for the US Large Hadron Collider experiments, as well as serving 250 LIGO scientists studying gravitational waves. Condor technology fuels new commercial offerings, such as Red Hat's MRG and Cycle Computing's CycleServer in the past year. These technologies are also used for mission-critical tasks in industry; e.g., Micron Technology uses Condor on 15,000 CPUs worldwide to validate memory chip manufacturing, J.P. Morgan uses Condor on 5,000 CPUs for real-time financial portfolio analysis, and Disney's fully animated film "The Wild" used Condor to render more than 70 million frames. Because the high throughput computing capabilities we offer effectively harness even desktop machines running Windows, it enables students and scientists at underrepresented institutions to pursue research and instruction requiring considerable computing in the absence of access to traditional high-end computing facilities. This proposal directly funds five students, and through GLOW it will provide an excellent training environment to new generations of interdisciplinary professionals.