Collaborative Research: Chameleon: A Large-Scale, Reconfigurable Experimental Environment for Cloud Research

协作研究：Chameleon：用于云研究的大规模、可重构实验环境

• 批准号: 1419141
• 负责人: Katarzyna Keahey
• 金额: $ 248.65万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Cooperative Agreement
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1419141&HistoricalAwards=false
• 关键词: Collaborative; Research; Chameleon; Large; Scale

A persistent problem facing academic cloud research is the lack of infrastructure and data to perform experimental research: large-scale hardware is needed to investigate the scalability of cloud infrastructure and applications, heterogeneous hardware is needed to investigate algorithmic and implementation tradeoffs, fully-configurable software environments are needed to investigate the performance of virtualization techniques and the differences between cloud software stacks, and data about how clouds are used is needed to evaluate virtual machine scheduling and data placement algorithms. The Chameleon project will addresses these needs by providing a large-scale, fully configurable experimental testbed driven by the needs of the cloud research and education communities. The testbed, and the ecosystem associated with it, will enable researchers to explore a range of cloud research challenges, from large scale to small scale, including exploring low-level problems in hardware architecture, systems research, network configuration, and software design, or at higher levels of abstraction looking at cloud scheduling, cloud platforms, and cloud applications. Chameleon will significantly enhance the ability of the computing research community to understand the behavior of Internet scale cloud systems, and to develop new software, ideas and algorithms for the cloud environment. As the tremendous shift to cloud as the primary means of providing computing infrastructure continues, a large-scale testbed tailored to researchers' needs is essential to the continued relevance of a large fraction of computing research. The project is led by the University of Chicago and includes partners from the Texas Advanced Computing Center (TACC), Northwestern University, the Ohio State University, and the University of Texas at San Antonio, comprising a highly qualified and experienced team, with research leaders from the cloud and networking world blended with providers of production quality cyberinfrastructure. The team includes members from the NSF-supported FutureGrid project and from the GENI community, both forerunners of the NSFCloud solicitation under which this project is funded. The Chameleon testbed, will be deployed at the University of Chicago (UC) and the Texas Advanced Computing Center (TACC) and will consist of 650 multi-core cloud nodes, 5PB of total disk space, and leverage 100 Gbps connection between the sites. While a large part of the testbed will consist of homogenous hardware to support large-scale experiments, a portion of it will support heterogeneous units allowing experimentation with high-memory, large-disk, low-power, GPU, and co-processor units. The project will also leverage existing FutureGrid hardware at UC and TACC in its first year to provide a transition period for the existing FutureGrid community of experimental users. To support a broad range of experiments emphasizing a range of requirements ranging from a high degree of control to ease of use the project will support a graduated configuration system allowing full user configurability of the stack, from provisioning of bare metal and network interconnects to delivery of fully functioning cloud environments. In addition, to facilitate experiments, Chameleon will support a set of services designed to meet researchers needs, including support for experimental management, reproducibility, and repositories of trace and workload data of production cloud workloads. To facilitate the latter, the project will form a set of partnerships with commercial as well as academic clouds, such as Rackspace and Open Science Data Cloud (OSDC). It will also partner with other testbeds, notably GENI and INRIA's Grid5000 testbed, and reach out to the user community to shape the policy an direction of the testbed. The Chameleon project will bring a new dimension and scale of resources to the CS community who wish to educate their students about design, implementation, operation and applications of cloud computing, a critical skillset for future computing professionals. It will enhance the understanding and application of experimental methodology in computer science and generate new educational materials and resources, with the participation of, and for, Minority Serving Institution (MSI) students.

学术云研究面临的一个长期问题是缺乏进行实验研究的基础设施和数据：需要大规模硬件来研究云基础设施和应用程序的可伸缩性，需要异构硬件来研究算法和实现的权衡，需要完全可配置的软件环境来研究虚拟化技术的性能和云软件堆栈之间的差异，需要关于如何使用云的数据来评估虚拟机调度和数据放置算法。变色龙项目将通过提供一个大规模的、完全可配置的实验测试平台来满足这些需求，以满足云研究和教育社区的需求。测试平台和与之相关的生态系统将使研究人员能够探索从大规模到小规模的一系列云研究挑战，包括探索硬件架构、系统研究、网络配置和软件设计中的低级问题，或者在更高的抽象层次上研究云调度、云平台和云应用。变色龙将显著提高计算研究界理解互联网规模云系统行为的能力，并为云环境开发新的软件、思想和算法。随着向云作为提供计算基础设施的主要手段的巨大转变继续进行，为研究人员的需求量身定制的大规模测试平台对于大部分计算研究的持续相关性至关重要。该项目由芝加哥大学领导，包括来自德克萨斯高级计算中心（TACC）、西北大学、俄亥俄州立大学和德克萨斯大学圣安东尼奥分校的合作伙伴，组成了一支高素质、经验丰富的团队，其中包括来自云和网络世界的研究领导者，以及生产质量网络基础设施的提供商。该团队包括来自nsf支持的FutureGrid项目和GENI社区的成员，这两个社区都是nsf云招标的前身，该项目是在nsf云招标的基础上资助的。变色龙测试平台将部署在芝加哥大学（UC）和德克萨斯高级计算中心（TACC），将由650个多核云节点组成，总磁盘空间为5PB，并在站点之间利用100 Gbps的连接。虽然测试平台的大部分将由支持大规模实验的同质硬件组成，但它的一部分将支持异构单元，允许使用高内存、大磁盘、低功耗、GPU和协处理器单元进行实验。该项目还将在第一年利用UC和TACC现有的FutureGrid硬件，为现有的FutureGrid实验用户社区提供一个过渡期。为了支持广泛的实验，强调从高度控制到易用性的一系列要求，该项目将支持一个分级配置系统，允许完全的用户可配置堆栈，从裸机和网络互连的供应到全功能云环境的交付。此外，为了方便实验，变色龙将支持一组旨在满足研究人员需求的服务，包括支持实验管理、可重复性以及生产云工作负载的跟踪和工作负载数据存储库。为了促进后者，该项目将与商业和学术云（如Rackspace和开放科学数据云（OSDC））建立一系列合作伙伴关系。它还将与其他测试平台合作，特别是GENI和INRIA的Grid5000测试平台，并与用户社区接触，以制定测试平台的政策和方向。变色龙项目将为CS社区带来一个新的维度和规模的资源，他们希望教育他们的学生关于云计算的设计、实现、操作和应用，这是未来计算专业人员的关键技能。它将加强对计算机科学实验方法的理解和应用，并在少数民族服务机构（MSI）学生的参与下产生新的教育材料和资源。