Collaborative Research: Chameleon: A Large-Scale, Reconfigurable Experimental Environment for Cloud Research

协作研究：Chameleon：用于云研究的大规模、可重构实验环境

• 批准号: 1419138
• 负责人: Joel Mambretti
• 金额: $ 60万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Cooperative Agreement
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2018-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1419138&HistoricalAwards=false
• 关键词: Collaborative; Research; Chameleon; Large; Scale

A persistent problem facing academic cloud research is the lack of infrastructure and data to perform experimental research: large-scale hardware is needed to investigate the scalability of cloud infrastructure and applications, heterogeneous hardware is needed to investigate algorithmic and implementation tradeoffs, fully-configurable software environments are needed to investigate the performance of virtualization techniques and the differences between cloud software stacks, and data about how clouds are used is needed to evaluate virtual machine scheduling and data placement algorithms. The Chameleon project will addresses these needs by providing a large-scale, fully configurable experimental testbed driven by the needs of the cloud research and education communities. The testbed, and the ecosystem associated with it, will enable researchers to explore a range of cloud research challenges, from large scale to small scale, including exploring low-level problems in hardware architecture, systems research, network configuration, and software design, or at higher levels of abstraction looking at cloud scheduling, cloud platforms, and cloud applications. Chameleon will significantly enhance the ability of the computing research community to understand the behavior of Internet scale cloud systems, and to develop new software, ideas and algorithms for the cloud environment. As the tremendous shift to cloud as the primary means of providing computing infrastructure continues, a large-scale testbed tailored to researchers' needs is essential to the continued relevance of a large fraction of computing research. The project is led by the University of Chicago and includes partners from the Texas Advanced Computing Center (TACC), Northwestern University, the Ohio State University, and the University of Texas at San Antonio, comprising a highly qualified and experienced team, with research leaders from the cloud and networking world blended with providers of production quality cyberinfrastructure. The team includes members from the NSF-supported FutureGrid project and from the GENI community, both forerunners of the NSFCloud solicitation under which this project is funded. The Chameleon testbed, will be deployed at the University of Chicago (UC) and the Texas Advanced Computing Center (TACC) and will consist of 650 multi-core cloud nodes, 5PB of total disk space, and leverage 100 Gbps connection between the sites. While a large part of the testbed will consist of homogenous hardware to support large-scale experiments, a portion of it will support heterogeneous units allowing experimentation with high-memory, large-disk, low-power, GPU, and co-processor units. The project will also leverage existing FutureGrid hardware at UC and TACC in its first year to provide a transition period for the existing FutureGrid community of experimental users. To support a broad range of experiments emphasizing a range of requirements ranging from a high degree of control to ease of use the project will support a graduated configuration system allowing full user configurability of the stack, from provisioning of bare metal and network interconnects to delivery of fully functioning cloud environments. In addition, to facilitate experiments, Chameleon will support a set of services designed to meet researchers needs, including support for experimental management, reproducibility, and repositories of trace and workload data of production cloud workloads. To facilitate the latter, the project will form a set of partnerships with commercial as well as academic clouds, such as Rackspace and Open Science Data Cloud (OSDC). It will also partner with other testbeds, notably GENI and INRIA's Grid5000 testbed, and reach out to the user community to shape the policy an direction of the testbed. The Chameleon project will bring a new dimension and scale of resources to the CS community who wish to educate their students about design, implementation, operation and applications of cloud computing, a critical skillset for future computing professionals. It will enhance the understanding and application of experimental methodology in computer science and generate new educational materials and resources, with the participation of, and for, Minority Serving Institution (MSI) students.

学术云研究面临的一个长期问题是缺乏基础设施和数据来进行实验研究：需要大规模硬件来研究云基础设施和应用的可扩展性，需要异构硬件来研究算法和实现权衡，需要完全可配置的软件环境来研究虚拟化技术的性能和云软件栈之间的差异，并且需要关于如何使用云的数据来评估虚拟机调度和数据放置算法。Chameleon项目将通过提供一个由云研究和教育社区需求驱动的大规模、完全可配置的实验测试平台来满足这些需求。该测试平台以及与之相关的生态系统将使研究人员能够探索从大规模到小规模的一系列云研究挑战，包括探索硬件架构、系统研究、网络配置和软件设计中的低级问题，或者在更高的抽象层次上研究云调度、云平台和云应用程序。Chameleon将显著增强计算研究社区理解互联网规模云系统行为的能力，并为云环境开发新的软件、想法和算法。随着云计算作为提供计算基础设施的主要手段的巨大转变，为研究人员的需求量身定制的大规模测试平台对于大部分计算研究的持续相关性至关重要。该项目由芝加哥大学牵头，合作伙伴包括来自德克萨斯州高级计算中心（TACC）、西北大学、俄亥俄州州立大学和德克萨斯大学圣安东尼奥分校的合作伙伴，他们组成了一支高素质、经验丰富的团队，其中包括来自云和网络世界的研究领导者以及生产质量网络基础设施的提供商。 该团队包括来自NSF支持的FutureGrid项目和GENI社区的成员，这两个项目都是NSFCloud征集的先驱。Chameleon测试平台将部署在芝加哥大学（UC）和德克萨斯州高级计算中心（TACC），由650个多核云节点组成，总磁盘空间为5 PB，并利用站点之间的100 Gbps连接。虽然测试平台的很大一部分将由同质硬件组成，以支持大规模实验，但其中一部分将支持异构单元，允许使用高内存，大磁盘，低功耗，GPU和协处理器单元进行实验。该项目还将在第一年利用UC和TACC现有的FutureGrid硬件，为现有的FutureGrid实验用户社区提供过渡期。为了支持广泛的实验，强调从高度控制到易用性的一系列要求，该项目将支持分级配置系统，允许堆栈的完全用户可配置性，从提供裸金属和网络互连到交付功能齐全的云环境。此外，为了促进实验，Chameleon将支持一系列旨在满足研究人员需求的服务，包括支持实验管理，再现性以及生产云工作负载的跟踪和工作负载数据存储库。为了促进后者，该项目将与商业和学术云建立一系列合作伙伴关系，如Rackspace和开放科学数据云（OSDC）。它还将与其他测试平台合作，特别是GENI和INRIA的Grid5000测试平台，并与用户社区接触，以制定测试平台的政策和方向。Chameleon项目将为CS社区带来新的资源维度和规模，CS社区希望教育学生有关云计算的设计，实施，操作和应用，这是未来计算专业人员的关键技能。它将在少数族裔服务机构（MSI）学生的参与下，增强对计算机科学实验方法论的理解和应用，并生成新的教育材料和资源。