SI2-SSE: BenchLab: Open Community Tools and Infrastructure for Performance Research in Cloud, Mobile and Green Computing

大规模互联网应用和服务的出现推动了对云平台、虚拟化、数据中心架构和绿色计算的研究激增。然而，新的研究原型的实际性能评估仍然是一个主要的挑战。研究人员经常使用的国产性能评估工具不再能够捕捉当今真实系统和应用程序的复杂性。为了解决这个缺点，该项目寻求开发BenchLab，这是一个开放、灵活的社区基础设施，包括应用程序、工作负载和工具，以便系统研究人员能够进行现实的性能评估和基准测试。BenchLab是一个开放的框架，其中源代码和工作负载数据集是免费的，供研究人员修改和使用，用于他们的特定实验。该框架由一套服务器端基准应用程序和工作负载组成，这些应用程序和工作负载代表云、移动web和绿色计算环境。BenchLab采用模块化、可扩展的架构，旨在支持一系列服务器应用程序和工作负载，能够添加对新应用程序和工作负载的支持，并淘汰过时的应用程序和工作负载。BenchLab被设计为易于在商业云（如Amazon EC2）或实验室环境中的虚拟化集群中进行“大规模”实验系统研究。BenchLab将为研究社区提供开放工具和工作负载，使研究人员能够运行更大、更现实的性能评估实验，更好地模拟当今的真实系统。BenchLab将纳入动手实验作业，以教学生实验绩效评估的科学和艺术。

The emergence of large-scale Internet applications and services has driven a surge in research on cloud platforms, virtualization, data center architectures, and green computing. However, realistic performance evaluation of new research prototypes continues to be a major challenge. Home-grown performance evaluation tools that are often used by researchers are no longer able to capture the complexity of today's real systems and applications. To address this drawback, the project seeks to develop BenchLab, an open, flexible community infrastructure comprising applications, workloads and tools to enable realistic performance evaluation and benchmarking by systems researchers. BenchLab is an open framework where source code and workload datasets are freely available for modification and use by researchers for their specific experiments. The framework consists of a suite of server-side benchmark applications and workloads that represent cloud, mobile web, and green computing environments. BenchLab employs a modular, extensible architecture that is designed to support a range of server applications and workloads, with the ability to add support for newer applications and workloads and retire outdated ones. BenchLab is designed to be easy to use for experimental systems research "at scale" in commercial clouds, such as Amazon EC2, or virtualized clusters in laboratory settings. BenchLab will provide open tools and workloads for the research community to enable researchers to run larger and more realistic performance evaluation experiments that better emulate today's real-word systems. BenchLab will be incorporated into hands-on lab assignments to teach students the science and art of experimental performance evaluation.