SI2-SSE: BenchLab: Open Community Tools and Infrastructure for Performance Research in Cloud, Mobile and Green Computing

大规模互联网应用和服务的出现推动了对云平台、虚拟化、数据中心架构和绿色计算的研究激增。 然而，新的研究原型的现实性能评估仍然是一个重大挑战。 研究人员经常使用的自行开发的性能评估工具不再能够捕捉当今真实的系统和应用程序的复杂性。 为了解决这个问题，该项目寻求开发BenchLab，这是一个开放，灵活的社区基础设施，包括应用程序，工作负载和工具，以实现系统研究人员的现实性能评估和基准测试。 BenchLab是一个开放的框架，源代码和工作负载数据集可供研究人员免费修改和使用，用于他们的特定实验。该框架由一套服务器端基准应用程序和工作负载组成，代表云、移动的Web和绿色计算环境。 BenchLab采用模块化、可扩展的架构，旨在支持一系列服务器应用程序和工作负载，并能够为新的应用程序和工作负载添加支持，并淘汰过时的应用程序和工作负载。 BenchLab旨在易于在商业云（如Amazon EC2）或实验室环境中的虚拟化集群中进行“大规模”实验系统研究。 BenchLab将为研究社区提供开放的工具和工作负载，使研究人员能够运行更大、更真实的性能评估实验，更好地模拟当今的真实系统。BenchLab将被纳入动手实验作业，教学生实验性能评估的科学和艺术。

The emergence of large-scale Internet applications and services has driven a surge in research on cloud platforms, virtualization, data center architectures, and green computing. However, realistic performance evaluation of new research prototypes continues to be a major challenge. Home-grown performance evaluation tools that are often used by researchers are no longer able to capture the complexity of today's real systems and applications. To address this drawback, the project seeks to develop BenchLab, an open, flexible community infrastructure comprising applications, workloads and tools to enable realistic performance evaluation and benchmarking by systems researchers. BenchLab is an open framework where source code and workload datasets are freely available for modification and use by researchers for their specific experiments. The framework consists of a suite of server-side benchmark applications and workloads that represent cloud, mobile web, and green computing environments. BenchLab employs a modular, extensible architecture that is designed to support a range of server applications and workloads, with the ability to add support for newer applications and workloads and retire outdated ones. BenchLab is designed to be easy to use for experimental systems research "at scale" in commercial clouds, such as Amazon EC2, or virtualized clusters in laboratory settings. BenchLab will provide open tools and workloads for the research community to enable researchers to run larger and more realistic performance evaluation experiments that better emulate today's real-word systems. BenchLab will be incorporated into hands-on lab assignments to teach students the science and art of experimental performance evaluation.