PPoSS: Planning: CP2: Towards Systems Correctness Checkability and Performance Predictability at Scale

PPoSS：规划：CP2：实现大规模系统正确性可检查性和性能可预测性

• 批准号: 2028427
• 负责人: Haryadi Gunawi
• 金额: $ 24.8万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2028427&HistoricalAwards=false
• 关键词: PPoSS; Planning; CP2; Towards; Systems

As a critical backend for many of today's applications and services, large-scale distributed systems must be highly reliable. In the last couple of years the field witnessed a phenomenal scale of deployment; Google is known to run clusters with thousands of machines each, Apple deploys over 100,000 database machines, and Netflix runs tens of database clusters with 500 nodes each. This new era of cloud-scale distributed systems has given birth to a new class of faults, scalability faults---faults whose symptoms surface in large-scale deployments but not necessarily in small/medium-scale deployments. The CP2 project is proposed to solve the problem of correctness checkability and performance predictability of systems at extreme scale. Specifically the project will analyze over 500 real-world scalability faults in over a dozen large-scale systems, develop a single-machine scale-checking framework that allows developers to test large distributed code on one or a few machines, and provide groundwork for compute- and I/O-performance predictability of large-scale jobs on both existing and future architectures. These tasks will advance debugging, testing, learning, and prediction methods both on traditional hardware platforms and emerging ones and ultimately lead to correct-by-construction development methods. The CP2 project will have impact in multiple disciplines including systems (cloud/datacenter systems reliability), programming languages/compilers (new static/dynamic analysis techniques), architecture (compute/storage prediction for heterogeneous hardware), algorithms (the use of learning methods), and high-performance computing (benchmarking of HPC systems/applications).In terms of societal benefits, the CP2 project addresses paramount issues mentioned in the NSF Strategic Plan for 2018-2022. More specifically, society increasingly depends on complicated systems that are products of human ingenuity, including ecosystems of large and complex software with millions of lines of code running on thousands of machines. CP2 will address the challenges of understanding and predicting the behavior of such systems. Furthermore, as society’s reliance on complex systems grows, learning about their robustness and understanding how to strengthen them are of increasing importance. In terms of education, the CP2 project gives unique hands-on research and education with cutting-edge systems technology in which students will be trained to operate software on a large number of machines and analyze their performance and correctness. The results of the CP2 project will be released through the classic medium of publication, through the development of numerous software artifacts which will be open-sourced, and finally through collaboration with various industry partners to help shape the next generation of large-scale systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

作为当今许多应用程序和服务的关键后端，大规模的分布式系统必须非常可靠。在过去的几年中，该领域见证了一个惊人的部署规模。众所周知，Google可以运行每台数千台机器，Apple部署超过100,000个数据库机器，Netflix运行数十个数据库群集，每台有500个节点。这个云规模分布式系统的新时代已经诞生了一类新的故障，可伸缩性故障 - - 症状在大规模部署中表现出来，但不一定是在中小型/中等规模的部署中。提出了CP2项目来解决系统的正确性可检查性和性能可预测性的问题。具体而言，该项目将在十几个大型系统中分析500多个现实世界可伸缩性故障，开发一个单次尺度检查框架，使开发人员能够在一台或几个机器上测试大型分布式代码，并为对现有和未来架构的大规模可预测工作提供基础，并为计算和I/O/O-O-O-O-O-O-O-O-Scormentance可预测性。这些任务将提高传统硬件平台和新兴平台上的调试，测试，学习和预测方法，并最终导致纠正的开发方法。 The CP2 project will have impact in multiple disciplines including systems (cloud/datacenter systems reliability), programming languages/compilers (new static/dynamic analysis techniques), architecture (compute/storage prediction for heterogeneous hardware), algorithms (the use of learning methods), and high-performance computing (benchmarking of HPC systems/applications).In terms of society benefits, the CP2 project addresses 2018 - 2022年NSF战略计划中提到的派拉蒙问题。更具体地说，社会越来越多地取决于复杂的系统，这些系统是人类邪恶的产物，包括大而复杂的软件的生态系统，其中数百万条代码在数千台机器上运行。 CP2将解决理解和预测此类系统行为的挑战。此外，随着社会对复杂系统的缓解，了解了它们的稳健性并了解如何加强它们的重要性。在教育方面，CP2项目通过尖端的系统技术提供了独特的动手研究和教育，其中将培训学生在大量机器上操作软件并分析其性能和正确性。 The results of the CP2 project will be released through the classic medium of publication, through the development of numerous software artifacts which will be open-sourced, and finally through collaboration with various industry partners to help shape the next generation of large-scale systems.This award reflects NSF's statutory mission and has We were deemed honestly of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.