Exploring a New Design Paradigm for Distributed Systems in Modern Data Centers

探索现代数据中心分布式系统的新设计范式

• 批准号: RGPIN-2017-04180
• 负责人: AlKiswany, Samer
• 金额: $ 2.26万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=710911
• 关键词: Exploring; New; Design; Paradigm; Distributed

The end-to-end design “principle” pervades the design of virtually every modern distributed system. In its extreme form, critical functionality is implemented solely in end hosts, with a dumb but fast network to connect them. Unfortunately, such a network-oblivious approach is fundamentally inefficient, as it provides one-size-fits-all functionality to a multifarious set of applications and platforms. While this guideline has served us well in the past 30 years, technology evolution and tensions at the network interface suggest that it may be productive to revisit it. The goal of the proposed research program is to explore an alternative paradigm for designing distributed systems to leverage the capabilities of the emerging technology of software-defined networks (SDNs). The new paradigm will co-design distributed systems' logic and networking support to realize more efficient, scalable, and reliable systems. Achieving this vision requires a drastic shift in our approach to building, engineering, testing, securing, and deploying systems. I plan to establish a leading research group that will explore this new paradigm for designing a complete modern data center software/hardware stack, including building, testing, and deploying systems; exploring new security, management, accounting, and QoS techniques; and investigating new programming frameworks, domain-specific languages, and testing and debugging tools. This Discovery Grant funding is instrumental in bootstrapping this research group. The immediate research context is a network-integrated design for distributed storage systems, focusing on KV stores. To this end, the research program aims to (1) characterize the current KV systems to understand their network load and consistency properties; (2) design and prototype a network-integrated KV store; (3) explore designs for end-to-end QoS support; and (4) study the impact of departing from the end-to-end principle on security properties. The innovative paradigm we will explore has the potential to have a wide influence on technology, including many Canadian companies that build KV stores (e.g., NetApp, Ericsson, and IBM) and more than 150 Canadian data centers that use these systems. In particular, the program will result in four main contributions. First, the resulting designs will provide a blueprint for future systems and present novel solutions for stagnant QoS and scalability problems. Second, this program will build network-integrated designs for core distributed system mechanisms (e.g., QoS, security, consistency, and load balancing) that are applicable beyond KV stores. Third, this program will provide timely feedback on the suitability of the SDN API for enabling more efficient designs. Fourth, this program will provide an invaluable opportunity for training highly qualified personnel on cutting-edge networking and storage technology.

端到端设计“原则”几乎渗透到每个现代分布式系统的设计中。在其极端形式中，关键功能仅在终端主机中实现，并使用哑但快速的网络连接它们。不幸的是，这种与网络无关的方法从根本上是低效的，因为它为各种各样的应用程序和平台提供了一刀切的功能。