Performance optimization for multi-datacenter cloud platforms

多数据中心云平台性能优化

• 批准号: 499448-2016
• 负责人: Li, Baochun
• 金额: $ 7.03万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2016
• 资助国家: 加拿大
• 起止时间: 2016-01-01 至 2017-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=607448
• 关键词: Performance; optimization; multi; datacenter; cloud

In the era of cloud computing, large volumes of data are located in geographically distributed datacenters around the world, and need to be analyzed with machine learning algorithms in data analytic applications. Traditionally, jobs in these data analytic applications are supported by data parallel frameworks within a single cluster. Yet, since the data to be analyzed are located in multiple geographically distributed datacenters, it becomes necessary to deploy jobs in data analytic applications across these datacenters. Such a research trend is motivated by the fact that migrating large volumes of data to the same datacenter over a wide-area network is exceedingly slow, and will dramatically affect application performance. In this project, we seek to design and build a new system to optimize the performance of data analytic applications spanning multiple geographically distributed datacenters. We will design new algorithms for job scheduling, as well as new proactive push mechanisms for deploying data analytic jobs across datacenters. These new strategies will be implemented in a new real-world system framework based on the prevalent big data processing framework, Apache Spark, and experimentally evaluated across multiple geographically distributed datacenters.

在云计算时代，大量数据分布在全球各地的地理位置分散的数据中心，在数据分析应用中需要使用机器学习算法进行分析。传统上，这些数据分析应用程序中的作业由单个集群中的数据并行框架支持。然而，由于要分析的数据位于多个地理上分布的数据中心中，因此有必要在这些数据中心之间的数据分析应用程序中部署作业。这种研究趋势的动机是，通过广域网将大量数据迁移到同一个数据中心是非常缓慢的，并会显着影响应用程序的性能。在这个项目中，我们试图设计和构建一个新的系统，以优化跨多个地理分布的数据分析应用程序的性能。我们将设计新的作业调度算法，以及新的主动推送机制，用于跨数据中心部署数据分析作业。这些新策略将在基于流行的大数据处理框架Apache Spark的新的真实世界系统框架中实现，并在多个地理分布的数据中心进行实验评估。