Performance optimization for multi-datacenter cloud platforms

在云计算时代，大量数据分布在全球各地的地理分布数据中心，在数据分析应用中需要使用机器学习算法进行分析。传统上，这些数据分析应用程序中的作业由单个集群中的数据并行框架支持。然而，由于要分析的数据位于多个地理分布的数据中心，因此有必要跨这些数据中心在数据分析应用程序中部署作业。这种研究趋势的动机是，通过广域网将大量数据迁移到同一个数据中心的速度非常慢，并且会极大地影响应用程序的性能。在这个项目中，我们寻求设计和构建一个新的系统来优化跨多个地理分布数据中心的数据分析应用程序的性能。我们将为作业调度设计新的算法，以及为跨数据中心部署数据分析作业设计新的主动推送机制。这些新策略将在基于流行的大数据处理框架Apache Spark的新的现实世界系统框架中实现，并在多个地理分布数据中心进行实验评估。

In the era of cloud computing, large volumes of data are located in geographically distributed datacenters around the world, and need to be analyzed with machine learning algorithms in data analytic applications. Traditionally, jobs in these data analytic applications are supported by data parallel frameworks within a single cluster. Yet, since the data to be analyzed are located in multiple geographically distributed datacenters, it becomes necessary to deploy jobs in data analytic applications across these datacenters. Such a research trend is motivated by the fact that migrating large volumes of data to the same datacenter over a wide-area network is exceedingly slow, and will dramatically affect application performance. In this project, we seek to design and build a new system to optimize the performance of data analytic applications spanning multiple geographically distributed datacenters. We will design new algorithms for job scheduling, as well as new proactive push mechanisms for deploying data analytic jobs across datacenters. These new strategies will be implemented in a new real-world system framework based on the prevalent big data processing framework, Apache Spark, and experimentally evaluated across multiple geographically distributed datacenters.