Performance optimization for multi-datacenter cloud platforms

点击翻译按钮获取中文摘要

In the era of cloud computing, large volumes of data are located in geographically distributed datacenters around the world, and need to be analyzed with machine learning algorithms in data analytic applications. Traditionally, jobs in these data analytic applications are supported by data parallel frameworks within a single cluster. Yet, since the data to be analyzed are located in multiple geographically distributed datacenters, it becomes necessary to deploy jobs in data analytic applications across these datacenters. Such a research trend is motivated by the fact that migrating large volumes of data to the same datacenter over a wide-area network is exceedingly slow, and will dramatically affect application performance. In this project, we seek to design and build a new system to optimize the performance of data analytic applications spanning multiple geographically distributed datacenters. We will design new algorithms for job scheduling, as well as new proactive push mechanisms for deploying data analytic jobs across datacenters. These new strategies will be implemented in a new real-world system framework based on the prevalent big data processing framework, Apache Spark, and experimentally evaluated across multiple geographically distributed datacenters.