Big Data Processing in Heterogeneous Cloud Environments

点击翻译按钮获取中文摘要

This proposal focuses on the intersection of Big Data and Cloud Computing, using advanced monitoring to allow the analysis and optimization of these complex interactions. New techniques can be developed, which combine concepts from both the Big Data and Cloud Computing fields and optimizes them for both performance and energy consumption. We will study how Big Data problems map to Cloud Computing systems with a focus on scaling and workload aware design. For example, we have an opportunity to improve Big Data workload by using virtual machine placement, in order to avoid congestion in the data centre. However, intelligent workload placements using large scale cyber and physical monitoring requires in depth exploration. Our research program will focus on how to best augment and improve these technologies by combining workload awareness with an awareness of the infrastructure and physical world.******Although the Cloud has been a powerful concept and tool, there are still vast improvements that can be made. One such advancement will come in the form of continued augmentation of the existing cloud with new processing capabilities. We will study and improve these complex heterogeneous environments with the goal of improving performance and lowering energy consumption. An avenue that needs further exploration is the deployment of GPGPU (general-purpose computing on graphics processing units) and FPGA (field programmable gate array) in these complex environments. The wide deployment of these specialized components will greatly expand the potential of today's cloud offerings, not only from a user's perspective but also from that of a cloud provider. Users will be able to improve their data processing applications, for example accelerating Spark by utilizing a GPU or FPGA. Cloud providers can reduce their service hosting overhead by using these specialized devices to perform important infrastructure critical tasks such as processing packets in the virtual network. However, the presence of multiple processing engines such as the CPU, GPU, and FPGA presents an interesting problem for both the cloud user and cloud provider. The mapping of workloads to these devices to maximize the performance of the cloud becomes paramount. Of equal importance, which mapping of tasks results in lower energy consumption and heat production of the physical cloud systems.******Research collaboration with industrial and government partners will play a pivotal role in my research program. For example, we are working with Compute Canada to help optimize and develop the next generation large scale computing platforms and applications by studying the Cedar super computer. Through monitoring, analysis, and modeling we will look at performance and issues in these large scale systems and devise optimizations. The development and improvement of these advanced platforms will continue to keep Canada at the forefront of research computing.