Big Data Processing in Heterogeneous Cloud Environments

异构云环境中的大数据处理

• 批准号: RGPIN-2018-06847
• 负责人: Shea, Ryan
• 金额: $ 1.68万
• 依托单位: Simon Fraser University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=664931
• 关键词: Big; Data; Processing; Heterogeneous; Cloud

This proposal focuses on the intersection of Big Data and Cloud Computing, using advanced monitoring to allow the analysis and optimization of these complex interactions. New techniques can be developed, which combine concepts from both the Big Data and Cloud Computing fields and optimizes them for both performance and energy consumption. We will study how Big Data problems map to Cloud Computing systems with a focus on scaling and workload aware design. For example, we have an opportunity to improve Big Data workload by using virtual machine placement, in order to avoid congestion in the data centre. However, intelligent workload placements using large scale cyber and physical monitoring requires in depth exploration. Our research program will focus on how to best augment and improve these technologies by combining workload awareness with an awareness of the infrastructure and physical world.******Although the Cloud has been a powerful concept and tool, there are still vast improvements that can be made. One such advancement will come in the form of continued augmentation of the existing cloud with new processing capabilities. We will study and improve these complex heterogeneous environments with the goal of improving performance and lowering energy consumption. An avenue that needs further exploration is the deployment of GPGPU (general-purpose computing on graphics processing units) and FPGA (field programmable gate array) in these complex environments. The wide deployment of these specialized components will greatly expand the potential of today's cloud offerings, not only from a user's perspective but also from that of a cloud provider. Users will be able to improve their data processing applications, for example accelerating Spark by utilizing a GPU or FPGA. Cloud providers can reduce their service hosting overhead by using these specialized devices to perform important infrastructure critical tasks such as processing packets in the virtual network. However, the presence of multiple processing engines such as the CPU, GPU, and FPGA presents an interesting problem for both the cloud user and cloud provider. The mapping of workloads to these devices to maximize the performance of the cloud becomes paramount. Of equal importance, which mapping of tasks results in lower energy consumption and heat production of the physical cloud systems.******Research collaboration with industrial and government partners will play a pivotal role in my research program. For example, we are working with Compute Canada to help optimize and develop the next generation large scale computing platforms and applications by studying the Cedar super computer. Through monitoring, analysis, and modeling we will look at performance and issues in these large scale systems and devise optimizations. The development and improvement of these advanced platforms will continue to keep Canada at the forefront of research computing.

该提案侧重于大数据和云计算的交叉点，使用高级监控来分析和优化这些复杂的交互。可以开发新的技术，这些技术结合了大数据和云计算领域的联合收割机概念，并针对性能和能耗进行了优化。我们将研究大数据问题如何映射到云计算系统，重点是扩展和工作负载感知设计。例如，我们有机会通过使用虚拟机布局来改善大数据工作负载，以避免数据中心的拥塞。然而，使用大规模网络和物理监控的智能工作负载放置需要深入探索。我们的研究计划将专注于如何通过将工作负载感知与基础设施和物理世界的感知相结合来最好地增强和改进这些技术。**虽然云已经是一个强大的概念和工具，但仍然可以进行巨大的改进。其中一个进步将是通过新的处理能力继续增强现有的云计算。我们将研究和改进这些复杂的异构环境，以提高性能和降低能耗为目标。在这些复杂的环境中，需要进一步探索的途径是GPGPU（通用计算图形处理单元）和FPGA（现场可编程门阵列）的部署。这些专用组件的广泛部署将极大地扩展当今云产品的潜力，不仅从用户的角度来看，而且从云提供商的角度来看。用户将能够改进其数据处理应用程序，例如通过使用GPU或FPGA加速Spark。云提供商可以通过使用这些专用设备来执行重要的基础设施关键任务（例如处理虚拟网络中的数据包）来减少其服务托管开销。然而，多个处理引擎（如CPU、GPU和FPGA）的存在给云用户和云提供商带来了一个有趣的问题。将工作负载映射到这些设备以最大限度地提高云的性能变得至关重要。同样重要的是，任务的映射导致物理云系统的能耗和热量产生更低。与工业和政府合作伙伴的研究合作将在我的研究计划中发挥关键作用。例如，我们正在与加拿大计算公司合作，通过研究雪松超级计算机，帮助优化和开发下一代大规模计算平台和应用程序。通过监控、分析和建模，我们将研究这些大规模系统的性能和问题，并设计优化方案。这些先进平台的开发和改进将继续使加拿大保持在研究计算的最前沿。