Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications

跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序

基本信息

  • 批准号:
    RGPIN-2016-06281
  • 负责人:
  • 金额:
    $ 4.74万
  • 依托单位:
  • 依托单位国家:
    加拿大
  • 项目类别:
    Discovery Grants Program - Individual
  • 财政年份:
    2016
  • 资助国家:
    加拿大
  • 起止时间:
    2016-01-01 至 2017-12-31
  • 项目状态:
    已结题

项目摘要

Both big data analytics and mobile cloud applications are increasingly geographically distributed, relying upon services from multiple datacenters around the world. Yet, wide-area inter-datacenter links are costly to build and to upgrade, and inter-datacenter bandwidth is bound to become a scarce resource in the near future. Traditional data analytics need to first move all raw data to one datacenter before they can be processed, which adversely affect their performance and are bandwidth intensive. Real-time streaming analytics and interactive mobile cloud applications such as video conferencing and messaging, on the other hand, require low latencies. As such, there is a pressing need to manage and allocate inter-datacenter resources at a fine granularity to meet the diverse performance targets from both data analytics and mobile cloud applications. In this program, we propose to design and implement a new software-defined inter-datacenter infrastructure, in order to optimize the utilization of inter-datacenter bandwidth. Operating in the application layer, our infrastructure provides a substrate and a public application programming interface (API) to support existing data analytic frameworks and interactive mobile cloud applications. It schedules data analytic tasks, routes data streams, and manages bandwidth allocations to application flows at a fine granularity, with the objective of utilizing scarce inter-datacenter bandwidth with optimal efficiency. Following the principles of software-defined networking, our infrastructure fully decouples the control and data planes, and makes task scheduling, flow routing, and traffic engineering decisions based on live analytics of network conditions. We will propose new algorithms to make these decisions in the controller, catering for the specific needs of each category of applications. For batch and streaming analytics applications, our new algorithms are designed to optimize task scheduling and assignment in multiple datacenters, so that both the volume and latency of inter-datacenter traffic can be optimized without prior knowledge of the communication patterns of data analytics jobs. For interactive mobile cloud applications that are bandwidth-intensive and latency-sensitive, our algorithms are designed to optimize resource allocations according to the delay sensitivity of competing application flows. To be more expressive, our infrastructure uses executable scripts to implement control plane decisions in the data plane. We plan to implement our new infrastructure in a real-world inter-datacenter testbed involving multiple datacenters worldwide. This program will lead to fundamentally new ways of using a large number of datacenters around the world to support big data analytics and mobile applications, both of which will soon to be critical and indispensable to businesses and consumers alike in Canada.
大数据分析和移动的云应用程序在地理上的分布越来越多,依赖于来自世界各地多个服务中心的服务。然而,广域数据中心间链路的构建和升级成本很高,并且数据中心间带宽在不久的将来必然会成为稀缺资源。传统的数据分析需要首先将所有原始数据移动到一个数据中心,然后才能进行处理,这会对其性能产生不利影响,并且占用大量带宽。另一方面,实时流媒体分析和交互式移动的云应用程序(如视频会议和消息传递)需要较低的延迟。因此,迫切需要以细粒度管理和分配数据中心间资源,以满足数据分析和移动的云应用的不同性能目标。 在这个项目中,我们提出设计和实现一个新的软件定义的数据中心间的基础设施,以优化数据中心间的带宽利用率。我们的基础设施在应用层运行,提供底层和公共应用程序编程接口(API),以支持现有的数据分析框架和交互式移动的云应用程序。它以精细的粒度调度数据分析任务、路由数据流并管理应用程序流的带宽分配,目的是以最佳效率利用稀缺的数据中心间带宽。遵循软件定义网络的原则,我们的基础设施完全融合了控制和数据平面,并根据对网络状况的实时分析做出任务调度、流量路由和流量工程决策。 我们将提出新的算法来在控制器中做出这些决定,以满足每类应用程序的特定需求。对于批处理和流分析应用程序,我们的新算法旨在优化多个数据中心中的任务调度和分配,以便在不事先了解数据分析作业的通信模式的情况下优化数据中心间流量的容量和延迟。对于带宽密集型和延迟敏感的交互式移动的云应用程序,我们的算法被设计为根据竞争应用程序流的延迟敏感性来优化资源分配。为了更具表现力,我们的基础设施使用可执行脚本来实现数据平面中的控制平面决策。我们计划在涉及全球多个数据中心的真实数据中心间测试平台中实施我们的新基础设施。该计划将带来全新的方式,利用世界各地的大量数据中心来支持大数据分析和移动的应用程序,这两者很快将成为加拿大企业和消费者不可或缺的关键。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Li, Baochun其他文献

On achieving maximum multicast-throughput in undirected networks
Integrated Analysis of Metabolome and Transcriptome Reveals Insights for Low Phosphorus Tolerance in Wheat Seedling.
  • DOI:
    10.3390/ijms241914840
  • 发表时间:
    2023-10-02
  • 期刊:
  • 影响因子:
    5.6
  • 作者:
    Li, Pengcheng;Ma, Xiaole;Wang, Juncheng;Yao, Lirong;Li, Baochun;Meng, Yaxiong;Si, Erjing;Yang, Ke;Shang, Xunwu;Zhang, Xueyong;Wang, Huajun
  • 通讯作者:
    Wang, Huajun
Wheat centromeric retrotransposons: the new ones take a major role in centromeric structure
  • DOI:
    10.1111/tpj.12086
  • 发表时间:
    2013-03-01
  • 期刊:
  • 影响因子:
    7.2
  • 作者:
    Li, Baochun;Choulet, Frederic;Zhang, Xueyong
  • 通讯作者:
    Zhang, Xueyong
Production of selectable marker-free transgenic tobacco plants using a non-selection approach: chimerism or escape, transgene inheritance, and efficiency
  • DOI:
    10.1007/s00299-008-0640-8
  • 发表时间:
    2009-03-01
  • 期刊:
  • 影响因子:
    6.2
  • 作者:
    Li, Baochun;Xie, Claire;Qiu, Hui
  • 通讯作者:
    Qiu, Hui
Collaborative Caching in Wireless Video Streaming Through Resource Auctions
通过资源拍卖在无线视频流中进行协作缓存

Li, Baochun的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Li, Baochun', 18)}}的其他基金

Scalable Federated Learning and Analytics with Communication Efficiency in Mobile Cloud Computing
移动云计算中具有通信效率的可扩展联合学习和分析
  • 批准号:
    RGPIN-2022-04782
  • 财政年份:
    2022
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2021
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2020
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2019
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2018
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2017
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Performance optimization for multi-datacenter cloud platforms
多数据中心云平台性能优化
  • 批准号:
    499448-2016
  • 财政年份:
    2016
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Collaborative Research and Development Grants
Thrift: optimizing performance in complex mobile internet applications with network coding
Thrift:通过网络编码优化复杂移动互联网应用程序的性能
  • 批准号:
    238994-2011
  • 财政年份:
    2015
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
A cloud-assisted crowdsourcing machine-to-machine networking platform for vehicular applications
用于车辆应用的云辅助众包机器对机器网络平台
  • 批准号:
    447493-2013
  • 财政年份:
    2015
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Strategic Projects - Group
Thrift: optimizing performance in complex mobile internet applications with network coding
Thrift:通过网络编码优化复杂移动互联网应用程序的性能
  • 批准号:
    238994-2011
  • 财政年份:
    2014
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual

相似海外基金

Developing Educational Resources to Roll Out the Change-Makers Youth Leadership Programme Across Africa
开发教育资源,在非洲推出变革者青年领导力计划
  • 批准号:
    AH/W001543/1
  • 财政年份:
    2022
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Research Grant
Creation of a platform for across-subjects learning "regional resources" including subject content
打造学科内容跨学科学习“区域资源”平台
  • 批准号:
    22K02627
  • 财政年份:
    2022
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Collaborative research: From Community to Practice: Evaluating How Open Educational Resources Facilitate Implementation of Vision and Change Principles Across Diverse Institutions
合作研究:从社区到实践:评估开放教育资源如何促进不同机构愿景和变革原则的实施
  • 批准号:
    2125998
  • 财政年份:
    2021
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Standard Grant
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2021
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Collaborative research: From Community to Practice: Evaluating How Open Educational Resources Facilitate Implementation of Vision and Change Principles Across Diverse Institutions
合作研究:从社区到实践:评估开放教育资源如何促进不同机构愿景和变革原则的实施
  • 批准号:
    2125990
  • 财政年份:
    2021
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Standard Grant
Collaborative research: From Community to Practice: Evaluating How Open Educational Resources Facilitate Implementation of Vision and Change Principles Across Diverse Institutions
合作研究:从社区到实践:评估开放教育资源如何促进不同机构愿景和变革原则的实施
  • 批准号:
    2126110
  • 财政年份:
    2021
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Standard Grant
A study on the dynamics of human resources of the ethnic minority in adjacent areas across national borders between China & Mainland Southeast Asia
我国跨国界毗邻地区少数民族人力资源动态研究
  • 批准号:
    21K01069
  • 财政年份:
    2021
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
跨地理分布式数据中心管理资源以进行大数据分析和移动云应用程序
  • 批准号:
    RGPIN-2016-06281
  • 财政年份:
    2020
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Discovery Grants Program - Individual
Mental health planning in the era of a pandemic: appropriately matching mental health needs with resources across Canada through a rapid assessment of COVID-19's impact on adults and older adults.
大流行时代的心理健康规划:通过快速评估 COVID-19 对成年人和老年人的影响,将心理健康需求与加拿大各地的资源适当匹配。
  • 批准号:
    432494
  • 财政年份:
    2020
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Operating Grants
Mobility of Human Resources across Borders and the Future of Foreign Credential Recognition
人力资源的跨境流动和外国学历认证的未来
  • 批准号:
    19KK0061
  • 财政年份:
    2019
  • 资助金额:
    $ 4.74万
  • 项目类别:
    Fund for the Promotion of Joint International Research (Fostering Joint International Research (B))
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了