Intelligent Internet-Scale Multimedia Storage and Delivery

智能互联网规模多媒体存储和传输

• 批准号: 436170-2013
• 负责人: Niu, Di
• 金额: $ 1.82万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=660993
• 关键词: Intelligent; Internet; Scale; Multimedia; Storage

Video services, such as Youtube and Netflix, are enormously popular on the Internet nowadays. With the advent of datacenters, cloud computing and increased Internet bandwidth, video delivery is experiencing a major architectural change. As a typical Internet-scale video delivery system, Netflix hosts its content in Amazon Simple Storage Service (S3), which organizes a storage system spanning geographically distributed datacenters. The videos are then delivered to end users through multiple Content Delivery Networks (CDNs) as edge servers. Our grand goal in this proposal is to build intelligence into future video storage and delivery systems at the Internet scale by incorporating data-intensive decision making in computer networking. The proposed system will utilize various types of data collected from the network, including system traces, logs, user reports and even social information, for workload prediction and performance inference. The processed data will then be used to assist content placement, resource management, data transfers and load routing, which can be modelled as data-intensive optimization problems, at multiple levels in the video distribution hierarchy. The key challenges to be addressed include how to learn useful information from data, how to approach various combinatorial optimization problems required by the application, and especially, how to efficiently solve such data-intensive optimizations on-the-fly to satisfy application requirements. In order to scale to very large systems and a large amount of input data, we will particularly investigate novel distributed and parallel algorithms and computation paradigms, which either lead to decentralized decisions in the network or efficient parallel computation through Hadoop and cloud computing services. The proposed system will eventually integrate large-scale computer networking and traffic engineering with data-intensive computing and decision making at the application level. Our study will be based on a large amount of data collected from various video delivery and CDN companies. We aim to test our approaches in real network experiments involving PlanetLab nodes, Amazon EC2 datacenters and server clusters, depending on resource availability.**

视频服务，如Youtube和Netflix，现在在互联网上非常受欢迎。随着视频中心、云计算和互联网带宽的增加，视频交付正在经历一场重大的架构变革。作为一个典型的互联网规模的视频交付系统，Netflix将其内容托管在Amazon Simple Storage Service（S3）中，该服务组织了一个跨越地理分布的存储中心的存储系统。然后，视频通过多个内容交付网络（CDN）作为边缘服务器交付给最终用户。我们在这个提案中的宏伟目标是通过将计算机网络中的数据密集型决策纳入互联网规模的未来视频存储和传输系统中。拟议的系统将利用从网络收集的各种类型的数据，包括系统跟踪、日志、用户报告甚至社交信息，用于工作量预测和性能推断。然后，处理后的数据将用于辅助内容放置、资源管理、数据传输和负载路由，这些可以建模为视频分发层次结构中多个级别的数据密集型优化问题。需要解决的关键挑战包括如何从数据中学习有用的信息，如何处理应用程序所需的各种组合优化问题，特别是如何有效地实时解决此类数据密集型优化以满足应用程序需求。为了扩展到非常大的系统和大量的输入数据，我们将特别研究新的分布式和并行算法和计算范例，这些算法和范例可以在网络中实现分散决策，也可以通过Hadoop和云计算服务实现高效的并行计算。拟议的系统最终将在应用层面上将大规模计算机网络和流量工程与数据密集型计算和决策相结合。我们的研究将基于从各种视频交付和CDN公司收集的大量数据。我们的目标是在真实的网络实验中测试我们的方法，这些实验涉及PlanetLab节点、Amazon EC2数据中心和服务器集群，具体取决于资源的可用性。**