CIF:Small:Collaborative Research:Codes for Storage with Queues for Access

CIF：小：协作研究：带有访问队列的存储代码

• 批准号: 1717314
• 负责人: Emina Soljanin
• 金额: $ 30.92万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717314&HistoricalAwards=false
• 关键词: CIF; Small; Collaborative; Research; Codes

Large volumes of data, which are being collected for the purpose of knowledge extraction, have to be reliably, efficiently, and securely stored. Retrieval of large data files from storage has to be fast. Large-scale cloud data storage and distributed file systems have become the backbone of many applications such as web searching, e-commerce, and cluster computing. Cloud services are implemented on top of a distributed storage layer that acts as a middleware to the applications, and also provides the desired content to the users, whose interests range from performing data analytics to watching movies. Users of cloud systems demand that their content and services be readily available and their data be reliably stored. Although there are apparent connections and trade-offs between these two objectives, so far they have been addressed mostly separately. The proposed research will characterize the interplay between reliable data storage and fast data access, and develop methods to jointly optimize these two main objectives of cloud storage. The project team will develop the methodology for design, analysis, and performance evaluation of a broad range of techniques for distributed storage that enable high reliability, robustness, and fast data retrieval. The research team will also develop schemes for storage and distributed computing which will optimize both reliability and the latency of data access. The methodology developed in the course of this project will allow the system operators to make their content and services readily available to users and support delay-sensitive applications ranging from individual video streaming to using online collaborative tools. This research will minimize the energy requirements of data centers, which have increased massively in recent years. The project will contribute to the broader areas of coding theory, information theory, and queueing theory, and open new ways of cross-pollination.This project focuses on efficient data access in distributed file systems that employ codes for reliable and efficient storage. Users of cloud systems demand that their content and services be readily available and their data be reliably stored. Although there are apparent connections and trade-offs between these two objectives, thus far they have been addressed mostly separately. This proposal follows findings that analyze how some of today?s solutions for reliable data storage affect the speed of data download under certain access models. This preliminary research has shown that, in some scenarios, the coding schemes used for increasing storage reliability can be further exploited for fast data access, while in others, the coding schemes that seemingly increase data availability actually fail to provide efficient access to popular content (so-called ?hot data?). The proposed research aims to characterize the interplay between reliable data storage and fast data access, and develop methods to jointly optimize these two main objectives of cloud storage. The proposed research will first identify and design schemes for coded-data access and derive (bounds on and estimates of) the expected download time for these schemes. Regardless of which data access scheme is used, the expected download time will depend on realistic service models as well as the distributed system service capacity provisioning and allocation schemes, which are then addressed. The work will also focus on the connections between the proposed research and the areas of efficient distributed computing and reduction in data center energy consumption. Preliminary results indicate that these areas are closely connected and the techniques developed for one area can benefit other areas.

为了提取知识而收集的大量数据必须可靠、高效和安全地存储。从存储器中检索大数据文件的速度必须很快。大规模云数据存储和分布式文件系统已经成为web搜索、电子商务和集群计算等许多应用程序的支柱。云服务是在分布式存储层之上实现的，分布式存储层充当应用程序的中间件，还向用户提供所需的内容，用户的兴趣范围从执行数据分析到观看电影。云系统的用户要求他们的内容和服务随时可用，他们的数据可靠地存储。虽然这两个目标之间有明显的联系和权衡，但到目前为止，它们主要是分开处理的。提出的研究将描述可靠数据存储和快速数据访问之间的相互作用，并开发方法来共同优化云存储的这两个主要目标。项目团队将开发分布式存储技术的设计、分析和性能评估方法，以实现高可靠性、健壮性和快速数据检索。研究小组还将开发存储和分布式计算方案，以优化数据访问的可靠性和延迟。在这个项目过程中开发的方法将允许系统运营商向用户提供他们的内容和服务，并支持从个人视频流到使用在线协作工具的延迟敏感应用。这项研究将最大限度地减少数据中心的能源需求，近年来数据中心的能源需求大幅增加。该项目将为编码理论、信息论和排队论的更广泛领域做出贡献，并开辟异花授粉的新途径。该项目侧重于分布式文件系统中的高效数据访问，该系统采用代码进行可靠和高效的存储。云系统的用户要求他们的内容和服务随时可用，他们的数据可靠地存储。虽然这两个目标之间有明显的联系和权衡，但到目前为止，它们主要是分开处理的。这一建议是在分析了今天的一些发现之后提出的？在某些访问模式下，可靠的数据存储方案会影响数据下载的速度。这项初步研究表明，在某些情况下，用于提高存储可靠性的编码方案可以进一步用于快速数据访问，而在其他情况下，表面上增加数据可用性的编码方案实际上无法提供对流行内容的有效访问(所谓的？热数据?)。提出的研究旨在描述可靠数据存储和快速数据访问之间的相互作用，并开发联合优化云存储这两个主要目标的方法。提出的研究将首先确定和设计编码数据访问的方案，并推导这些方案的预期下载时间（边界和估计）。无论使用哪种数据访问方案，预期的下载时间都将取决于实际的服务模型以及分布式系统服务容量供应和分配方案，然后对其进行处理。这项工作还将侧重于拟议的研究与高效分布式计算和减少数据中心能源消耗领域之间的联系。初步结果表明，这些领域是紧密相连的，一个领域开发的技术可以使其他领域受益。

项目成果

Straggler Mitigation at Scale

• DOI: 10.1109/tnet.2019.2946464
• 发表时间: 2019-12-01
• 期刊: IEEE-ACM TRANSACTIONS ON NETWORKING
• 影响因子: 3.7
• 作者: Aktas, Mehmet Fatih;Soljanin, Emina
• 通讯作者: Soljanin, Emina

Evaluating Load Balancing Performance in Distributed Storage With Redundancy

评估具有冗余的分布式存储中的负载平衡性能

• DOI: 10.1109/tit.2021.3054385
• 发表时间: 2021
• 期刊: IEEE Transactions on Information Theory
• 影响因子: 2.5
• 作者: Aktas, Mehmet Fatih;Far, Amir Behruzi;Soljanin, Emina;Whiting, Philip
• 通讯作者: Whiting, Philip

A Geometric View of the Service Rates of Codes Problem and its Application to the Service Rate of the First Order Reed-Muller Codes

编码服务率问题的几何观及其在一阶Reed-Muller码服务率中的应用

• DOI: 10.1109/isit44484.2020.9174345
• 发表时间: 2020
• 期刊: 2020 IEEE International Symposium on Information Theory (ISIT'20
• 作者: Kazemi, Fatemeh;Kurz, Sascha;Soljanin, Emina
• 通讯作者: Soljanin, Emina

Diversity/Parallelism Trade-Off in Distributed Systems With Redundancy

• DOI: 10.1109/tit.2021.3127920
• 发表时间: 2020-10
• 期刊: IEEE Transactions on Information Theory
• 影响因子: 2.5
• 作者: Pei Peng;E. Soljanin;P. Whiting

Service Rate Region of Content Access from Erasure Coded Storage

• DOI: 10.1109/itw.2018.8613504
• 发表时间: 2018-11
• 期刊: 2018 IEEE Information Theory Workshop (ITW)
• 作者: S. Anderson;A. Johnston;Gauri Joshi;Gretchen L. Matthews;Carolyn Mayer;E. Soljanin