CIF: Small: Collaborative Research:Synchronization and Deduplication of Distributed Coded Data: Fundamental Limits and Algorithms
CIF: Small: Collaborative Research:Synchronization and Deduplication of Distributed Coded Data: Fundamental Limits and Algorithms
批准号:
1526875
负责人:
Olgica Milenkovic
金额:
$15.0万
依托单位国家:
美国
项目类别:
Standard Grant
财政年份:
2015
资助国家:
美国
项目状态:
已结题
起止时间:
2015-07-01 至 2021-06-30
中文摘要
第一部分:由于信息技术的快速发展和需要在大规模网络中存储和传播的大数据格式的出现,分布式存储系统的编码在过去几年中引起了极大的关注。由于典型的分布式系统需要确保通过通信网络连接的一组节点上的低延迟数据访问和存储大量文件,因此开发新的分布式编码方案以保护系统免受意外组件故障的影响是必要的。分布式系统中使用的代码的两个关键功能,即通过访问节点子集来重建文件和修复故障节点,需要在用户通过符号/块插入、删除或替换编辑访问和处理文件时保留。系统级重复数据删除导致数据删除频繁,当对部分文件进行重复数据删除或编辑时,信息内容的变化需要以最小的通信成本传递给冗余存储节点。当前用于同步经过编辑的数据的解决方案假设数据是未编码的,并且它们没有充分利用信息的分布式特性。此外,它们大多忽略了重复数据删除协议的存在。这使得分布式存储体系结构在存储、用户访问时间和错误保护方面效率低下。因此,提出的研究计划的目标是开发一套新的协议和编码方案,以支持新一代通用和可更新的编码分布式存储系统。第2部分:基于研究人员的初步工作,本提案旨在为编码同步和重复数据删除的新领域奠定基础,目标是推导基本的性能限制,为这两类问题开发有效的算法解决方案,并构建新的分布式存储代码,使编码数据同步和编码重复数据删除成为可能。特别是,该提案解决了以下综合问题:1)描述了已知和新的(非)编码同步方案的通信速率限制,不同结构化或编码数据格式和不同类型的通信通道的重复数据删除和数据修复性能之间的权衡。2)介绍并分析了用于结构化/编码数据的近似同步和重复数据删除的通信速率失真(CRD)功能,特别关注延迟敏感应用。3)在图像和视频数据编码中,根据网络拓扑和用户不同的优先级需求,开发可动态更新的同步和重复数据删除算法。
英文摘要
Part 1: Coding for distributed storage systems has garnered significant attention in the past few years due to the rapid development of information technologies and the emergence of Big Data formats that need to be stored and disseminated across large-scale networks. As typical distributed systems need to ensure low-latency data access and store a large number of files over a set of nodes connected through a communication network, it is imperative to develop new distributed coding schemes that protect the systems from undesired component failures. The two key functionalities of codes used in distributed systems, namely the reconstruction of files via access to a subset of the nodes and repair of failed nodes, need to be retained when the files are accessed and processed by the users via symbol/block insertion, deletion, or substitution edits. Deletions frequently arise due to system-level data deduplication: when parts of files are deduplicated or edited, the changes in the information content need to be communicated to the redundant storage nodes with minimum communication cost. Current solutions for synchronizing data that underwent edits assume that data is uncoded and they do not fully exploit the distributed nature of information. Furthermore, they mostly ignore the presence of deduplication protocols. This makes distributed storage architectures inefficient in terms of storage, user access times, and error protection. Hence, the goals of the proposed research program are to develop a new set of protocols and coding schemes that will support a new generation of versatile and updatable coded distributed storage systems. Part 2: Building on the preliminary work of the investigators, this proposal aims to set the foundations of the new field of coded synchronization and deduplication, with the goal of deriving fundamental performance limits, developing efficient algorithmic solutions for the two families of problems, and constructing new distributed storage codes that enable synchronization of coded data and coded deduplication. In particular, the proposal addresses the following comprehensive issues: 1) Characterizing the communication rate limits of known and new (un)coded synchronization schemes, trade-offs between deduplication and data repair performance for different structured or encoded data formats and different types of communication channels.2) Introducing and analyzing the communication rate-distortion (CRD) function for approximate synchronization and deduplication of structured/encoded data, with a special focus on delay-sensitive applications.3) Developing dynamically updatable synchronization and deduplication algorithms cognizant of the network topology and of different prioritization needs of the users, as encountered in image and video data coding.
期刊论文(0)
专著(0)
科研奖励(0)
会议论文
Collaborative Research: CIF-Medium: Privacy-preserving Machine Learning on Graphs
-
批准号:2402815
-
项目类别:Standard Grant
-
资助金额:$40.0万
-
财政年份:2024
-
负责人:Olgica Milenkovic
-
依托单位:
Collaborative Research: CIF: Medium: Group testing for Real-Time Polymerase Chain Reactions: From Primer Selection to Amplification Curve Analysis
-
批准号:2107344
-
项目类别:Continuing Grant
-
资助金额:$60.0万
-
财政年份:2021
-
负责人:Olgica Milenkovic
-
依托单位:
Collaborative Research: CIF: Small: Coded String Reconstruction Problems in Molecular Storage
-
批准号:2008125
-
项目类别:Standard Grant
-
资助金额:$23.15万
-
财政年份:2020
-
负责人:Olgica Milenkovic
-
依托单位:
Collaborative Research: CIF: Medium: New Methods for Learning on Hypergraphs for Single-Cell Chromatin Data Analysis
-
批准号:1956384
-
项目类别:Continuing Grant
-
资助金额:$50.81万
-
财政年份:2020
-
负责人:Olgica Milenkovic
-
依托单位:
CIF: Small: Collaborative Research:Leveraging Data Popularity in Distributed Storage Systems via Constrained Design Theory
-
批准号:1816913
-
项目类别:Standard Grant
-
资助金额:$25.0万
-
财政年份:2018
-
负责人:Olgica Milenkovic
-
依托单位:
SemiSynBio: An On-Chip Nanoscale Storage System Using Chimeric DNA
-
批准号:1807526
-
项目类别:Continuing Grant
-
资助金额:$200.0万
-
财政年份:2018
-
负责人:Olgica Milenkovic
-
依托单位:
CIF: Small: Coding for DNA-Based Storage Systems
-
批准号:1618366
-
项目类别:Standard Grant
-
资助金额:$50.0万
-
财政年份:2016
-
负责人:Olgica Milenkovic
-
依托单位:
CIF: Small: Collaborative Research: Ordinal Data Compression
-
批准号:1527636
-
项目类别:Standard Grant
-
资助金额:$25.0万
-
财政年份:2015
-
负责人:Olgica Milenkovic
-
依托单位:
CIF: Small: Collaborative Research: A General Theory of Group Testing for Genotyping
-
批准号:1218764
-
项目类别:Standard Grant
-
资助金额:$24.99万
-
财政年份:2012
-
负责人:Olgica Milenkovic
-
依托单位:
CIF: Small: Nonlinear Matrix and Tensor Completion with Applications in Systems Biology
-
批准号:1117980
-
项目类别:Standard Grant
-
资助金额:$47.8万
-
财政年份:2011
-
负责人:Olgica Milenkovic
-
依托单位:
Collaborative Research: Design and Analysis of Compressed Sensing DNA Microarrays
-
批准号:0821910
-
项目类别:Continuing Grant
-
资助金额:$29.7万
-
财政年份:2007
-
负责人:Olgica Milenkovic
-
依托单位:
CAREER: Coding Theoretic Problems in Genetic Data Acquisition, Modeling and Analysis
-
批准号:0809895
-
项目类别:Continuing Grant
-
资助金额:$36.65万
-
财政年份:2007
-
负责人:Olgica Milenkovic
-
依托单位:
Collaborative Research: Design and Analysis of Compressed Sensing DNA Microarrays
-
批准号:0729216
-
项目类别:Continuing Grant
-
资助金额:$29.7万
-
财政年份:2007
-
负责人:Olgica Milenkovic
-
依托单位:
CAREER: Coding Theoretic Problems in Genetic Data Acquisition, Modeling and Analysis
-
批准号:0644427
-
项目类别:Continuing Grant
-
资助金额:$39.97万
-
财政年份:2007
-
负责人:Olgica Milenkovic
-
依托单位:
Collaborative Research: Constrained and Error-Control Coding for DNA Computers
-
批准号:0514875
-
项目类别:Standard Grant
-
资助金额:$8.77万
-
财政年份:2005
-
负责人:Olgica Milenkovic
-
依托单位:
国内基金
海外基金
登录
查看更多内容
昼夜节律性small RNA在血斑形成时间推断中的法医学应用研究
-
批准号:
-
项目类别:省市级项目
-
资助金额:--
-
批准年份:2024
-
负责人:
-
依托单位:
tRNA-derived small RNA上调YBX1/CCL5通路参与硼替佐米诱导慢性疼痛的机制研究
-
批准号:
-
项目类别:省市级项目
-
资助金额:10.0万元
-
批准年份:2022
-
负责人:张祥忠
-
依托单位:
Small RNA调控I-F型CRISPR-Cas适应性免疫性的应答及分子机制
-
批准号:32000033
-
项目类别:青年科学基金项目
-
资助金额:24.0万元
-
批准年份:2020
-
负责人:林平
-
依托单位:
Small RNAs调控解淀粉芽胞杆菌FZB42生防功能的机制研究
-
批准号:31972324
-
项目类别:面上项目
-
资助金额:58.0万元
-
批准年份:2019
-
负责人:高学文
-
依托单位:
变异链球菌small RNAs连接LuxS密度感应与生物膜形成的机制研究
-
批准号:81900988
-
项目类别:青年科学基金项目
-
资助金额:21.0万元
-
批准年份:2019
-
负责人:毛梦莹
-
依托单位:
肠道细菌关键small RNAs在克罗恩病发生发展中的功能和作用机制
-
批准号:31870821
-
项目类别:面上项目
-
资助金额:56.0万元
-
批准年份:2018
-
负责人:陈江宁
-
依托单位:
基于small RNA 测序技术解析鸽分泌鸽乳的分子机制
-
批准号:31802058
-
项目类别:青年科学基金项目
-
资助金额:26.0万元
-
批准年份:2018
-
负责人:麻慧
-
依托单位:
Small RNA介导的DNA甲基化调控的水稻草矮病毒致病机制
-
批准号:31772128
-
项目类别:面上项目
-
资助金额:60.0万元
-
批准年份:2017
-
负责人:吴建国
-
依托单位:
基于small RNA-seq的针灸治疗桥本甲状腺炎的免疫调控机制研究
-
批准号:81704176
-
项目类别:青年科学基金项目
-
资助金额:20.0万元
-
批准年份:2017
-
负责人:赵继梦
-
依托单位:
水稻OsSGS3与OsHEN1调控small RNAs合成及其对抗病性的调节
-
批准号:91640114
-
项目类别:重大研究计划
-
资助金额:85.0万元
-
批准年份:2016
-
负责人:何祖华
-
依托单位: