Collaborative Research: CIF: Small: Versatile Data Synchronization: Novel Codes and Algorithms for Practical Applications

据估计，创建或复制的全球数字数据总量大约每三年翻一番。这种快速增长导致了对个人、企业和科学环境中可靠且通用的数据访问的需求日益增长。为了满足这些要求，数据同步已成为管理数据的关键方面。数据同步指的是在不同主机上存储的不同数据版本之间保持一致性的过程。然而，最先进的同步工具存在严重的缺点和低效，导致成本增加和高延迟访问。该项目旨在开发基于纠错码的具有最佳通信带宽的数据同步算法，并将同步的适用性扩大到当前工具不足的实际环境中。除了科学和技术进步外，该项目还有可能促进农村地区等宽带互联网接入有限的用户访问分布式存储系统；帮助减少与数据传输相关的能源消耗；并提供接触和培训本科生研究人员的机会。该项目的目标将通过三个研究项目来实现。第一个推力旨在通过设计低冗余的系统编辑纠错码以及高效的编解码算法来提高数据同步的效率。第二个重点是同步压缩数据。由于传统压缩通常会破坏相关文件之间的相似性，该项目将开发相互兼容的压缩和同步方法，鉴于数据压缩的盛行，这种方法有可能大大扩大对大型数据集的同步使用。最后，第三个重点将从理论和实践的角度解决经常被忽视的现实世界对同步的限制。特别是，在一方受到通信或计算限制的情况下，将确定信息交换的界限。此外，将开发增量和自适应同步协议，以在管理数据更新和修改的随机过程的统计数据未知时高效地同步数据。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The total volume of global digital data created or copied is estimated to double approximately every three years. This rapid growth has led to an increasing need for reliable and universal access to data in personal, enterprise, and scientific environments. To meet these requirements, data synchronization, which refers to the process of maintaining consistency between different versions of data stored on separate hosts, has become a crucial aspect of managing data. However, state-of-the-art synchronization tools have significant shortcomings and inefficiencies, resulting in increased costs and high-latency access. This project aims to develop data synchronization algorithms with optimal communication bandwidth based on error-correcting codes and to broaden the applicability of synchronization to real-world settings where current tools are inadequate. In addition to scientific and technological advances, the project has the potential to facilitate access to distributed storage systems for users with limited access to broadband Internet, such as in rural areas; help reduce energy consumption associated with data transmission; and provide opportunities to engage and train undergraduate researchers. The goals of the project will be achieved through three research thrusts. The first thrust aims to increase the efficiency of data synchronization by designing low-redundancy systematic edit-correcting codes, along with efficient encoding and decoding algorithms. The second thrust focuses on synchronizing compressed data. As conventional compression typically destroys the similarity between related files, the project will develop mutually compatible compression and synchronization methods which, given the prevalence of data compression, have the potential to significantly expand the use of synchronization for large datasets. Finally, the third thrust will address often-overlooked real-world constraints on synchronization from theoretical and practical points of view. In particular, bounds on the information exchange will be established where one party is under communication or computational constraints. Furthermore, incremental and adaptive synchronization protocols will be developed to efficiently synchronize data when the statistics of the stochastic processes governing data update and modification are unknown.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.