Collaborative Research: CIF: Small: Versatile Data Synchronization: Novel Codes and Algorithms for Practical Applications

据估计，创建或复制的全球数字数据总量大约每三年翻一番。这种快速增长导致对个人、企业和科学环境中的数据的可靠和普遍访问的需求日益增加。为了满足这些要求，数据同步（指的是在存储在不同主机上的不同版本的数据之间保持一致性的过程）已成为管理数据的一个重要方面。然而，最先进的同步工具具有显著的缺点和效率低下，导致成本增加和高延迟访问。该项目旨在开发基于纠错码的具有最佳通信带宽的数据同步算法，并将同步的适用性扩大到当前工具不足的现实世界环境。除了科学和技术进步，该项目还有可能促进农村地区等宽带互联网接入有限的用户使用分布式存储系统;帮助减少与数据传输有关的能源消耗;并提供机会吸引和培训本科研究人员。该项目的目标将通过三个研究重点来实现。第一个目标是通过设计低冗余系统编辑校正码沿着有效的编码和解码算法来提高数据同步的效率。第二个重点是同步压缩数据。由于传统的压缩通常会破坏相关文件之间的相似性，该项目将开发相互兼容的压缩和同步方法，鉴于数据压缩的普遍性，这些方法有可能大大扩大大型数据集同步的使用。最后，第三个推力将解决经常被忽视的现实世界的限制同步从理论和实践的角度来看。特别是，信息交换的界限将建立在一方通信或计算的限制。此外，增量和自适应同步协议将被开发，以有效地同步数据时，统计的随机过程管理的数据更新和修改是unknow.This奖项反映了NSF的法定使命，并已被认为是值得的支持，通过评估使用基金会的智力价值和更广泛的影响审查标准。

The total volume of global digital data created or copied is estimated to double approximately every three years. This rapid growth has led to an increasing need for reliable and universal access to data in personal, enterprise, and scientific environments. To meet these requirements, data synchronization, which refers to the process of maintaining consistency between different versions of data stored on separate hosts, has become a crucial aspect of managing data. However, state-of-the-art synchronization tools have significant shortcomings and inefficiencies, resulting in increased costs and high-latency access. This project aims to develop data synchronization algorithms with optimal communication bandwidth based on error-correcting codes and to broaden the applicability of synchronization to real-world settings where current tools are inadequate. In addition to scientific and technological advances, the project has the potential to facilitate access to distributed storage systems for users with limited access to broadband Internet, such as in rural areas; help reduce energy consumption associated with data transmission; and provide opportunities to engage and train undergraduate researchers. The goals of the project will be achieved through three research thrusts. The first thrust aims to increase the efficiency of data synchronization by designing low-redundancy systematic edit-correcting codes, along with efficient encoding and decoding algorithms. The second thrust focuses on synchronizing compressed data. As conventional compression typically destroys the similarity between related files, the project will develop mutually compatible compression and synchronization methods which, given the prevalence of data compression, have the potential to significantly expand the use of synchronization for large datasets. Finally, the third thrust will address often-overlooked real-world constraints on synchronization from theoretical and practical points of view. In particular, bounds on the information exchange will be established where one party is under communication or computational constraints. Furthermore, incremental and adaptive synchronization protocols will be developed to efficiently synchronize data when the statistics of the stochastic processes governing data update and modification are unknown.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.