Advanced coding solutions for large-scale data storage & communication

动机 由于对云存储、视频点播、直播和文件共享等在线服务的需求增加，全球IP流量正在迅速增长。这些在线服务负责大部分互联网流量，严重依赖分布式存储系统（DSS）。因此，在世界各地，DSS的数量和规模近年来迅速增加。事实上，DSS已经是现代世界碳排放的主要贡献者之一。例如，在美国，它们仅次于航空业，预计到2020年将成为最大的贡献者。因此，DSS的高效数据存储技术的研究最近吸引了学术界和工业界的极大兴趣。 背景 DSS中的硬件和软件故障会导致数据丢失和不可用。当然，许多服务提供商为了可靠性和可用性而保留多个数据副本。当然，缺点是巨大的存储开销（通常为200%）和相关的维护成本。作为一种解决方案，最近，纠错编码被建议用于DSS。不幸的是，大多数现有的纠错码不能在这些大规模的应用中使用，因为它们创建的修复流量超出了DSS的网络容量。因此，强烈需要更有效的代码，适合大规模的应用程序，感觉。 目标 在这项研究计划中，利用我们过去对现代编码的了解和我们最近对DSS编码的活动，我们将为分布式存储系统开发高效和可扩展的纠错编码解决方案。我们还将寻求不同的性能指标（如编码开销，代码可靠性，修复流量，更新复杂性等）之间的基本关系的理论结果。 在这个领域有大量的高影响力的开放问题，都适合培训HQP。 影响 这个研究计划遵循了几个目标，这些目标对编码和信息理论领域的实践者和理论家都非常重要。提高大规模数据存储的效率不仅可以改善我们的在线体验，还可以显著减少DSS的碳足迹。我们希望我们在整个工作中的成果以及我们与其他加拿大研究人员的贡献能够将加拿大变成该领域的主要科学中心，可以在技术和科学上支持位于加拿大的任何数据中心。事实上，由于冷却是数据中心的主要成本，加拿大因其较冷的气候和安全性而被认为是托管数据中心的理想场所。 在过去十年中，全世界的在线活动显著增加。因此，大规模数据存储/通信目前是并将继续是一个巨大的工程挑战。我们在这一领域的活动将培养高需求的HQP，并参与非常高影响力的工程活动。

Motivation The global IP traffic is growing rapidly due to increased demand for online services such as cloud storage, video on demand, live streaming, and file sharing. These online services, which are responsible for the bulk of the Internet traffic, heavily rely on distributed storage systems (DSS). Hence, around the world, the number and size of DSSs have rapidly increased in recent years. In fact, DSSs are already among the top contributors of carbon emission in the modern world. In the US, for example, they are only second to the airline industry and expected to be the top contributor by 2020. Thus, research on efficient data storage techniques for DSSs has recently attracted a lot of interest both in academia and in industry. Background Hardware and software failures in a DSS result in data loss and unavailability. Naturally, many service providers keep multiple copies of data for reliability and availability. The downside, of course, is the huge storage overhead (typically 200%) and the associated maintenance costs. As a solution, recently, error correction coding is suggested for DSSs. Unfortunately, most existing error correction codes cannot be used in these large-scale applications, because the repair traffic that they create is beyond the network capacity of the DSSs. Hence, a strong need for more efficient codes, tailored for large-scale applications, is felt. Goals In this research program, using our past knowledge of modern coding and our recent activities on coding for DSSs, we will develop efficient and scalable error correction coding solutions for distributed storage systems. We will also seek theoretical results on the fundamental relations among different performance measures (such as coding overhead, code reliability, repair traffic, update complexity, etc.). There are a large number of high-impact open problems in this area all suitable for training HQPs. Impact This research proposal follows several goals that are exceedingly important to both practitioners and theorists in the area of coding and information theory. Improving the efficiency of large-scale data storage not only improves our online experience, but also significantly reduces the carbon footprint of DSSs. We hope our results throughout this work and our contribution with other Canadian researchers turn Canada into a major scientific center in this field, which can technically and scientifically support any data center located in Canada. In fact, since cooling is a major cost in data centers, Canada because of its colder climate and its safety is considered as an ideal place to host data centers. In the past decade, the world has seen a significant increase in online activity. As a result, large-scale data storage/communication currently is and will continue to be a big engineering challenge. Our activity in this field will train HQPs that will be in high demand and involved in very high impact engineering activities.