Encoded computing for efficient and robust large-scale distributed optimization

用于高效、稳健的大规模分布式优化的编码计算

• 批准号: RGPIN-2019-05828
• 负责人: Draper, Stark
• 金额: $ 3.35万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=751581
• 关键词: Encoded; computing; efficient; robust; large

The availability of big data and the use of these data in training artificial intelligence (AI) systems is changing the way companies do business in a wide swath of industries, from finance to entertainment to drug discovery. These advances rely on a computing, networking, and data storage infrastructure, the growing capabilities of which are key to realizing the promise of AI. However, as data sets and processing requirements scale up massively, classic single-processor computing paradigms cannot keep up. There has therefore been a renaissance in understanding how to bring large-scale parallelization to bear on algorithmic design for learning-specific workloads. As we outline in this proposal, we can draw on techniques, perspectives, and solutions from digital communications and error-correction coding to develop novel robust and resource-efficient approaches to parallelized computing. Our long-term vision is that by drawing on the theory and practice of digital communications we can deliver novel and unexpected impact on the theory and practice of computation. In the shorter term, we will build towards our vision by following three coupled research themes. These themes draw on and contribute to the theory and practice of error-correction coding and distributed optimization to enable large-scale, robust, and efficient computing systems. In the first theme we design large-scale distributed optimization techniques that deliver idealized performance while coping with the non-idealities of real-world cloud computing systems including the effects of "straggler" nodes and network delays. In the second, we draw on perspectives from information theory and error correction to develop novel "coding-for-computing" paradigms that deliver robust and efficient computation. In the final theme we work closely with systems groups to help push our results into application and to improve our understanding of the myriad real-world issues facing computing systems, motivating further research and deepening collaborations.

大数据的可用性以及将这些数据用于训练人工智能（AI）系统，正在改变从金融到娱乐再到药物研发等众多行业的公司开展业务的方式。这些进步依赖于计算、网络和数据存储基础设施，这些基础设施的不断增长的能力是实现人工智能承诺的关键。然而，随着数据集和处理需求的大规模扩展，经典的单处理器计算范式无法跟上。因此，在理解如何将大规模并行化引入特定于学习工作负载的算法设计方面出现了复兴。正如我们在本提案中概述的那样，我们可以利用数字通信和纠错编码的技术、观点和解决方案来开发新的健壮且资源高效的并行计算方法。我们的长期愿景是，通过利用数字通信的理论和实践，我们可以对计算的理论和实践产生新颖和意想不到的影响。在短期内，我们将通过以下三个相互关联的研究主题来实现我们的愿景。这些主题借鉴并促进了纠错编码和分布式优化的理论和实践，以实现大规模、健壮和高效的计算系统。在第一个主题中，我们设计了大规模分布式优化技术，以提供理想的性能，同时应对现实世界云计算系统的非理想性，包括“离散”节点和网络延迟的影响。在第二部分中，我们借鉴了信息论和纠错的观点，开发了新的“为计算而编码”范式，提供了鲁棒和高效的计算。在最后一个主题中，我们与系统小组密切合作，帮助将我们的成果推向应用，并提高我们对计算系统面临的无数现实世界问题的理解，从而推动进一步的研究和深化合作。