CIF: Small: Distributed Machine Learning in the Age of Fast Data Streams

最近的技术进步导致了许多分散/分布式系统的出现，这些系统由相互连接的组件组成，这些组件通过无线链路和互联网主干相互通信，以进行协调和决策。此类系统的示例包括传感器网络、物联网（IoT）系统、多代理系统、高性能计算集群和联邦计算系统。许多这些分布式系统的一个定义特征是通过单个系统组件（例如，mote，机器人，物联网设备，蜂窝塔，GPU节点等）不断收集新数据样本。这些系统的几个用例，范围从智能农业和智能家居到智能电网和智能交通，正在设想通过采用复杂的机器学习技术，从传入的分布式“数据流”中实时提取可操作的信息。但是，世界对数据的需求不断增长，再加上传感、存储、计算和带宽的价格和容量预测，在不久的将来，（可负担的）带宽容量将开始落后于分布式系统收集新数据样本的速度。这样的未来对分布式系统来说并不是一个好兆头，因为分布式系统需要依靠机器学习的进步来进行有效的决策。因此，开发能够在（相对）低吞吐量通信链路上操作的同时容纳大量数据的分布式学习策略至关重要。该项目解决了这一挑战，并提供了一套全面的分析和算法框架，用于从（可能损坏的）数据（以（极）快流的形式到达多个相互关联的实体）中进行通信感知和基于优化的分布式机器学习。在这样做的过程中，该项目通过最先进的分布式系统的进步直接使国民经济受益，这将导致减少能源成本和浪费，提高工业效率，更好地遏制环境灾害，有效地监测国家基础设施等。此外，该项目还将通过培训两名研究生和几名本科生，帮助解决机器学习和数据科学关键领域人才短缺的问题。该项目开发并分析了一种用于实时网络内机器学习的算法框架，该框架承认并解释了许多新兴应用中通信速率和分布式数据流速率之间的不匹配，在这些应用中，连续数据收集成本很低，通信是通过无基础设施的设备对设备和/或机器对机器链接进行的。研究者将这种设置形式化为分布式随机逼近问题，其中使用随机数据流迭代训练最佳机器学习模型到各个设备和机器。然后，研究的重点是设计和分析在（极）快的流媒体速率下运行的协作策略。这些策略考虑了网络的拓扑结构，通信和数据流速率之间不匹配的严重程度，以及学习问题的凹凸性和结构。此外，它们还解释了现实世界网络和数据收集的挑战，包括间歇性通信链路、异构数据模式、相关数据流以及损坏或丢失的数据。其结果是一套全面的技术和分析，提供快速、可靠的学习和对网络学习性能的透彻理解。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent technological advances have resulted in the emergence of many decentralized/distributed systems comprising interconnected components that communicate among themselves over wireless links and the internet backbone for coordination and decision making. Examples of such systems include sensor networks, Internet-of-Things (IoT) systems, multiagent systems, high-performance computing clusters, and federated computing systems. One defining characteristics of many of these distributed systems is the continuous gathering of new data samples by the individual system components (e.g., motes, robots, IoT devices, cell towers, GPU nodes, etc.). Several use cases of these systems, which range from smart agriculture and smart homes to smart grids and smart transportation, are being envisioned that extract actionable information in real time from the incoming distributed "data streams" through adoption of sophisticated machine learning techniques. But the world's growing appetite for data coupled with the price and capacity projections for sensing, storage, computation, and bandwidth point to a near future in which (affordable) bandwidth capacity will start lagging behind the rate at which distributed systems gather new data samples. Such a future does not bode well for distributed systems that are expected to rely on machine learning advances for effective decision making. As such, it is crucial to develop distributed learning strategies that accommodate high volumes of data while operating over (relatively) low-throughput communication links. This project addresses this challenge and delivers a comprehensive set of analytical and algorithmic frameworks for communications-aware and optimization-based distributed machine learning from (possibly corrupted) data arriving in the form of (extremely) fast streams at multiple interconnected entities. In doing so, the project directly benefits the national economy through advances in the state-of-the-art in distributed systems, which will lead to reduction in both energy costs and wastage, increase in industrial efficiency, better containment of environmental disasters, efficient monitoring of nation's infrastructure, etc. Further, this project will also help address the shortage of talent in the critical areas of machine learning and data science by training two graduate and several undergraduate students. This project develops and analyzes an algorithmic framework for real-time, in-network machine learning that acknowledges and accounts for the mismatch between the communications rate and the rate of distributed data streams in many emerging applications, where continuous data gathering is cheap and communications is over infrastructure-free device-to-device and/or machine-to-machine links. The investigator formalizes this setting as a distributed stochastic approximation problem, in which the optimum machine learning model is iteratively trained using the random data streaming into individual devices and machines. The research then focuses on the design and analysis of collaborative strategies that operate in the regime of (extremely) fast streaming rates. These strategies account for the topology of the network, the severity of the mismatch between communications and data streaming rates, and the convexity and structure of the learning problem. Further, they account for the challenges of real-world networks and data gathering, including intermittent communications links, heterogeneous data modalities, correlated data streams, and corrupt or missing data. The result is a comprehensive set of techniques and analysis that provide fast, reliable learning and a thorough understanding of network learning performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.