CIF: Small: Distributed Machine Learning in the Age of Fast Data Streams

最近的技术进步导致了许多分散/分布式系统的出现，这些系统包括相互连接的组件，这些组件之间通过无线链路和互联网主干进行通信，以进行协调和决策。这类系统的例子包括传感器网络、物联网(IoT)系统、多代理系统、高性能计算集群和联合计算系统。这些分布式系统中的许多系统的一个定义特征是由各个系统组件(例如，微尘、机器人、物联网设备、蜂窝塔、GPU节点等)不断收集新的数据样本。这些系统的几个用例，从智能农业和智能家居到智能电网和智能交通，正在被设想通过采用复杂的机器学习技术从传入的分布式“数据流”中实时提取可操作的信息。但世界对数据日益增长的需求，再加上传感、存储、计算和带宽的价格和容量预测，预示着在不久的将来，(负担得起的)带宽容量将开始落后于分布式系统收集新数据样本的速度。这样的未来对分布式系统来说并不是一个好兆头，因为它们预计将依靠机器学习的进步来进行有效的决策。因此，开发分布式学习策略是至关重要的，这种策略在通过(相对)低吞吐量的通信链路运行的同时适应大量数据。该项目解决了这一挑战，并提供了一套全面的分析和算法框架，用于从以(极快的)数据流形式到达多个相互连接的实体的(可能被破坏的)数据进行通信感知和基于优化的分布式机器学习。通过这样做，该项目通过在分布式系统方面的最先进技术的进步直接造福于国家经济，这将导致能源成本和浪费的降低、工业效率的提高、更好地遏制环境灾害、对国家基础设施的有效监测等。此外，该项目还将通过培养两名研究生和几名本科生，帮助解决机器学习和数据科学关键领域的人才短缺问题。该项目开发并分析了实时网络内机器学习的算法框架，该框架承认并解决了许多新兴应用中的通信速率和分布式数据流的速率之间的不匹配，在这些应用中，持续的数据收集成本很低，并且通信是通过无基础设施的设备到设备和/或机器到机器的链路进行的。研究人员将这种设置形式化为分布式随机逼近问题，其中使用流入单个设备和机器的随机数据迭代训练最优机器学习模型。然后，研究重点放在设计和分析在(极)高速流媒体速率下运行的协作策略。这些策略考虑了网络的拓扑、通信和数据流速率之间不匹配的严重程度以及学习问题的凸性和结构。此外，它们还考虑了现实世界网络和数据收集的挑战，包括断断续续的通信链路、不同的数据模式、相关的数据流以及损坏或丢失的数据。其结果是一套全面的技术和分析，提供快速、可靠的学习和对网络学习性能的透彻理解。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent technological advances have resulted in the emergence of many decentralized/distributed systems comprising interconnected components that communicate among themselves over wireless links and the internet backbone for coordination and decision making. Examples of such systems include sensor networks, Internet-of-Things (IoT) systems, multiagent systems, high-performance computing clusters, and federated computing systems. One defining characteristics of many of these distributed systems is the continuous gathering of new data samples by the individual system components (e.g., motes, robots, IoT devices, cell towers, GPU nodes, etc.). Several use cases of these systems, which range from smart agriculture and smart homes to smart grids and smart transportation, are being envisioned that extract actionable information in real time from the incoming distributed "data streams" through adoption of sophisticated machine learning techniques. But the world's growing appetite for data coupled with the price and capacity projections for sensing, storage, computation, and bandwidth point to a near future in which (affordable) bandwidth capacity will start lagging behind the rate at which distributed systems gather new data samples. Such a future does not bode well for distributed systems that are expected to rely on machine learning advances for effective decision making. As such, it is crucial to develop distributed learning strategies that accommodate high volumes of data while operating over (relatively) low-throughput communication links. This project addresses this challenge and delivers a comprehensive set of analytical and algorithmic frameworks for communications-aware and optimization-based distributed machine learning from (possibly corrupted) data arriving in the form of (extremely) fast streams at multiple interconnected entities. In doing so, the project directly benefits the national economy through advances in the state-of-the-art in distributed systems, which will lead to reduction in both energy costs and wastage, increase in industrial efficiency, better containment of environmental disasters, efficient monitoring of nation's infrastructure, etc. Further, this project will also help address the shortage of talent in the critical areas of machine learning and data science by training two graduate and several undergraduate students. This project develops and analyzes an algorithmic framework for real-time, in-network machine learning that acknowledges and accounts for the mismatch between the communications rate and the rate of distributed data streams in many emerging applications, where continuous data gathering is cheap and communications is over infrastructure-free device-to-device and/or machine-to-machine links. The investigator formalizes this setting as a distributed stochastic approximation problem, in which the optimum machine learning model is iteratively trained using the random data streaming into individual devices and machines. The research then focuses on the design and analysis of collaborative strategies that operate in the regime of (extremely) fast streaming rates. These strategies account for the topology of the network, the severity of the mismatch between communications and data streaming rates, and the convexity and structure of the learning problem. Further, they account for the challenges of real-world networks and data gathering, including intermittent communications links, heterogeneous data modalities, correlated data streams, and corrupt or missing data. The result is a comprehensive set of techniques and analysis that provide fast, reliable learning and a thorough understanding of network learning performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.