Collaborative Research: CIF: Small: Nonasymptotic Analysis for Stochastic Networks and Systems: Foundations and Applications

云计算、机器学习/人工智能和物联网/传感器技术的融合正在以前所未有的方式改变社会，并导致自主系统、医疗保健、生物信息学、社交网络、在线和实体零售行业以及教育领域的创新。在这些广泛不同的领域，突破性的发展使用了机器学习和云计算，以及数百万台服务器来帮助数据驱动的决策，这些决策使用的是tb级的数据，有些是实时的，有些是离线的。这些大规模机器学习和云计算应用的核心是大规模的随机动态系统。分析和优化这样的系统往往是困难的，因为这类系统中潜在随机性的大小和未知的统计描述。该项目旨在通过开发一种新的分析方法来理解大型随机系统的性能，该方法综合了概率、机器学习和随机网络的工具，并将在设计用于训练大规模机器学习模型的快速、更有效的计算系统方面取得新进展，同时对深度强化学习算法产生新的基本见解。该项目将通过将理论和算法整合到研究生水平的课程中，并让本科生和来自代表性不足群体的学生参与研究，为教育和劳动力发展做出贡献。本文发展了一种利用李雅普诺夫漂移分析获得非渐近界的新解析方法。该方法将漂移分析与Stein方法的思想、状态空间坍缩的降维以及复制核希尔伯特空间的特性结合起来。该项目利用三个关键思想来推进最先进的技术：Stein的方法选择适当的Lyapunov函数来研究平均场极限，使用状态空间崩溃的概念识别低阶模型，并使用矩生成函数或特征函数作为测试函数来获得随机系统性能的高矩界。在这个项目的过程中，该方法被应用于两个应用程序：(i)鲁棒和超低延迟计算网络，用于支持具有并发和依赖任务的复杂机器学习作业，这些任务在异构服务器场中处理；（ii）深度强化学习，用于为神经时间差学习和Actor-Critic算法导出新的性能界限。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The confluence of cloud computing, machine learning/artificial intelligence and Internet-of-things/sensor technology is transforming society in unprecedented ways, and leading to innovations in autonomous systems, healthcare, bioinformatics, social networks, online and in-store retail industry, and education. Breakthrough developments in these widely disparate fields use machine learning and cloud computing with millions of servers to aid data-driven decision-making using terabytes of data, some in real-time and some offline. At the heart of these large-scale machine-learning and cloud-computing applications are stochastic dynamical systems of enormous scale. Analyzing and optimizing such systems are often difficult because of the size and the unknown statistical description of the underlying randomness in such systems. This project is aimed at understanding the performance of large stochastic systems by developing a new analytical method that synthesizes tools from probability, machine learning, and stochastic networks, and will lead to new advances in the design of fast and more efficient computing systems for training large-scale machine-learning models, while yielding new fundamental insights into deep reinforcement-learning algorithms. The project will contribute to education and workforce development by integrating the theories and algorithms into the graduate-level courses and by involving undergraduate and students from underrepresented groups in the research. This project develops a new analytical method for obtaining non-asymptotic bounds using Lyapunov drift analysis. The method combines drift analysis with ideas from Stein's method, dimensionality reduction from state-space collapse and properties of reproducing kernel Hilbert spaces, as appropriate. The project leverages three key ideas to advance the state-of-the-art: Stein's method to choose appropriate Lyapunov functions to study mean-field limits, identifying lower-order models using the notion of state-space collapse, and using moment-generating functions or characteristic functions as test functions to obtain higher-moment bounds on the performance of stochastic systems. During the course of this project, the method is applied to two applications: (i) robust and ultra-low latency computing networks for supporting complex machine-learning jobs with concurrent and dependent tasks, which are processed in heterogeneous server farms; and (ii) deep reinforcement-learning for deriving new performance bounds for neural temporal-difference learning and for the Actor-Critic algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.