CNS Core: Small: Closing the Reality Gap for Learning-Augmented Network Systems

CNS 核心：小型：缩小学习增强网络系统的现实差距

• 批准号: 2131826
• 负责人: Junchen Jiang
• 金额: $ 50万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2131826&HistoricalAwards=false
• 关键词: CNS; Core; Small; Closing; Reality

Modern Internet applications rely on sophisticated algorithms and systems to share network resources and deliver high quality of experience to each user (e.g., fast loading of web pages and smooth high-resolution video streaming). For instance, a video streaming system monitors the current speed of a user's internet connection and constantly changes the video quality to ensure smooth streaming at a high video quality. A key challenge of these systems is to ensure desirable user experience under different network environments, including different network speeds and different levels of network bandwidth changes. With the recent advances in machine learning (which makes predictions from data without following explicit instructions), many industry operators and researchers are exploring a new approach that automatically trains these algorithms as machine-learning models. While these learning-based systems show good performance in network environments similar to those the algorithms are trained in, they often do not perform well in new real-world network environments. Therefore, as new learning-based systems are developed and deployed every year, improving their generalization has become increasingly pressing.The goal of this project is to create a reusable framework to enhance the generalization of learning-based network systems. It focuses on systems that use deep reinforcement learning (DRL), and to improve their generalization, it applies formal tools from the machine learning literature and makes them efficient and effective for network systems by leveraging the traditional rule-based heuristics in the networking literature. The insight is that compared to DRL policies, rule-based heuristics (though suboptimal in some workloads) are less sensitive to differences between real systems/workloads and the simulated training environments and are more trusted by network operators. The project has three synergistic research thrusts. (1) It explores the use of rule-based heuristics to identify an appropriate level of randomization that should be introduced to the simulation-based training, in order to make the simulator-trained policies perform well in the real world. (2) To allow the offline-trained policies to generalize to a large and diverse operational space, the project iteratively improves the trained policy by periodically promoting difficult, yet improvable environments indicated by the performance of rule-based heuristics. (3) To cope with environment drifts in real network systems, the project proposes to run a fail-safe rule-based logic to collect the feedback data and use it to re-train the DRL policy in an unbiased and data-efficient fashion.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代互联网应用依赖于复杂的算法和系统来共享网络资源并向每个用户提供高质量的体验（例如，网页的快速加载和流畅的高分辨率视频流）。例如，视频流传输系统监控用户的互联网连接的当前速度，并不断改变视频质量，以确保高视频质量下的流畅流传输。这些系统的关键挑战是确保在不同网络环境（包括不同网络速度和不同级别的网络带宽变化）下的期望的用户体验。随着机器学习的最新进展（它在不遵循明确指令的情况下根据数据进行预测），许多行业运营商和研究人员正在探索一种新的方法，将这些算法自动训练为机器学习模型。虽然这些基于学习的系统在类似于算法训练的网络环境中表现出良好的性能，但它们在新的现实网络环境中往往表现不佳。因此，随着每年新的基于学习的系统的开发和部署，提高其泛化能力变得越来越紧迫。本项目的目标是创建一个可重用的框架，以提高基于学习的网络系统的泛化能力。它专注于使用深度强化学习（DRL）的系统，为了提高其泛化能力，它应用了机器学习文献中的形式化工具，并通过利用网络文献中传统的基于规则的算法，使其对网络系统高效有效。与DRL策略相比，基于规则的策略（尽管在某些工作负载中不是最佳的）对真实的系统/工作负载和模拟训练环境之间的差异不太敏感，并且更受网络运营商的信任。该项目有三个协同研究重点。 (1)它探讨了使用基于规则的算法来确定一个适当的随机化水平，应该引入到基于模拟的训练，以使模拟器训练的政策在真实的世界中表现良好。(2)为了让离线训练的策略推广到一个大而多样的操作空间，该项目通过定期推广困难但可改进的环境来迭代改进训练的策略，这些环境由基于规则的算法的性能指示。(3)为了科普真实的网络系统中的环境漂移，该项目建议运行一个基于故障安全规则的逻辑来收集反馈数据，并使用它来重新训练DRL政策，以公正和数据高效的方式。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

项目成果

Genet: automatic curriculum generation for learning adaptation in networking

• DOI: 10.1145/3544216.3544243
• 发表时间: 2022-02
• 期刊: Proceedings of the ACM SIGCOMM 2022 Conference
• 作者: Zhengxu Xia;Yajie Zhou;Francis Y. Yan;Junchen Jiang

Estimating WebRTC Video QoE Metrics Without Using Application Headers

• DOI: 10.1145/3618257.3624828
• 发表时间: 2023-06
• 期刊: Proceedings of the 2023 ACM on Internet Measurement Conference
• 作者: Taveesh Sharma;Tarun Mangla;Arpit Gupta;Junchen Jiang;N. Feamster