CNS Core: Medium: Resource Constrained Reinforcement Learning for Computing Systems

CNS 核心：中：计算系统的资源受限强化学习

• 批准号: 1955997
• 负责人: Christina Yu
• 金额: $ 120万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955997&HistoricalAwards=false
• 关键词: CNS; Core; Medium; Resource; Constrained

Reinforcement Learning has surfaced as a promising algorithmic paradigm for online decision making and control. Recent efforts demonstrate impressive results, but with relatively unlimited computational resources, data, and training. However, many real-world decision-making and control problems require real-time decisions but are severely constrained in terms of throughput, latency, memory, and power. Existing algorithms are employed in software on general purpose processors, and do not translate well to real-timeresource-constrained computation in hardware. This project will develop new algorithmic paradigms as well as hardware acceleration primitives that work in sync to enable real-time reinforcement learning at scale in embedded applications. From the algorithmic perspective, this work will study efficiency in computational resource tradeoffs, and design reinforcement learning algorithms that efficiently adapt to given hardware constraints and hardware primitives. From the hardware perspective, this work will design new hardware primitives that support the developed reinforcement learning algorithms for real-time applications in a storage- and energy-efficient manner.Our reinforcement learning algorithms, analyses, and experiments will shed new insights about how to make optimal trade-offs between performance and different system constraints such as memory, power, and latency. The research will be guided by systems applications including switch scheduling for network routers, resource management for data centers, network congestion control, memory management for computers, and adaptive sensing for wireless networks and the Internet of things. The outcomes of this project have the potential to transform how data centers, communication networks, and wireless systems are managed. Applications beyond computer and network systems include operational challenges in large-scale systems, such as inventory management and resource allocation, or low latency control problems that arise in brain-machine interfaces. The project also includes an extensive outreach plan that involves women and underrepresented minorities in computing research in reinforcement learning from an algorithmic and architecture-level perspective.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

强化学习已经成为在线决策和控制的一种有前途的算法范例。最近的努力展示了令人印象深刻的结果，但计算资源、数据和训练相对有限。然而，许多现实世界的决策和控制问题需要实时决策，但在吞吐量，延迟，内存和功率方面受到严重限制。现有的算法都是在通用处理器上的软件中使用的，并且不能很好地转化为硬件中的实时资源受限计算。该项目将开发新的算法范例以及同步工作的硬件加速原语，以在嵌入式应用中实现大规模的实时强化学习。从算法的角度来看，这项工作将研究计算资源权衡的效率，并设计强化学习算法，有效地适应给定的硬件约束和硬件原语。从硬件的角度来看，这项工作将设计新的硬件原语，以存储和节能的方式支持开发的强化学习算法用于实时应用。我们的强化学习算法，分析和实验将揭示如何在性能和不同的系统约束（如内存，功耗和延迟）之间进行最佳权衡的新见解。该研究将以系统应用为指导，包括网络路由器的交换机调度，数据中心的资源管理，网络拥塞控制，计算机的内存管理以及无线网络和物联网的自适应传感。该项目的成果有可能改变数据中心、通信网络和无线系统的管理方式。计算机和网络系统之外的应用包括大规模系统中的操作挑战，例如库存管理和资源分配，或脑机接口中出现的低延迟控制问题。该项目还包括一个广泛的推广计划，从算法和架构层面的角度让女性和代表性不足的少数民族参与强化学习的计算研究。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Sequential Fair Allocation: Achieving the Optimal Envy-Efficiency Trade-off Curve

• DOI: 10.1287/opre.2022.2397
• 发表时间: 2023-11-23
• 期刊: OPERATIONS RESEARCH
• 影响因子: 2.7
• 作者: Sinclair，Sean R.;Jain，Gauri;Yu，Christina Lee
• 通讯作者: Yu，Christina Lee

CSI-Based Multi-Antenna and Multi-Point Indoor Positioning Using Probability Fusion

• DOI: 10.1109/twc.2021.3109789
• 发表时间: 2020-09
• 期刊: IEEE Transactions on Wireless Communications
• 影响因子: 10.4
• 作者: Emre Gonultacs;E. Lei;Jack Langerman;Howard Huang;Christoph Studer

Nonasymptotic Analysis of Monte Carlo Tree Search

蒙特卡罗树搜索的非渐近分析

• DOI: 10.1287/opre.2021.2239
• 发表时间: 2022
• 期刊: Operations Research
• 影响因子: 2.7
• 作者: Shah, Devavrat;Xie, Qiaomin;Xu, Zhi
• 通讯作者: Xu, Zhi

Nonparametric Matrix Estimation with One-Sided Covariates

• DOI: 10.1109/isit50566.2022.9834608
• 发表时间: 2021-10
• 期刊: 2022 IEEE International Symposium on Information Theory (ISIT)
• 作者: C. Yu

ORSuite: Benchmarking Suite for Sequential Operations Models

ORSuite：顺序操作模型的基准测试套件

• DOI: 10.1145/3512798.3512819
• 发表时间: 2022
• 期刊: ACM SIGMETRICS Performance Evaluation Review
• 作者: Archer, Christopher;Banerjee, Siddhartha;Cortez, Mayleen;Rucker, Carrie;Sinclair, Sean R.;Solberg, Max;Xie, Qiaomin;Lee Yu, Christina
• 通讯作者: Lee Yu, Christina