权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RTML: Small: Real-Time Model-Based Bayesian Reinforcement Learning

RTML：小型：基于实时模型的贝叶斯强化学习

基本信息

批准号：
1937396
负责人：
Jiang Hu
金额：
$ 50万
依托单位：
Texas A&M Engineering Experiment Station
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2023-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1937396&HistoricalAwards=false
关键词：
RTML Small Real Time Model

项目摘要

The applicability of many machine-learning techniques is limited to very specific, fixed application environments. As opposed to this, model-based Bayesian reinforcement learning technique is a more sophisticated framework, enabling human-like decision-making with evolving objectives and without complete knowledge of the environment. However, its computational complexity is very high, and consequently, its progress has been limited to small-scale demonstrations on limited applications. The goal of this project is to overcome this hurdle by developing innovative algorithm-hardware co-design techniques. The research outcome is expected to greatly accelerate the computation of model-based Bayesian reinforcement learning for practical, large-scale real-life applications, especially those with real-time constraints. The research results will benefit many fields that directly impact society, e.g., driver-less cars, unmanned aerial vehicles, smart agricultural irrigation, robotics for disaster relief and robotic assistants for handicapped people. The research will also train students, including women and other underrepresented groups, for the much needed U.S. workforce in related areas of technology.The computational kernel of model-based Bayesian reinforcement learning is random sampling over a decision tree. In this project, the computational acceleration will be realized by making use of the intrinsic parallelism offered by the sampling process. Both the memory and the arithmetic bottlenecks of traditional approaches will be addressed. First, a logic circuit based technique will be developed to represent probabilities, and thereby greatly reduce the memory utilization of the algorithm. Second, powerful new arithmetic techniques will be explored to achieve area-efficient computations for the newly proposed number representation. Third, a new sampling method, that is friendly to circuit implementation, will be investigated. Fourth, an approximation technique will be studied to alleviate the complexity arising from tracing sampling histories. Finally, path-aware parallel sampling will be exploited to avoid the redundant computations in software implementations. These techniques and their overall effectiveness will be validated via experiments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

许多机器学习技术的适用性仅限于非常具体、固定的应用环境。与此相反，基于模型的贝叶斯强化学习技术是一个更复杂的框架，可以在不完全了解环境的情况下实现具有不断变化的目标的类人决策。然而，其计算复杂度非常高，因此其进展仅限于有限应用的小规模演示。该项目的目标是通过开发创新的算法-硬件协同设计技术来克服这一障碍。该研究成果预计将大大加速基于模型的贝叶斯强化学习的计算，适用于实际的大规模现实生活应用，特别是那些具有实时约束的应用。研究成果将惠及无人驾驶汽车、无人机、智能农业灌溉、救灾机器人、残疾人机器人助手等诸多直接影响社会的领域。该研究还将培训学生，包括女性和其他代表性不足的群体，为相关技术领域急需的美国劳动力提供培训。基于模型的贝叶斯强化学习的计算核心是对决策树进行随机采样。在该项目中，将通过利用采样过程提供的固有并行性来实现计算加速。传统方法的内存和算术瓶颈都将得到解决。首先，将开发基于逻辑电路的技术来表示概率，从而大大减少算法的内存利用率。其次，将探索强大的新算术技术，以实现新提出的数字表示的区域高效计算。第三，将研究一种有利于电路实现的新采样方法。第四，将研究近似技术以减轻跟踪采样历史所产生的复杂性。最后，将利用路径感知并行采样来避免软件实现中的冗余计算。这些技术及其整体有效性将通过实验进行验证。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Scaled Population Division for Approximate Computing

用于近似计算的规模化总体划分

DOI：
发表时间：
2023
期刊：
Proceedings International Symposium on Low Power Electronics and Design
影响因子：
0
作者：
Kunal Bharathi
通讯作者：
Kunal Bharathi

Scaled Population Arithmetic for Efficient Stochastic Computing

DOI：
10.1109/asp-dac47756.2020.9045292
发表时间：
2020-01
期刊：
2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)
影响因子：
0
作者：
He Zhou;S. Khatri;Jiang Hu;Frank Liu
通讯作者：
He Zhou;S. Khatri;Jiang Hu;Frank Liu

Scaled Population Subtraction for Approximate Computing

DOI：
10.1109/iccd50377.2020.00065
发表时间：
2020-10
期刊：
2020 IEEE 38th International Conference on Computer Design (ICCD)
影响因子：
0
作者：
K. Bharathi;Jiang Hu;S. Khatri
通讯作者：
K. Bharathi;Jiang Hu;S. Khatri

TD3lite: FPGA Acceleration of Reinforcement Learning with Structural and Representation Optimizations

DOI：
10.1109/fpl57034.2022.00023
发表时间：
2022-08
期刊：
2022 32nd International Conference on Field-Programmable Logic and Applications (FPL)
影响因子：
0
作者：
Chan-Wei Hu;Jiangkun Hu;S. Khatri
通讯作者：
Chan-Wei Hu;Jiangkun Hu;S. Khatri