CPS: Breakthrough: Reinforcement Learning Algorithms for Cyber-Physical Systems

本项目研究新的强化学习算法，以实现计算机物理系统(CPS)的长期实时自主学习。CPS的复杂性使得手动编程安全高效的控制器变得困难。对于CP来说，要发挥他们的潜力，他们需要方法使他们能够学习和适应他们不是为之编程的新情况。强化学习(RL)是一种学习序列决策过程的范例，具有解决这一问题的潜力。然而，现有的RL算法并不能满足CPS中的所有学习要求。CPS的新算法的有效性在智能建筑和自动驾驶汽车的背景下进行了评估。网络物理系统(CPS)通过启用智能建筑、交通、医疗技术和电网，具有变革社会的潜力。这个项目的成功可能会导致新一代CP能够适应他们的情况，并随着时间的推移自主提高他们的表现。除了传统的传播方法外，该项目还将开发和发布实现新的强化学习算法的开源代码。与该项目相关的教育和外联活动包括一门新生研究计划课程，参加德克萨斯大学奥斯汀分校的年度开放日，吸引了许多未被充分代表的少数族裔，以引起公众对计算机科学和一般科学的兴趣，以及该部门为高中女孩举办的名为First Bytes的年度暑期学校。

This project investigates new reinforcement learning algorithms to enable long-term real-time autonomous learning by cyber-physical systems (CPS). The complexity of CPS makes hand-programming safe and efficient controllers for them difficult. For CPS to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes with potential for solving this problem. However, existing RL algorithms do not meet all of the requirements of learning in CPS. Efficacy of the new algorithms for CPS is evaluated in the context of smart buildings and autonomous vehicles.Cyber-physical systems (CPS) have the potential to revolutionize society by enabling smart buildings, transportation, medical technology, and electric grids. Success of this project could lead to a new generation of CPS that are capable of adapting to their situation and improving their performance autonomously over time. In addition to the traditional methods of dissemination, this project will develop and release open-source code implementing the new reinforcement learning algorithms. Education and outreach activities associated with the project include a Freshman Research Initiative course, participation in a UT Austin annual open house that draws in many underrepresented minorities to interest the public in computer science and science in general, and the department's annual summer school for high school girls called First Bytes.