CPS: Breakthrough: Reinforcement Learning Algorithms for Cyber-Physical Systems

该项目研究新的强化学习算法，以实现网络物理系统（CPS）的长期实时自主学习。CPS的复杂性使得手动编程安全有效的控制器变得困难。为了让CPS发挥他们的潜力，他们需要一些方法，使他们能够学习和适应他们没有被编程的新情况。强化学习（RL）是一种学习序列决策过程的范例，具有解决这一问题的潜力。然而，现有的强化学习算法并不能满足CPS学习的所有要求。 在智能建筑和自动驾驶汽车的背景下评估了CPS新算法的有效性。通过启用智能建筑、交通、医疗技术和电网，网络物理系统（CPS）有可能彻底改变社会。该项目的成功可能会导致新一代的CPS能够适应他们的情况，并随着时间的推移自主提高他们的性能。除了传统的传播方法外，该项目还将开发和发布实现新强化学习算法的开源代码。 与该项目相关的教育和推广活动包括新生研究计划课程，参加UT奥斯汀年度开放日，吸引了许多代表性不足的少数民族，以吸引公众对计算机科学和科学的兴趣，以及该部门每年的暑期学校高中女生称为第一次。

This project investigates new reinforcement learning algorithms to enable long-term real-time autonomous learning by cyber-physical systems (CPS). The complexity of CPS makes hand-programming safe and efficient controllers for them difficult. For CPS to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes with potential for solving this problem. However, existing RL algorithms do not meet all of the requirements of learning in CPS. Efficacy of the new algorithms for CPS is evaluated in the context of smart buildings and autonomous vehicles.Cyber-physical systems (CPS) have the potential to revolutionize society by enabling smart buildings, transportation, medical technology, and electric grids. Success of this project could lead to a new generation of CPS that are capable of adapting to their situation and improving their performance autonomously over time. In addition to the traditional methods of dissemination, this project will develop and release open-source code implementing the new reinforcement learning algorithms. Education and outreach activities associated with the project include a Freshman Research Initiative course, participation in a UT Austin annual open house that draws in many underrepresented minorities to interest the public in computer science and science in general, and the department's annual summer school for high school girls called First Bytes.