权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

EAGER: Real-Time: Reinforcement, Meta, and Episodic Learning for Control under Uncertainty

EAGER：实时：不确定性下控制的强化、元和情景学习

基本信息

批准号：
1839429
负责人：
Pramod Khargonekar
金额：
$ 29.93万
依托单位：
University of California-Irvine
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-10-01 至 2021-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1839429&HistoricalAwards=false
关键词：
EAGER Real Time Reinforcement Meta

项目摘要

Machine learning and artificial intelligence are among the most important general purpose technologies for the coming decades, with potential to transform all aspects of society from health, to manufacturing, to business, to education, and to security. In the last decade, there have been very important and impressive advances in machine learning driven by the use of deep neural networks, innovative training algorithms, computational resources including specialized hardware (graphics processors, tensor processing units), and large datasets. Some of these developments have connections to emerging understanding from neuroscience on how the human brain learns to makes decisions in real-time. However, there are major challenges in the use of these techniques in real-time control and decision making for engineering systems where stability, reliability, and safety are paramount concerns. This project aims to connect major advances in machine learning and neuroscience to control systems and thereby advance myriad application domains. Modern engineered systems are increasingly complicated. They comprise large heterogeneous distributed networks of (IoT) connected devices, systems, and human/social agents, e.g., transportation, energy, water, manufacturing, health and agriculture. A major challenge is performance, stability and reliability of these systems under large uncertainties. The goal is to expand our understanding and integration of learning and control to derive principles and algorithms for the development of learning-based control systems for a variety of engineering applications. While there are significant historical connections between reinforcement learning and stochastic dynamic control, the potential for leveraging ongoing and future advances in machine learning for control remains significantly under- explored. The field of control systems has deep and solid theoretical and mathematical foundations with comprehensive and well-established frameworks for linear, nonlinear, robust, adaptive, stochastic, distributed, and model-predictive control systems. Equally importantly, control systems have applications in multiple domains, such as aerospace, automotive, manufacturing, energy, transportation, agriculture, water, and many other engineered and socio-technical systems. Despite this rich spectrum of theoretical foundations and important applications, the domain of applicability of traditional control techniques is limited to situations where good mathematical models of the underlying systems are available, and where the environmental uncertainty is not too large. This exploratory research project is aimed at overcoming these limitations via novel problem formulations in systems and control inspired by new insights coming from recent developments in machine learning. A key focus will be on novel control architectures inspired by neuroscience and reinforcement learning. Besides architectural innovations, the project will explore questions of stability, performance, and uncertainty by integrating ideas from rapid (one-shot) learning, meta-learning, and episodic control into control algorithms. The ideas from this project will be at the core of a new graduate level course in learning for control which will be taught at the University of California, Irvine. The resulting course materials will be made available to the research community and will benefit interested graduate students across the nation. In addition, short courses will be offered at major professional conferences, e. g., American Control Conference, IEEE Conference on Decision and Control.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习和人工智能是未来几十年最重要的通用技术之一，有可能改变社会的各个方面，从健康到制造业，到商业，到教育和安全。在过去的十年中，机器学习取得了非常重要和令人印象深刻的进步，这是由深度神经网络、创新训练算法、包括专用硬件（图形处理器、张量处理单元）在内的计算资源和大型数据集的使用驱动的。其中一些发展与神经科学对人类大脑如何学习实时决策的新兴理解有关。然而，在工程系统的稳定性，可靠性和安全性是最重要的问题，在实时控制和决策中使用这些技术存在重大挑战。该项目旨在将机器学习和神经科学的重大进展与控制系统联系起来，从而推动无数应用领域的发展。现代工程系统越来越复杂。它们包括（IoT）连接设备、系统和人类/社会代理的大型异构分布式网络，例如，交通、能源、水、制造业、卫生和农业。一个主要的挑战是这些系统在大的不确定性下的性能，稳定性和可靠性。我们的目标是扩大我们对学习和控制的理解和整合，以获得各种工程应用的基于学习的控制系统开发的原理和算法。虽然强化学习和随机动态控制之间存在着重要的历史联系，但利用机器学习进行控制的持续和未来进展的潜力仍显着不足。控制系统领域有着深厚而坚实的理论和数学基础，为线性、非线性、鲁棒、自适应、随机、分布式和模型预测控制系统提供了全面而完善的框架。同样重要的是，控制系统在多个领域中有应用，例如航空航天、汽车、制造、能源、运输、农业、水以及许多其他工程和社会技术系统。尽管有丰富的理论基础和重要的应用，传统控制技术的适用范围仅限于底层系统的良好数学模型可用的情况下，以及环境的不确定性不是太大。这个探索性的研究项目旨在通过系统和控制中的新问题公式来克服这些限制，这些问题公式受到机器学习最近发展的新见解的启发。一个关键的重点将是受神经科学和强化学习启发的新型控制架构。除了架构创新之外，该项目还将通过将快速（一次性）学习，元学习和情景控制的想法整合到控制算法中来探索稳定性，性能和不确定性问题。从这个项目的想法将在一个新的研究生水平的课程，在学习控制，将在加州大学欧文分校教授的核心。由此产生的课程材料将提供给研究界，并将有利于全国各地感兴趣的研究生。此外，短期课程将在主要的专业会议上提供，如。例如，在一个实施例中，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。