CAREER: Specification-Guided Imitation Learning

模仿学习(IL)是一种强大的学习范式，使机器人或人工智能(AI)系统等机器能够从人类专家或专家代理提供的演示中学习。然而，在实践中，人工演示可能是不充分的、部分的、不完美的、特定于环境的或次优的。为了应对这些挑战，该项目引入了一个新的框架，允许正式规范来指导IL的数据驱动学习过程。该项目的新颖性是将不同形式的形式规范与模仿学习中的数据相结合的新理论和算法。该项目的影响是多方面的，包括克服了当前IL方法对数据的过度依赖，提高了它们的性能和健壮性，并在机器学习和正式方法的交叉点上催化了新的跨学科研究。在强有力的机构支持下，该项目正在为来自弱势背景的K-12学生提供外展机会，扩大妇女和少数族裔对STEM研究的参与，并帮助激励年轻一代攻读大学学位或未来的工程职业。该项目背后的核心理念是，专家的意见不必仅限于演示。例如，对于专家来说，指定学习到的策略必须满足的安全属性可能会更自然、更经济，而不是展示系统可能失败的许多不同方式(想象一下，仅仅为了生成这些“负面例子”，就将一辆昂贵的汽车撞数百次)。除了提供补充信息外，规范还可以为学习过程提供结构，并改善整体学习结果。例如，基于自动机的奖励比马尔可夫奖励更适合于对时间延长的任务进行建模。该项目正在探索统一学习框架中数据和正式规范之间的协同效应，并有可能改变我们未来开发人工智能系统的方式。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Imitation learning (IL) is a powerful learning paradigm that enables machines, such as robots or artificial intelligence (AI) systems, to learn from demonstrations provided by human experts or expert agents. However, in practice, human demonstrations can be inadequate, partial, imperfect, environment-specific, or suboptimal. To address these challenges, this project introduces a novel framework that allows formal specifications to guide the data-driven learning process of IL. The project’s novelties are new theories and algorithms for combining formal specifications of different forms with data in imitation learning. The project's impacts are manifold, including overcoming the over-reliance on data in current IL approaches, improving their performance and robustness, and catalyzing new cross-disciplinary research at the intersection of machine learning and formal methods. With strong institutional support, the project is providing outreach opportunities to K-12 students from disadvantaged backgrounds, broadening the participation of women and minorities in STEM research, and helping inspire younger generations to pursue a college degree or a future career in engineering.The central idea behind this project is that expert inputs need not be limited to demonstrations. For instance, it can be more natural and economical for an expert to specify safety properties that the learned policy must satisfy, instead of showing the many different ways that the system can fail (imagine crashing an expensive vehicle hundreds of times just to generate these "negative examples"). In addition to offering complementary information, specifications can also provide structures to the learning process and improve the overall learning outcomes. For example, automata-based rewards are much more suitable for modeling temporally extended tasks than Markovian rewards. This project is exploring the synergies between data and formal specifications in a unified learning framework, and has the potential to transform how we develop AI systems in the future.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.