CAREER: Specification-Guided Imitation Learning

模仿学习（IL）是一种强大的学习范式，它使机器（如机器人或人工智能（AI）系统）能够从人类专家或专家代理提供的演示中学习。然而，在实践中，人类的示范可能是不充分的，部分的，不完美的，特定于环境的，或次优的。为了应对这些挑战，该项目引入了一个新的框架，允许正式规范来指导IL的数据驱动学习过程。该项目的新颖之处在于将不同形式的正式规范与模仿学习中的数据相结合的新理论和算法。该项目的影响是多方面的，包括克服当前IL方法对数据的过度依赖，提高其性能和鲁棒性，并在机器学习和正式方法的交叉点上催化新的跨学科研究。在强有力的机构支持下，该项目为来自弱势背景的K-12学生提供外展机会，扩大女性和少数族裔对STEM研究的参与，并帮助激励年轻一代攻读大学学位或未来的工程职业生涯。该项目背后的中心思想是专家的投入不需要仅限于演示。例如，对于专家来说，指定学习策略必须满足的安全属性可能更自然和经济，而不是显示系统可能失败的许多不同方式（想象一下，为了生成这些“负面示例”而撞毁昂贵的车辆数百次）。除了提供补充信息外，规格还可以为学习过程提供结构，并改善整体学习成果。例如，基于自动机的奖励比马尔可夫奖励更适合建模时间扩展的任务。该项目是在统一的学习框架中探索数据和正式规范之间的协同作用，并有可能改变我们未来开发人工智能系统的方式。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Imitation learning (IL) is a powerful learning paradigm that enables machines, such as robots or artificial intelligence (AI) systems, to learn from demonstrations provided by human experts or expert agents. However, in practice, human demonstrations can be inadequate, partial, imperfect, environment-specific, or suboptimal. To address these challenges, this project introduces a novel framework that allows formal specifications to guide the data-driven learning process of IL. The project’s novelties are new theories and algorithms for combining formal specifications of different forms with data in imitation learning. The project's impacts are manifold, including overcoming the over-reliance on data in current IL approaches, improving their performance and robustness, and catalyzing new cross-disciplinary research at the intersection of machine learning and formal methods. With strong institutional support, the project is providing outreach opportunities to K-12 students from disadvantaged backgrounds, broadening the participation of women and minorities in STEM research, and helping inspire younger generations to pursue a college degree or a future career in engineering.The central idea behind this project is that expert inputs need not be limited to demonstrations. For instance, it can be more natural and economical for an expert to specify safety properties that the learned policy must satisfy, instead of showing the many different ways that the system can fail (imagine crashing an expensive vehicle hundreds of times just to generate these "negative examples"). In addition to offering complementary information, specifications can also provide structures to the learning process and improve the overall learning outcomes. For example, automata-based rewards are much more suitable for modeling temporally extended tasks than Markovian rewards. This project is exploring the synergies between data and formal specifications in a unified learning framework, and has the potential to transform how we develop AI systems in the future.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.