NRI: FND: A Formal Methods Approach to Safe, Composable, and Distributed Reinforcement Learning for co-Robots

许多应用程序需要不同类型的机器人团队相互协作，并与人类协作来完成复杂的任务。例如，考虑一个未来的机器人餐厅，其目标是制作热狗，并将它们与饮料一起提供给即将到来的顾客。几个机器人操纵者有传感器和执行器，允许他们操纵和烤热狗，把它们放在面包里，并添加香料。另一个机器人有一个抓手，可以拿杯子和倒饮料。移动轮式机器人可以在餐厅周围移动，向顾客打招呼，然后为他们提供热狗和饮料。一名人类主管向机器人团队提供高级任务规范，以及一些有用的事实，然后观察团队的工作。如果出现问题，或者机器人无法有效协调，主管可以进行干预并给出更多指示。许多其他应用领域都有类似的场景，包括农业、军事监视、搜索和救援。本项目提出了一种解决此类问题的方法，该方法利用机器人的操纵和合作能力，允许丰富的任务规范和与人类的交互，同时确保整体操作的安全性。该研究计划与教育和推广计划相结合，该计划包括为本科生和高中生提供丰富的机器人相关活动。提出的技术方法汇集了机器（强化）学习、形式化方法和最优控制的工具。将开发一种丰富的、易于理解的时态逻辑规范语言来形式化需求，例如上面的例子中的需求，并指定先验知识。计算框架的核心是一个度量规范满意度的度量，它将用于指导学习过程。这一指标将与控制屏障功能相结合，以确保安全。提出的方法是组合的——新任务的策略将从一个学习过的策略库中构建，几乎没有额外的探索。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Many applications require heterogeneous teams of robots to collaborate with each other and with humans to accomplish complex tasks. Consider, for example, a futuristic robotic restaurant, in which the goal is to make hotdogs and serve them together with drinks to incoming customers. A couple of robotic manipulators have sensors and actuators allowing them to manipulate and grill the hotdogs, put them in buns, and add spices. Another robot has a gripper that allows it to handle glasses and pour drinks. Mobile wheeled robots can move around the restaurant, greet the customers, and then serve them hotdogs and drinks. A human supervisor gives the robotic team high level task specifications, together with some useful facts, and then watches the team working. If something goes wrong, or the robots do not manage to coordinate efficiently, the supervisor can intervene and give more instructions. Many other application areas share similar scenarios, including agriculture, military surveillance, search and rescue. This project proposes an approach to solve such problems that exploits the robots’ manipulation and cooperation capabilities, allows for rich task specifications and interactions with humans, while at the same time ensuring the safety of the overall operation. The research plan is integrated with an education and outreach plan that includes a rich spectrum of robotic-related activities for undergraduate and high-school students.The proposed technical approach brings together tools from machine (reinforcement) learning, formal methods, and optimal control. A rich, easy-to-understand, temporal logic specification language will be developed to formalize requirements such as the ones from the example above, and to specify prior knowledge. Central to the computational framework is a metric that measures the satisfaction of the specifications, which will be used to guide the learning process. This metric will be combined with control barrier functions to ensure safety. The proposed approach is compositional - policies for new tasks will be constructed from a library of learned policies with little to no additional exploration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.