RI: Small: High Confidence, Efficient Learning Under Rich Task Specifications

自动驾驶汽车和个人机器人等人工智能机器有望在许多经济领域做出贡献，但如果没有对它们能够正确运行的可衡量的信心，就无法大规模部署。对于安全关键系统来说尤其如此，在这些系统中，不正确的行为可能会导致人员受伤或基础设施损坏。拟议的研究将通过开发允许智能代理学习执行具有挑战性的任务和适应新情况的方法来解决这一关键问题，同时提供正确性和安全性的强有力保证。一旦部署，这些未来的机器人系统将对社会产生广泛的影响，从自动化小型制造业到通过安全和个性化的家庭护理为残疾人和老年人提供新的自由。提议的机器人应用程序还将为互动教育K-12课程创造机会，以鼓励对STEM领域的兴趣，以及本科和研究生教育。为了实现这些目标，建议的工作集中在三个主要的研究重点上：1)我们将设计安全的学习算法，为策略的预期奖励及其相对于高级规范的正确性提供理论上的概率满足和数据效率保证。2)为了解决理论和实际学习效率之间的差距，我们将开发基于模型和无模型的非策略评估方法，利用主动抽样策略和自举来实现实际效率。(3)发展结合和放大理论保障和实践保障优势的混合技术。所提出算法的优点将在机器人可重构制造中复杂的现实问题上进行系统评估，这些问题需要学习优化，但在具有低到中等数量可用数据的设置中具有高度置信度的安全策略。

Artificially intelligent machines such as autonomous vehicles and personal robots are poised to contribute in many economic sectors, but cannot be deployed on a large scale without measurable confidence that they will operate correctly. This is especially true for safety-critical systems in which humans could be injured or infrastructure could be damaged by incorrect behavior. The proposed research will address this key issue by developing methods that allow intelligent agents to learn to perform challenging tasks and adapt to new situations, while simultaneously providing strong guarantees of correctness and safety. Once deployed, these future robotic systems will have broad impacts on society ranging from automating small manufacturing to giving new freedom to disabled and elderly populations through safe and personalized in-home care. The proposed robotics applications will additionally create opportunities for interactive educational K-12 programs to encourage interest in STEM areas, as well as undergraduate and graduate education.Towards these goals, the proposed work focuses on three primary research thrusts: 1) We will design safe learning algorithms that provide theoretical probabilistic satisfaction and data efficiency guarantees over both the expected reward of a policy and its correctness with respect to a high-level specification. 2) In order to account for the gap between theoretical and practical efficiency in learning, we will develop model-based and model-free off-policy evaluation methods that leverage active sampling strategies and bootstrapping to achieve practical efficiency. 3) We will develop hybrid techniques that combine and amplify the advantages of both strong theoretical and efficient practical guarantees. The merit of the proposed algorithms will be systematically evaluated on complex, real-world problems in robotic reconfigurable manufacturing that require learning optimized, yet safe policies with a high degree of confidence in settings with low-to-medium quantities of available data.