CAREER: A Multiagent Teacher/Student Framework for Sequential Decision Making Tasks

物理（机器人）代理和虚拟（软件）代理在工业、教育和家庭环境中变得越来越普遍。尽管最近的研究进展使智能体能够在没有人类干预的情况下学习如何完成任务，但对于人类如何最好地教导智能体或智能体如何教导其他智能体，甚至智能体如何教导人类，人们知之甚少。考虑到智能体/人类学习的完整矩阵，其中智能体或人类都可以扮演老师或学生的角色，将增加利用人类和智能体的专业知识和知识的潜在收益。该项目旨在研究顺序决策问题背景下的智能体/人类学习，这是对现实世界智能体系统至关重要的一类问题。该项目旨在开发一种新的教师/学生框架，将自主学习与另一个代理或人类的教学相结合。该项目计划开发和评估一套核心算法，以实现：(1)智能体教导智能体，从而实现智能体之间的鲁棒知识共享；(2)人类教导智能体，从而允许人类与智能体共享或转移常识或特定领域的知识；(3)智能体教导人类，从而帮助人类更好地理解如何执行或重塑自主智能体已经理解或执行的顺序决策任务。在所有情况下，目标都是开发相对于没有老师指导的学习显著提高学习成绩的方法。需要探讨的问题包括教师/学生能力的不匹配、向多名教师学习以及教师/学生之间共享的知识表示。PI计划关注几个场景，每个场景都有关于学生或老师的知识或技能以及他们之间可能的互动类型的不同假设（例如，老师是否能告诉学生该采取什么行动）。该项目中开发的技术将在各种测试领域中进行评估，并将涉及模拟和实际机器人。教师/学生框架将使智能体能够教授其他智能体和人类，并将自主学习与智能体和人类教学相结合。理解如何最好地教授代理对于开发可部署代理系统至关重要。这种与平台和领域无关的方法结合了多智能体系统、机器学习、人机交互和人机交互社区的思想，并有可能影响这些领域。这项工作朝着将代理从仅供专家使用的专门系统转变为对没有编程专业知识的人有用的工具和团队成员迈出了一步。这个项目有很强的教育成分。PI在本科学院任教，本科生将在整个项目中发挥关键作用。此外，该项目的研究成果将被纳入PI的五门课程，为吸引和留住计算机科学专业的学生提供令人兴奋的新材料。PI还将继续通过拉斐特学院的S-STEM和更高成就项目向中学生和代表性不足的群体伸出援助之手。

Physical (robotic) agents and virtual (software) agents are becoming increasingly common in industry, education, and domestic environments. Although recent research advances have enabled agents to learn how to complete tasks without human intervention, little is known about how best to have humans teach agents or agents teach other agents or even how agents might teach humans. Considering the full matrix of agent/human learning, in which either an agent or a human can play the role of teacher or student, would increase the potential benefits of leveraging human and agent expertise and knowledge. This project aims to study agent/human learning in the context of sequential decision-making problems, a class of central importance for real-world agent systems. This project aims to develop a novel teacher/student framework that integrates autonomous learning with teaching by another agent or a human. The project plans to develop and evaluate a set of core algorithms to allow: (1) agents to teach agents, thus enabling robust knowledge sharing among agents; (2) humans to teach agents, thus allowing humans to share or transfer common sense or domain-specific knowledge with agents; and (3) agents to teach humans, thus helping humans better understand how to perform or recast sequential decision-making tasks already understood or performed by autonomous agents. In all cases, the goal is to develop methods that significantly improve learning performance relative to learning without guidance from a teacher. Issues to be explored include mismatch between teacher/student abilities, learning from multiple teachers, and shared knowledge representation between teacher/student. The PI plans to focus on several scenarios, each with different sets of assumptions about the knowledge or skill of the student or teacher and the kind of interaction possible between them (e.g., whether the teacher can tell the student what action to take). The techniques developed in the project will be evaluated in a variety of tests domains and will involve simulations as well as actual robots.The teacher/student framework will enable agents to teach other agents and humans, as well as integrate autonomous learning with agent and human teaching. Understanding how to best teach agents is of key importance in developing deployable agent systems. The platform- and domain-independent approach incorporates ideas from multiagent systems, machine learning, human-computer interaction, and human-robot interaction communities, and has the potential to impact each of these areas. This work takes a step towards transitioning agents from specialized systems usable only by experts into useful tools and teammates for people without programming expertise. This project has a strong educational component. The PI teaches at an undergraduate college and undergraduate students will play a crucial role throughout the project. Furthermore, the research produced by this project will be incorporated into five of the PI's courses, providing exciting new material to attract and retain computer science majors. The PI will also continue outreach to secondary school students as well as to underrepresented groups via Lafayette College's S-STEM and Higher Achievement programs.