CAREER: FLEXIBLE HIERARCHICAL ABSTRACTIONS FOR ACTIONABLE VISUAL PERCEPTION

当需要出现时，人类擅长对环境的不同部分进行优先排序和照顾。例如，给病人喂奶的护士可能会在拿起盘子里的食物时仔细考虑每一小块食物，但之后会将注意力集中在病人的嘴巴和面部表情上。相比之下，今天的机器人在观察、处理和与世界互动的方式上是僵化和僵化的。他们要么以变得迟缓为代价专注于一切，要么不分青红皂白地偷工减料，变得容易失败。该项目将构建创新的软件技术，使通用机器人能够像人类一样灵活地适应任务要求，提高它们的敏捷性和效率，并使它们更容易学习新任务。这将使这种机器人不仅应用于本项目将用于开展研究的家庭和医院任务中，还将应用于许多其他与社会相关的环境中，如农场、建筑工地和小规模制造。两名研究生和几名本科生将通过该项目直接接受研究性培训。此外，该项目还将利用其研究成果来改进研究生和本科课程以及针对高中生的暑期外展课程。该项目探索了一种假设，即一个关键的缺失部分是敏捷和高效的感知-行动循环，它可以根据任务要求在运行中进化。设想一个机器人用饭碗喂孩子吃东西。每一小块食物的位置和形状，以及孩子详细的姿势和嘴型，在任务过程中和每个进食阶段都是相关的。这种时变的任务需求是无处不在的，但它们很难被当今标准计算机视觉算法和控制循环中锁定的抽象所解释。这个项目的目的是在具体行动的背景下推进视觉识别方法，并开发利用这些进展的机器人学习方法。它重新设计了控制循环，使其根据任务需求灵活，分层分解以允许因子组件的新颖组合，并可自学习以实现可伸缩性。为此，它开发了一种阶段性但端到端可区分的控制环路结构，其中包括交错的特定于任务的、可调的组件和通用的、可重用的组件级。它进一步提出了新颖的主动自我学习方法，利用代理自己的体现来教授它。这些进展将提高机器人学习方法的样本效率以及任务执行的计算效率。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Humans excel at prioritizing and attending to different parts of the environment as the need arises. For example, a nurse feeding a patient might carefully consider individual morsels of food on a plate while picking them up, but afterwards devote his/her attention to the patient's mouth and facial expressions. By comparison, today's robots are rigid and inflexible in the ways that they observe, process, and interact with the world. They either focus on everything at the cost of becoming sluggish or instead cut corners indiscriminately and become prone to failures. This project will build innovative software technologies to allow general-purpose robots to flexibly adapt to task requirements, much like humans can, improving their agility and efficiency and making it easier for them to learn new tasks. This will enable applications of such robots not only in the household and hospital tasks which this project will use to develop the research, but also in many other socially relevant settings as farms, constructions sites, and small-scale manufacturing. Two graduate students and several undergraduate students will receive research training directly through this project. Further, this project will also draw from its research findings to improve graduate and undergraduate courses and summer outreach courses for high-school students.This project explores the hypothesis that one key missing piece is agile and efficient perception-action loops that can evolve on-the-fly in response to task requirements. Consider a robot feeding a child food from a rice bowl. The locations and shapes of individual morsels of food, and the child's detailed pose and mouth configuration are all relevant over the course of the task and in each of the eating phases. Such time-varying task requirements are ubiquitous, yet they are poorly accounted for by the locked-in abstractions in today's standard computer vision algorithms and control loops. This project aims to advance visual recognition approaches in the context of embodied action, and to develop robot learning approaches in tandem that exploit these advances. It redesigns control loops to be flexible according to task demands, hierarchically factorized to permit novel compositions of the factor components, and self-learnable for scalability. To this end, it develops a staged but end-to-end differentiable control loop structure with interleaved task-specific, fine-tunable components and task-generic, reusable component stages. It further proposes novel active self-learning approaches that exploit the agent's own embodiment to teach it. These advances will enable improved sample efficiency for robot learning approaches as well as computational efficiency for task execution.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.