CAREER: FLEXIBLE HIERARCHICAL ABSTRACTIONS FOR ACTIONABLE VISUAL PERCEPTION

人类擅长根据需要优先考虑和关注环境的不同部分。例如，一个护士喂病人可能会仔细考虑在一个盘子里的食物，同时拿起他们，但后来把他/她的注意力放在病人的嘴和面部表情。相比之下，今天的机器人在观察、处理和与世界互动的方式上是僵化和不灵活的。他们要么以变得迟钝为代价专注于一切，要么不分青红皂白地偷工减料，变得容易失败。该项目将构建创新的软件技术，使通用机器人能够像人类一样灵活地适应任务要求，提高其敏捷性和效率，并使其更容易学习新任务。这将使这种机器人不仅能够应用于本项目将用于开展研究的家庭和医院任务，而且还可以应用于许多其他社会相关的环境，如农场，建筑工地和小规模制造业。两名研究生和几名本科生将直接通过该项目接受研究培训。此外，本项目还将利用其研究成果，改善研究生和本科生课程以及高中生暑期拓展课程。本项目探讨了一个假设，即一个关键的缺失部分是敏捷和有效的感知-行动循环，可以根据任务要求动态进化。考虑一个机器人从饭碗里喂孩子食物。每一口食物的位置和形状，以及孩子的详细姿势和嘴部结构，在整个任务过程和每个进食阶段都是相关的。这种随时间变化的任务需求是普遍存在的，但它们在今天的标准计算机视觉算法和控制回路中被锁定的抽象所占的比例很低。该项目旨在推进视觉识别方法的具体行动的背景下，并开发机器人学习方法，利用这些进步。它重新设计的控制回路是灵活的，根据任务的需求，分层分解，以允许新的成分的因素，和自我学习的可扩展性。为此，它开发了一个阶段性的，但端到端的可微分控制回路结构与交错的任务特定的，可微调的组件和任务通用的，可重用的组件阶段。它进一步提出了新的主动自我学习方法，利用代理的自己的化身来教它。这些进步将能够提高机器人学习方法的样本效率以及任务执行的计算效率。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Humans excel at prioritizing and attending to different parts of the environment as the need arises. For example, a nurse feeding a patient might carefully consider individual morsels of food on a plate while picking them up, but afterwards devote his/her attention to the patient's mouth and facial expressions. By comparison, today's robots are rigid and inflexible in the ways that they observe, process, and interact with the world. They either focus on everything at the cost of becoming sluggish or instead cut corners indiscriminately and become prone to failures. This project will build innovative software technologies to allow general-purpose robots to flexibly adapt to task requirements, much like humans can, improving their agility and efficiency and making it easier for them to learn new tasks. This will enable applications of such robots not only in the household and hospital tasks which this project will use to develop the research, but also in many other socially relevant settings as farms, constructions sites, and small-scale manufacturing. Two graduate students and several undergraduate students will receive research training directly through this project. Further, this project will also draw from its research findings to improve graduate and undergraduate courses and summer outreach courses for high-school students.This project explores the hypothesis that one key missing piece is agile and efficient perception-action loops that can evolve on-the-fly in response to task requirements. Consider a robot feeding a child food from a rice bowl. The locations and shapes of individual morsels of food, and the child's detailed pose and mouth configuration are all relevant over the course of the task and in each of the eating phases. Such time-varying task requirements are ubiquitous, yet they are poorly accounted for by the locked-in abstractions in today's standard computer vision algorithms and control loops. This project aims to advance visual recognition approaches in the context of embodied action, and to develop robot learning approaches in tandem that exploit these advances. It redesigns control loops to be flexible according to task demands, hierarchically factorized to permit novel compositions of the factor components, and self-learnable for scalability. To this end, it develops a staged but end-to-end differentiable control loop structure with interleaved task-specific, fine-tunable components and task-generic, reusable component stages. It further proposes novel active self-learning approaches that exploit the agent's own embodiment to teach it. These advances will enable improved sample efficiency for robot learning approaches as well as computational efficiency for task execution.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.