CAREER: Intelligent Manipulation in the Real World via Modularity and Abstraction

这个教师早期职业发展（Career）项目旨在使智能机器人能够在现实世界的任务中观察、思考和行动。近年来，机器人技术在深度学习方面取得了长足的进步。然而，最先进的操作算法仍然缺乏广泛部署的泛化和鲁棒性。这对从工业自动化到通用家用机器人的实际应用构成了主要障碍。这项研究将加速熟练和可靠的机器人从研究设置到现实世界环境的部署。它将为实际应用中的变革性机器人技术做出根本性贡献，包括制造业、医疗保健和家庭辅助。作为该项目的一部分，教育计划将通过课程开发和研究指导将这项研究纳入本科和研究生教育。拓展活动将包括开源软件倡议和面向高中学生的K-12教育项目。该项目的总体目标是为现实世界中的智能机器人操作构建新的算法和工具。这项研究的关键是一套算法，它可以构建有效的抽象，并利用这些抽象来综合复杂的行为。具体来说，该研究有三个密切相关的研究主题：1)用对象中心表征发展感知抽象，2)用感觉运动技能建立运动抽象，以及3)用这些抽象建模复杂的操作任务。算法将在模块和系统级别进行严格评估，并部署在物理机器人硬件上，以执行一系列家庭任务。该项目由跨部门机器人基础研究项目支持，由工程（ENG）和计算机与信息科学与工程（CISE）联合管理和资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) project seeks to enable intelligent robots to see, think, and act in real-world tasks. Recent years have witnessed great strides in deep learning for robotics. Yet, state-of-the-art manipulation algorithms still fall short of generalization and robustness for widespread deployment. It poses a major obstacle to real-world applications from industrial automation to general-purpose household robots. This research will accelerate the deployment of adept and reliable robots from research settings to real-world environments. It will make fundamental contributions to transformative robotics technologies in practical applications, including manufacturing, healthcare, and home assistance. As part of the project, the education plan will integrate this research into undergraduate and graduate education through course development and research mentorship. The outreach activities will include an open-source software initiative and K-12 educational programs for high school students. The overall objective of this project is to build new algorithms and tools for intelligent robot manipulation in the real world. The crux of this research is a suite of algorithms that build effective abstractions and harness these abstractions to synthesize sophisticated behaviors. Concretely, the research has three intimately connected research themes: 1) developing perceptual abstraction with object-centric representations, 2) building motor abstraction with sensorimotor skills, and 3) modeling complex manipulation tasks with these abstractions. The algorithms will be rigorously evaluated at the module and system levels and deployed on the physical robot hardware to perform a spectrum of household tasks.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.