CAREER: Active Scene Understanding By and For Robot Manipulation
职业:机器人操作的活动场景理解
基本信息
- 批准号:2348698
- 负责人:
- 金额:$ 60万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2023
- 资助国家:美国
- 起止时间:2023-10-01 至 2027-04-30
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
Despite significant progress, most robot perception systems today remain limited to "seeing what they are asked to see" – detecting pre-defined categories of objects by watching static images or videos. In contrast, humans constantly decide "what to see" and "how to see it" using active exploration. This ability is central to problem-solving and adaptability to novel scenarios but remains missing from robots today. To bridge this gap, this Faculty Early Career Development (CAREER) project aims to study a self-improving robot perception system using manipulation skills – referred to as active scene understanding. The framework suggested in this project improves a robot's fundamental capabilities in perception and planning and therefore impacts many application domains such as service robots or field exploration, where robots need to rapidly analyze their environments in order to swiftly react to evolving situations. The research and education plans are integrated through a Cloud-Enabled Robot Learning Platform, which allows students to participate in robotics education and research without the limits of robot and compute hardware accessibility.This project tackles a number of challenges in active scene understanding to achieve a unified and practical framework. The key idea of the approach is to leverage the synergies between a robot's perception and interaction algorithms to create self-supervisory signals. On the one hand, the robot can use its own actions and the corresponding action effects (i.e., visual observation of subsequent states) as ground truth labels for training its visual predictive model. On the other hand, the robot can also use the statistics provided by the perception model (e.g., uncertainty, novelty, and predictability) as a reward signal to improve its manipulation policy. Ultimately, the robot could combine the learned visual predictive model and manipulation policy to facilitate efficient action planning for downstream tasks.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
尽管取得了重大进展,但今天大多数机器人感知系统仍然局限于“看到他们被要求看到的东西”-通过观看静态图像或视频来检测预定义的对象类别。相比之下,人类通过积极的探索,不断地决定“看什么”和“如何看”。这种能力是解决问题和适应新场景的核心,但今天的机器人仍然缺乏这种能力。为了弥合这一差距,这个教师早期职业发展(CAREER)项目旨在研究一个自我改进的机器人感知系统,使用操纵技能-被称为主动场景理解。该项目中建议的框架提高了机器人在感知和规划方面的基本能力,因此影响了许多应用领域,如服务机器人或现场勘探,机器人需要快速分析其环境,以便迅速对不断变化的情况做出反应。研究和教育计划通过云支持的机器人学习平台进行整合,该平台允许学生参与机器人教育和研究,而不受机器人和计算机硬件的限制。该项目解决了主动场景理解方面的许多挑战,以实现统一和实用的框架。该方法的关键思想是利用机器人的感知和交互算法之间的协同作用来创建自我监督信号。一方面,机器人可以使用自己的动作和相应的动作效果(即,后续状态的视觉观察)作为用于训练其视觉预测模型的基础事实标签。另一方面,机器人还可以使用由感知模型提供的统计数据(例如,不确定性,新奇和可预测性)作为改善其操纵政策的奖励信号。最终,机器人可以联合收割机结合学习的视觉预测模型和操作策略,以促进下游任务的有效行动规划。由工程局(ENG)和计算机与信息科学与工程局(CISE)共同管理和资助该奖项反映了NSF的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(3)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
RoboNinja: Learning an Adaptive Cutting Policy for Multi-Material Objects
- DOI:10.48550/arxiv.2302.11553
- 发表时间:2023-02
- 期刊:
- 影响因子:0
- 作者:Zhenjia Xu;Zhou Xian;Xingyu Lin;Cheng Chi;Zhiao Huang;Chuang Gan;Shuran Song
- 通讯作者:Zhenjia Xu;Zhou Xian;Xingyu Lin;Cheng Chi;Zhiao Huang;Chuang Gan;Shuran Song
Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
- DOI:10.48550/arxiv.2307.14535
- 发表时间:2023-07
- 期刊:
- 影响因子:0
- 作者:Huy Ha;Peter R. Florence;Shuran Song
- 通讯作者:Huy Ha;Peter R. Florence;Shuran Song
Structure from Action: Learning Interactions for 3D Articulated Object Structure Discovery
- DOI:10.1109/iros55552.2023.10342135
- 发表时间:2022-07
- 期刊:
- 影响因子:0
- 作者:Neil Nie;S. Gadre;Kiana Ehsani;Shuran Song
- 通讯作者:Neil Nie;S. Gadre;Kiana Ehsani;Shuran Song
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Shuran Song其他文献
Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learning
通过不确定性感知强化学习进行人机循环机器人代理的决策
- DOI:
- 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
Siddharth Singi;Zhanpeng He;Alvin Pan;Sandip Patel;Gunnar A. Sigurdsson;Robinson Piramuthu;Shuran Song;M. Ciocarlie - 通讯作者:
M. Ciocarlie
Cloth Funnels: Canonicalized-Alignment for Multi-Purpose Garment Manipulation
布料漏斗:多用途服装操作的规范化对齐
- DOI:
- 发表时间:
2022 - 期刊:
- 影响因子:0
- 作者:
Alper Canberk;Cheng Chi;Huy Ha;B. Burchfiel;Eric A. Cousineau;S. Feng;Shuran Song - 通讯作者:
Shuran Song
Pick2Place: Task-aware 6DoF Grasp Estimation via Object-Centric Perspective Affordance
Pick2Place:通过以对象为中心的视角可供性进行任务感知的 6DoF 抓取估计
- DOI:
- 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
Zhanpeng He;Nikhil Chavan;Jinwook Huh;Shuran Song;Volkan Isler - 通讯作者:
Volkan Isler
Experimental investigation on the spray characteristics of agricultural full-cone pressure swirl nozzle
- DOI:
10.25165/j.ijabe.20231604.7088 - 发表时间:
2023 - 期刊:
- 影响因子:
- 作者:
Xiuyun Xue;Xufeng Xu;Shilei Lyu1 2;Shuran Song;Xin Ai;Nengchao Li;Zhenyu Yang;Zhen Li - 通讯作者:
Zhen Li
3 DMatch : Learning Local Geometric Descriptors from RGB-D Reconstructions APPENDIX
3 DMatch:从 RGB-D 重建中学习局部几何描述符 附录
- DOI:
- 发表时间:
2017 - 期刊:
- 影响因子:0
- 作者:
Andy Zeng;Shuran Song;M. Nießner;Matthew Fisher;Jianxiong Xiao;T. Funkhouser - 通讯作者:
T. Funkhouser
Shuran Song的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Shuran Song', 18)}}的其他基金
NRI: Hierarchical Representation Learning for Robot Assistants
NRI:机器人助手的分层表示学习
- 批准号:
2405103 - 财政年份:2023
- 资助金额:
$ 60万 - 项目类别:
Standard Grant
NRI: Hierarchical Representation Learning for Robot Assistants
NRI:机器人助手的分层表示学习
- 批准号:
2132519 - 财政年份:2022
- 资助金额:
$ 60万 - 项目类别:
Standard Grant
CAREER: Active Scene Understanding By and For Robot Manipulation
职业:机器人操作的活动场景理解
- 批准号:
2143601 - 财政年份:2022
- 资助金额:
$ 60万 - 项目类别:
Continuing Grant
相似国自然基金
光—电驱动下的AIE-active手性高分子CPL液晶器件研究
- 批准号:92156014
- 批准年份:2021
- 资助金额:70.00 万元
- 项目类别:国际(地区)合作与交流项目
光-电驱动下的AIE-active手性高分子CPL液晶器件研究
- 批准号:
- 批准年份:2021
- 资助金额:70 万元
- 项目类别:
相似海外基金
CAREER: Active Scene Understanding By and For Robot Manipulation
职业:机器人操作的活动场景理解
- 批准号:
2143601 - 财政年份:2022
- 资助金额:
$ 60万 - 项目类别:
Continuing Grant
CHS: Small: DeepCrowd: A Crowd-assisted Deep Learning-based Disaster Scene Assessment System with Active Human-AI Interactions
CHS:小型:DeepCrowd:一种基于人群辅助、基于深度学习的灾难场景评估系统,具有主动人机交互功能
- 批准号:
2130263 - 财政年份:2021
- 资助金额:
$ 60万 - 项目类别:
Standard Grant
CHS: Small: DeepCrowd: A Crowd-assisted Deep Learning-based Disaster Scene Assessment System with Active Human-AI Interactions
CHS:小型:DeepCrowd:一种基于人群辅助、基于深度学习的灾难场景评估系统,具有主动人机交互功能
- 批准号:
2008228 - 财政年份:2021
- 资助金额:
$ 60万 - 项目类别:
Standard Grant
Efficient Active Online Learning for 3D Reconstruction and Scene Understanding
用于 3D 重建和场景理解的高效主动在线学习
- 批准号:
260350367 - 财政年份:2014
- 资助金额:
$ 60万 - 项目类别:
Priority Programmes
SCENE IN SPRINGFIELD (SIS) OUT, ACTIVE AND EATING WELL
斯普林菲尔德 (SIS) 外出的场景,活跃且饮食良好
- 批准号:
8521608 - 财政年份:2012
- 资助金额:
$ 60万 - 项目类别:
RI: Medium: Active Scene Interpretation by Entropy Pursuit
RI:中:熵追踪的活动场景解释
- 批准号:
0964416 - 财政年份:2010
- 资助金额:
$ 60万 - 项目类别:
Continuing Grant
Computational Auditory Scene Analysis Using Active Audio-Visual Integration in a Dynamically Changing Environment
在动态变化的环境中使用主动视听集成进行计算听觉场景分析
- 批准号:
22700165 - 财政年份:2010
- 资助金额:
$ 60万 - 项目类别:
Grant-in-Aid for Young Scientists (B)
Study on Computational Auditory Scene Analysis for Humanoids by Active Audition
基于主动听觉的类人计算听觉场景分析研究
- 批准号:
15200015 - 财政年份:2003
- 资助金额:
$ 60万 - 项目类别:
Grant-in-Aid for Scientific Research (A)
Integrated Sensing: Active Stereoscopic Visual Search Driven by Natural Scene Statistics
集成传感:自然场景统计驱动的主动立体视觉搜索
- 批准号:
0225451 - 财政年份:2002
- 资助金额:
$ 60万 - 项目类别:
Continuing Grant
CISE Research Instrumentation: Active Learning for Text, Scene, and Biosequence Analysis
CISE 研究仪器:用于文本、场景和生物序列分析的主动学习
- 批准号:
9617307 - 财政年份:1997
- 资助金额:
$ 60万 - 项目类别:
Standard Grant