权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CCRI: ENS: Activity-Centric Interactive Environments for Embodied AI

CCRI：ENS：以活动为中心的嵌入式人工智能交互环境

基本信息

批准号：
2120095
负责人：
Jiajun Wu
金额：
$ 183万
依托单位：
Stanford University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-10-01 至 2025-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2120095&HistoricalAwards=false
关键词：
CCRI ENS Activity Centric Interactive

项目摘要

Humans effortlessly engage in a diverse set of complex daily activities: cooking a meal, packing lunches, cleaning a bedroom. Designing artificial agents and robots that can perform such activities in dynamic environments would be immensely useful with many applications such as healthcare (e.g., assist the elderly). Still, it remains a long-standing, challenging problem. While recent research in artificial intelligence (AI) and robotics has shown promising progress, it has focused on learning and executing simple tasks such as pushing an object in a table-top environment. This project will build infrastructure that enables the development of intelligent agents that perceive and interact with humans in real-world household environments. In turn, this facilitates robotic systems that help humans accomplish household activities, with significant potential societal and economic impact. Researchers will also partner with educational and non-profit organizations to educate AI and robotics to underrepresented students and communicate the proposed infrastructure to families with diverse backgrounds, particularly from underserved communities.In this project, investigators will develop a virtual environment for embodied agents to perform interactive activities, with a formal description of everyday activities inspired by cognitive science findings, facilitating the development of embodied AI in complex, ecological scenes. The developed environment will enable simulation of activities in large-scale, realistic, interactive, home-sized scenes, populated with rigid and articulated objects with state changes (e.g., temperature). This project will build upon existing infrastructure from the team of researchers, notably the iGibson environment, but significantly extend it to include key features to facilitate embodied AI study. These features include (i) high-quality multisensory signals (visual, tactile, auditory), (ii) realistic and diverse geometry, material, and physics of objects, (iii) an intuitive human interface with a virtual reality setup, and (iv) a multi-user collaborative virtual reality system. In addition, the team of researchers will develop a taxonomy of daily activities, including their pre-conditions, effects, and hierarchy, using formal description in robotics, accompanied by human demonstrations from virtual reality for evaluating the performance of autonomous agents on such activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类毫不费力地从事各种复杂的日常活动：做饭，打包午餐，打扫卧室。设计可以在动态环境中执行这种活动的人工代理和机器人将对许多应用非常有用，例如医疗保健（例如，帮助老人）。尽管如此，它仍然是一个长期存在的具有挑战性的问题。虽然最近在人工智能（AI）和机器人技术方面的研究取得了可喜的进展，但它一直专注于学习和执行简单的任务，例如在桌面环境中推动物体。该项目将建立基础设施，使智能代理的发展，感知和与人类在现实世界的家庭环境中进行交互。反过来，这有助于机器人系统帮助人类完成家庭活动，具有重大的潜在社会和经济影响。研究人员还将与教育和非营利组织合作，向代表性不足的学生提供人工智能和机器人技术教育，并将拟议的基础设施传达给不同背景的家庭，特别是来自服务不足社区的家庭。在这个项目中，研究人员将开发一个虚拟环境，供实体代理执行交互活动，并对受认知科学发现启发的日常活动进行正式描述，促进在复杂的生态场景中开发嵌入式AI。开发的环境将能够模拟大规模，逼真，交互式，家庭大小的场景中的活动，填充有状态变化的刚性和铰接对象（例如，温度）。该项目将建立在研究团队现有的基础设施上，特别是iGibson环境，但将其扩展到包括促进嵌入式AI研究的关键功能。这些功能包括（i）高质量的多感官信号（视觉，触觉，听觉），（ii）逼真和多样化的几何形状，材料和物体的物理特性，（iii）具有虚拟现实设置的直观人机界面，以及（iv）多用户协作虚拟现实系统。此外，研究人员团队将开发一个日常活动的分类，包括他们的先决条件，效果和层次结构，使用机器人技术中的正式描述，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的评估来支持。影响审查标准。

项目成果

期刊论文数量（12）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

DOI：
10.1109/cvpr52688.2022.01034
发表时间：
2022-04
期刊：
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
影响因子：
0
作者：
Ruohan Gao;Zilin Si;Yen-Yu Chang;Samuel Clarke;J. Bohg;Li Fei-Fei-Li-Fei-Fei-48004138;Wenzhen Yuan;Jiajun Wu
通讯作者：
Ruohan Gao;Zilin Si;Yen-Yu Chang;Samuel Clarke;J. Bohg;Li Fei-Fei-Li-Fei-Fei-48004138;Wenzhen Yuan;Jiajun Wu

ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations

DOI：
发表时间：
2021-09
期刊：
ArXiv
影响因子：
0
作者：
Ruohan Gao;Yen-Yu Chang;Shivani Mall;Li Fei-Fei-Li-Fei-Fei-48004138;Jiajun Wu
通讯作者：
Ruohan Gao;Yen-Yu Chang;Shivani Mall;Li Fei-Fei-Li-Fei-Fei-48004138;Jiajun Wu

An Extensible Multimodal Multi-task Object Dataset with Materials

DOI：
10.48550/arxiv.2305.14352
发表时间：
2023-04
期刊：
ArXiv
影响因子：
0
作者：
Trevor Scott Standley;Ruohan Gao;Dawn Chen;Jiajun Wu;S. Savarese
通讯作者：
Trevor Scott Standley;Ruohan Gao;Dawn Chen;Jiajun Wu;S. Savarese

Neurosymbolic Models for Computer Graphics

DOI：
10.1111/cgf.14775
发表时间：
2023-04
期刊：
Computer Graphics Forum
影响因子：
2.5
作者：
Daniel Ritchie;Paul Guerrero;R. K. Jones;N. Mitra;Adriana Schulz;Karl D. D. Willis-Karl-D.-D.-Willis-2269914;Jiajun Wu
通讯作者：
Daniel Ritchie;Paul Guerrero;R. K. Jones;N. Mitra;Adriana Schulz;Karl D. D. Willis-Karl-D.-D.-Willis-2269914;Jiajun Wu

RealImpact: A Dataset of Impact Sound Fields for Real Objects

RealImpact：真实物体的冲击声场数据集

DOI：
发表时间：
2023
期刊：
IEEE/CVF Conference on Computer Vision and Pattern Recognition
影响因子：
0
作者：
Clarke, Samuel;Gao, Ruohan;Wang, Mason;Rau, Mark;Xu, Julia;Wang, Jui-Hsien;James, Doug L.;Wu, Jiajun
通讯作者：
Wu, Jiajun