CCRI: ENS: Activity-Centric Interactive Environments for Embodied AI

人类毫不费力地从事各种复杂的日常活动：做饭、打包午餐、打扫卧室。设计能够在动态环境中执行此类活动的人工代理和机器人对于医疗保健（例如，帮助老年人）等许多应用将非常有用。尽管如此，这仍然是一个长期存在的、具有挑战性的问题。虽然最近人工智能（AI）和机器人技术的研究显示出有希望的进展，但它主要集中在学习和执行简单的任务，比如在桌面环境中推动物体。该项目将建立基础设施，使智能代理能够在现实家庭环境中感知并与人类互动。这反过来又促进了机器人系统帮助人类完成家务活动，具有重大的潜在社会和经济影响。研究人员还将与教育和非营利组织合作，向代表性不足的学生传授人工智能和机器人技术，并将拟议的基础设施传达给不同背景的家庭，特别是来自服务不足社区的家庭。在这个项目中，研究人员将开发一个虚拟环境，让具身智能体进行互动活动，并根据认知科学发现对日常活动进行正式描述，促进具身人工智能在复杂生态场景中的发展。开发的环境将能够模拟大规模的、真实的、交互式的、家庭大小的场景中的活动，其中包含具有状态变化（例如温度）的刚性和铰接对象。该项目将建立在研究团队现有基础设施的基础上，特别是iGibson环境，但将其扩展到包括促进嵌入式人工智能研究的关键功能。这些特征包括(i)高质量的多感官信号（视觉、触觉、听觉），（ii）物体的逼真和多样的几何、材料和物理特性，（iii）具有虚拟现实设置的直观人机界面，以及（iv）多用户协作虚拟现实系统。此外，研究团队将开发日常活动的分类，包括其先决条件，效果和层次结构，使用机器人中的正式描述，并伴有虚拟现实中的人类演示，以评估自主代理在此类活动中的表现。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Humans effortlessly engage in a diverse set of complex daily activities: cooking a meal, packing lunches, cleaning a bedroom. Designing artificial agents and robots that can perform such activities in dynamic environments would be immensely useful with many applications such as healthcare (e.g., assist the elderly). Still, it remains a long-standing, challenging problem. While recent research in artificial intelligence (AI) and robotics has shown promising progress, it has focused on learning and executing simple tasks such as pushing an object in a table-top environment. This project will build infrastructure that enables the development of intelligent agents that perceive and interact with humans in real-world household environments. In turn, this facilitates robotic systems that help humans accomplish household activities, with significant potential societal and economic impact. Researchers will also partner with educational and non-profit organizations to educate AI and robotics to underrepresented students and communicate the proposed infrastructure to families with diverse backgrounds, particularly from underserved communities.In this project, investigators will develop a virtual environment for embodied agents to perform interactive activities, with a formal description of everyday activities inspired by cognitive science findings, facilitating the development of embodied AI in complex, ecological scenes. The developed environment will enable simulation of activities in large-scale, realistic, interactive, home-sized scenes, populated with rigid and articulated objects with state changes (e.g., temperature). This project will build upon existing infrastructure from the team of researchers, notably the iGibson environment, but significantly extend it to include key features to facilitate embodied AI study. These features include (i) high-quality multisensory signals (visual, tactile, auditory), (ii) realistic and diverse geometry, material, and physics of objects, (iii) an intuitive human interface with a virtual reality setup, and (iv) a multi-user collaborative virtual reality system. In addition, the team of researchers will develop a taxonomy of daily activities, including their pre-conditions, effects, and hierarchy, using formal description in robotics, accompanied by human demonstrations from virtual reality for evaluating the performance of autonomous agents on such activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.