NRI: FND: Learning Visual Dynamics from Interaction

该项目研究利用附近和可用的物理对象来执行任务的机器人，例如在灾区用各种瓦砾建造桥梁。资源丰富的机器人有潜力在应急响应、医疗保健和制造业中实现许多新的应用，这将提高整体人口的福利、安全和效率。该研究调查了视觉、声音和触觉等多种感官之间的模式如何帮助机器人在不需要人类教师的情况下解决交互任务，这有望提高自主机器人的灵活性和多功能性。该项目将为计算机科学和机械工程的研究生和本科生提供研究和教育机会。这项研究的成果将转化为计算机视觉，机器学习和机器人技术的新教材。本研究探讨机器人与现实环境的互动，以学习可重复使用的表示导航和操作任务。虽然利用机器学习解决计算机视觉和机器人问题已经取得了重大进展，但这两个领域的核心挑战是推广到物理世界的现实复杂性和多样性。虽然仿真已经证明在开发机器交互平台方面是有用的，但无约束的世界是广阔的，这使得仿真在计算上很困难。相反，研究人员的目标是通过模态和上下文的自然同步来利用物理环境的固有结构，以有效地学习自我监督的表示和策略，以便与不受约束的环境进行交互。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project studies robots that utilize the nearby and available physical objects to perform tasks, such as building a bridge out of miscellaneous rubble in a disaster area. Resourceful robots have the potential to enable many new applications in emergency response, healthcare, and manufacturing, which will improve the welfare, security, and efficiency of the overall population. The research investigates how the patterns between multiple senses, such as vision, sound, and touch, will help teach the robot to solve interaction tasks without needing a human teacher, which is expected to improve the flexibility and versatility of autonomous robots. The project will provide research and educational opportunities for both graduate and undergraduate students in computer science and mechanical engineering. Outcomes from this research will translate into new educational materials in computer vision, machine learning, and robotics. This research investigates robots that interact with realistic environments in order to learn reusable representations for navigation and manipulation tasks. While there has been significant advancements leveraging machine learning for computer vision and robotics problems, a central challenge in both fields is generalizing to the realistic complexity and diversity of the physical world. Although simulation has proved instrumental in developing platforms for machine interaction, the unconstrained world is vast, making it computationally difficult to simulate. Instead, the investigators aim to capitalize on the inherent structure of physical environments through the natural synchronization of modalities and context to efficiently learn self-supervised representations and policies for interaction with unconstrained environments. The investigators also plan several evaluations to analyze the generalization capabilities of such algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.