NRI: Collaborative Research: Shall I Touch This?: Navigating the Look and Feel of Complex Surfaces

该项目提高了自主机器人的感知能力，使未来的协作机器人可以环视任何场景，并准确估计抓取或踩在所有可见表面上的感觉。就像人类一样，机器人应该利用这些物理预测来指导它们与世界的互动，例如，在走路和开车时避开地面上危险的冰面，熟练地预测从冰块到填充动物等一切东西所需的抓握力。这些研究活动伴随着重要的外联努力，包括一个名为“看和触摸机器人”的新项目，旨在让中学生，特别是那些来自弱势群体的中学生，对计算机科学、工程和机器人产生兴趣。该程序使用简单的实验来强调视觉和触觉信息在与物理对象交互过程中的双重重要性，以及展示视觉-触觉智能的机器人演示。该项目还整合了研究和教育，让本科生参与研究，并通过首席研究员教授的视觉和机器人课程的实践项目。这项研究涉及使用视觉和触觉传感器从真实物体和表面广泛收集数据。通过分析记录的相互作用，可以发现视觉线索，从而使机器人能够推断表面的物理特性，如滑度、硬度和粗糙度。这个问题是通过深度学习来解决的，深度学习是一种最近开发的方法，它已经成功地使机器人能够在不同的环境中视觉识别各种各样的物体。研究小组还建立了视觉-触觉记录和学习的跨模态感觉数据库，并在项目结束时将其提供给其他机器人研究人员。

This project improves autonomous robotic perception so that future co-robots can glance around any scene and accurately estimate how it would feel to grasp or step on all of the visible surfaces. Just as people do, robots should use such these physical predictions to guide their interactions with the world, for example avoiding dangerous ice patches on the ground when walking and driving, and adeptly anticipating the grasp force needed to pick up everything from ice cubes to stuffed animals. These research activities are accompanied by significant outreach efforts, including a new program on "Look and Touch Robotics" to get middle-school students, particularly those from under­represented groups, excited about computer science, engineering, and robotics. This program uses simple experiments to highlight the dual importance of visual and haptic information during interactions with physical objects, along with demonstrations of a robot showing visuo-­haptic intelligence. This project also integrates research and education by involving undergraduates in the research and via hands-on projects in the vision and robotics classes taught by the Principal Investigators.This research involves extensive collection of data from real objects and surfaces using both visual and haptic sensors. The recorded interactions are analyzed to uncover visual clues that can allow a robot to infer the physical characteristics of the surface, such as slipperiness, hardness, and roughness. This problem is addressed using deep learning, a recently developed approach that has been successful in enabling robots to visually recognize a wide variety of objects in diverse circumstances. The research team also builds the database of visuo-haptic recordings and the learned cross-modal sensory, and makes it available to other robotics researchers at the end of the project.