NRI: Collaborative Research: Shall I Touch This?: Navigating the Look and Feel of Complex Surfaces

这个项目改进了自主机器人的感知能力，这样未来的合作机器人就可以环视任何场景，并准确估计抓住或踩在所有可见表面上的感觉。就像人类所做的那样，机器人应该使用这些物理预测来指导它们与世界的互动，例如，在行走和驾驶时避免地面上危险的冰块，并熟练地预测捡起从冰块到填充动物的一切所需的抓取力。在这些研究活动的同时，还开展了重大的推广活动，包括一个新的“Look and Touch Robotics”项目，旨在让中学生，特别是那些来自代表人数不足的群体的学生，对计算机科学、工程学和机器人学感到兴奋。该项目使用简单的实验来强调视觉和触觉信息在与物理对象互动过程中的双重重要性，并展示了机器人展示视觉触觉智能。该项目还通过让本科生参与研究并通过首席研究员教授的视觉和机器人课程的实践项目来整合研究和教育。这项研究涉及使用视觉和触觉传感器从真实物体和表面广泛收集数据。分析记录的相互作用以揭示视觉线索，这些线索可以让机器人推断表面的物理特征，如光滑、硬度和粗糙度。这个问题是通过深度学习来解决的，深度学习是最近开发的一种方法，它成功地使机器人能够在不同的环境中视觉识别各种对象。研究小组还建立了视觉-触觉记录和学习的跨模式感觉的数据库，并在项目结束时提供给其他机器人研究人员使用。

This project improves autonomous robotic perception so that future co-robots can glance around any scene and accurately estimate how it would feel to grasp or step on all of the visible surfaces. Just as people do, robots should use such these physical predictions to guide their interactions with the world, for example avoiding dangerous ice patches on the ground when walking and driving, and adeptly anticipating the grasp force needed to pick up everything from ice cubes to stuffed animals. These research activities are accompanied by significant outreach efforts, including a new program on "Look and Touch Robotics" to get middle-school students, particularly those from under­represented groups, excited about computer science, engineering, and robotics. This program uses simple experiments to highlight the dual importance of visual and haptic information during interactions with physical objects, along with demonstrations of a robot showing visuo-­haptic intelligence. This project also integrates research and education by involving undergraduates in the research and via hands-on projects in the vision and robotics classes taught by the Principal Investigators.This research involves extensive collection of data from real objects and surfaces using both visual and haptic sensors. The recorded interactions are analyzed to uncover visual clues that can allow a robot to infer the physical characteristics of the surface, such as slipperiness, hardness, and roughness. This problem is addressed using deep learning, a recently developed approach that has been successful in enabling robots to visually recognize a wide variety of objects in diverse circumstances. The research team also builds the database of visuo-haptic recordings and the learned cross-modal sensory, and makes it available to other robotics researchers at the end of the project.