NRI: FND: Using Multi-Modal Data to Make Robotic Grasp Algorithms Aware of Human Preferences for Safe Collaborative Robot-Human Handover Interactions with Novel Objects

点击翻译按钮获取中文摘要

This project contributes advancements in promoting safe collaborative robot-to-human handovers by making robots with manipulator arms aware of human preferences for interactions with objects. In environments such as healthcare facilities, warehousing, retail, engine repair, and aircraft assembly, where robots may be expected to collaborate with humans for successful accomplishment of tasks, it is essential that robotic manipulators hand over objects such that people can optimally hold them, without fear of the object falling or the person being injured by the gripper or arm, and without the inconvenience of the object being unreachable. To enable safe handovers, the project will provide algorithms that use data on human interactions with objects captured from multiple viewpoints to automatically predict preferred locations of human grasp on objects, optimal orientation and distance of the object from the person, and safe point of release of the object by manipulator grippers. The research team will reach out to two-year and four-year colleges with limited technological opportunities in the North Country to provide research opportunities to women and students from underrepresented communities.The project advances research in ubiquitous co-robots by providing holistic fine-grained insight through multi-modal sensing on natural behaviors of people as they interact with each other and with objects in their environments. The project accomplishes three objectives to address the gap on propagating understanding of human handover preferences to large collections of novel in-the-wild objects for customizability of co-robots to new environments. First, the research team will collect a large multi-viewpoint multi-modal dataset on two-person handovers and perform empirical analysis of the collected data to understand preferences on hold locations, end pose, and release point using subject ratings of object presentations. Modalities used will consist of depth cameras to acquire understanding on object geometry and spatial relationships, and thermal cameras to analyze locations of human contact based on heat transferred to object surfaces. This work will provide a quantitative decomposition of human preferences for handover parameters in terms of geometric form and functionality of objects. Second, the team will create perception algorithms based on probabilistic models to perform prediction of handover parameters ranked in order of preference using depth images of objects as input. This work enables equipping co-robots with human-like awareness of diversity in preferences, and the priorities that people assign to interactions. Third, the team will provide robotic manipulators that use the trained perception algorithms to perform handover manipulations on novel objects while being aware of human behavior. Successful accomplishment of the project activities will enable rapid propagation of robotic manipulators aware of human handover behavior to new objects and environments for enhanced social acceptability of co-robots.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.