NRI: INT: COLLAB: Shared Autonomy for Unstructured Underwater Environments through Vision and Language

点击翻译按钮获取中文摘要

Existing underwater robotic systems typically provide one of two operating modes---full teleoperation or full autonomy. Teleoperation is by far the most common, particularly for tasks involving interaction with the environment, such as grasping and manipulation. Autonomy is restricted to non-contact survey missions and to controlled laboratory settings. The ability to operate between teleoperation and autonomy will improve the efficiency and effectiveness of tasks performed in underwater environments. This research will develop and evaluate a novel shared autonomy framework. The research leverages the different nature of humans and robots. This work will reduce the need for multiple, highly trained operators. It has the potential to drastically reduce the cost of underwater missions. The contributions of this research will impact the way in which humans work together with robots within a wide variety of applications, including space exploration, disaster relief, and assistive robotics.As robotic systems play an ever-larger role as our surrogates for marine science and exploration, the ability to leverage the complementary nature of humans and robots becomes critical for scientific discovery. This research will develop new models and algorithms that exploit multiple non-commensurate sensing and control modalities to realize intelligent shared autonomy in complex unstructured environments. Novel to this research is the use of natural language and vision as complementary forms of weak supervision to enable robots to learn human-collaborative sensorimotor manipulation policies opportunistically from narrated human demonstrations. Fundamental to these methods is their ability to then refine these policies in situ based upon interaction with a human operator. Together, these models and algorithms will enhance the efficiency and effectiveness of underwater scientific exploration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.