RI: Medium: Collaborative Research: Towards Practical Encoderless Robotics Through Vision-Based Training and Adaptation

随着机器人扩展到非结构化和动态的人类环境(如家庭、办公室和医院)，它们需要一种新的设计方法。这些机器人需要在人类旁边安全操作；它们应该处理人类环境中固有的频繁变化和不确定因素；它们应该尽可能便宜，以便能够广泛传播。这些标准导致了兼容/软机器人、3D打印机器人和廉价的消费级硬件的出现，所有这些都构成了工业中使用的具有严格公差的重型和刚性机器人的重大转变。适合于传感和控制刚性机器人运动的传统测量设备，即关节编码器，对于这些新型机器人中的许多来说是不兼容或不实用的。机器人技术在很大程度上缺少不依赖编码器的替代方法，必须为这些新的设计模型开发。这个项目研究了只使用摄像头来感知和控制机器人运动的方法。将推导出基于视觉的机器人行走、对象抓取和操作的算法。这些算法不仅将使新一代机器人在非结构化环境中使用，而且将显著降低传统机器人系统的成本，从而促进其在工业和教育目的的传播。该项目将专注于利用基于视觉的估计方案和学习方法来获取机器人的结构信息和任务模型，其中建模误差和环境不确定性通过稳健的视觉伺服方法来处理。视觉观测将被用来对执行器输入、机械手配置和任务状态之间的关系进行建模，并将它们与基于视觉的自适应控制方案相结合，这些方案对建模不确定性和干扰具有鲁棒性。该框架将从根本上依赖于使用卷积神经网络(CNN)来仅根据观察来建立模型，无论是对于构形的低维表示还是对于机械手的图像分割。强化学习方法也将被用来评估这种方法与离线学习表示法的模块组合的实用性，以执行复杂的定位和控制任务。这些方法将在手内操作、顺应性手术工具控制、3D打印多腿机器人的移动以及使用软连续操纵器的力控制抓取和钉插入的背景下进行评估。我们提出的工作的贡献是不需要机器人配置的先验模型，因为它是预先明确观察和推断的(系统识别)；影响任务性能的不确定性通过实时适应机器人动力学(通过确认建立模型)来解决；我们的方法的广泛适用性将通过在各种平台上的应用来展示。在这个项目上所做的工作将有助于在一系列领域实现更低成本的机器人和机电一体化硬件，并将特别影响控制合规和欠驱动结构的能力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As robots branch out into unstructured and dynamic human environments (such as homes, offices, and hospitals), they require a new design methodology. These robots need to be safe to operate next to humans; they are expected to handle frequent changes and uncertainties that are inherent in human environments; and they should be as inexpensive as possible to enable wide-spread dissemination. Such criteria have lead to the emergence of compliant/soft robots, 3D printed robots, and inexpensive consumer-grade hardware, all of which constitute a major shift from heavy and rigid robots with tight tolerances used in industry. Traditional measurement devices that are suitable for sensing and controlling the motion of the rigid robots, i.e. joint encoders, are incompatible or impractical for many of these new types of robot. Alternative approaches that do not rely on encoders are largely missing from robotics technology and must be developed for these novel design models. This project investigates ways of using only cameras for sensing and controlling the robot's motion. Vision-based algorithms for robotic walking, object grasping and manipulation will be derived. Such algorithms will not only enable the use of the new-wave robots in unstructured environments but will also significantly lower the cost of traditional robotic systems, and therefore, boost their dissemination for industry and educational purposes.The project will focus on utilizing vision-based estimation schemes and learning methods for acquiring both robot configuration information and task models within a framework where modeling inaccuracies and environment uncertainties are dealt with by robust visual servoing approaches. Visual observations will be used to model the relationship between actuator inputs, manipulator configuration, and task states, and they will be combined with adaptive vision-based control schemes that are robust to modeling uncertainties and disturbances. The framework will fundamentally rely on using convolutional neural networks (CNNs) to build the models from observation alone, both for a low-dimensional representation of configuration and for an image segmentation of the manipulator. Reinforcement learning methods will also be applied to assess the practicality of a modular combination of such methods with the offline learned representations to perform complex positioning and control tasks. These approaches will be evaluated in the context of within-hand manipulation, compliant surgical tool control, locomotion of a 3D-printed multi-legged robot, and force-controlled grasping and peg-insertion using a soft continuum manipulator. The contributions of our proposed work are that no prior model of a robot's configuration is needed because it is explicitly observed and inferred up-front (system identification); uncertainty affecting task performance is addressed by adapting the robot dynamics on-the-fly (model-through-confirmation); and the broad applicability of our methods will be demonstrated through application to a wide variety of platforms. Work done on this project will help to enable lower cost robotic and mechatronic hardware across a range of domains and will particularly impact the ability to control compliant and under-actuated structures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.