Opportunistic Visual and Tactile Sensing for Autonomous Systems Perception and Robotic Applications

自主机器人系统是保护人口、支持医疗保健和维持经济繁荣的战略途径。然而，最先进的机器人技术在其传感和感知能力方面仍然面临限制，限制了它们的实际自主水平。在遇到的挑战中，获取大量数据很快会导致处理能力饱和，并影响受过教育的决策。这项实验研究计划专注于开发机会主义的智能传感策略，将机器视觉和触觉探测相结合，以支持自主决策系统和先进的机器人应用。机会主义感知被定义为基于对给定环境的部分了解，从战略上智能地选择下一步要获取什么信息以及从哪里获取。最终目标是在增强自治系统的决策能力的同时最小化所需的总数据量。机会主义感知侧重于收集对机器人操作的成功、安全、精度和健壮性具有战略重要性的区域或对象的信息。拟议的研究计划建立在我们最近的成功基础上，i)用于工业应用的RGB-D传感器的机会传感，ii)异类机器人代理之间的最佳任务分配，以及iii)用于表征可变形物体的视觉和触觉传感。拟议研究计划的主要目标是开发集成的感知和决策机器人解决方案，将视觉、力和触觉感知结合起来，目标是检测、有选择地测量、建模和解释复杂环境。对场景的理解可以支持协作移动机器人或机器人手臂/手组件的操作，以进一步选择性地探测、交互和操纵对象。这一目标将通过以下方式实现：i)扩展我们正式的机会主义3D成像方法，以解决传感器的移动性问题；ii)利用最先进的机器学习和视觉注意方法，提高分布式机器人代理的协调性能和稳健性；以及iii)开发正式方法，以高效地获取视觉和触觉信息，并在高效框架中结合视觉和触觉信息，以支持机器人与可变形对象的灵活交互。这项研究将为机器人应用提供与有效获取、表示和决策方法相关的基础知识。在机器人可以干预的一系列现实世界场景中，它为无人系统获取知识的方式提供了重大突破的潜力。这项技术将在实际的传感和机器人平台上开发和验证，应用范围包括国防/安全、协作移动机器人、灵活操纵可变形物体和先进制造。

Autonomous robotic systems represent a strategic avenue to protect populations, support health care, and sustain economic prosperity. However, state-of-the-art robotic technologies still face limitations in their sensing and perception capability that limit their actual level of autonomy. Among the challenges encountered, the acquisition of large amounts of data quickly leads to saturation of processing capabilities and compromises educated decision making. This experimental research program focuses on the development of opportunistic, intelligent sensing strategies that combine machine vision and tactile probing to support autonomous decision systems and advanced robotic applications. Opportunistic sensing is defined as intelligently selecting what information to acquire next and from where, strategically based upon a partial understanding of a given environment. The ultimate goal is to minimize the total amount of data required while enhancing decision capabilities of an autonomous system. Opportunistic sensing focuses the collection of information over regions or objects that are of strategic importance for the success, safety, precision and robustness of a robotic operation. The proposed research program builds upon our recent successes in i) opportunistic sensing with RGB-D sensors for industrial applications, ii) optimal task allocation among heterogeneous robotic agents, and iii) visual and tactile sensing for the characterization of deformable objects. The main objective of the proposed research program is to develop integrated perceptual and decisional robotic solutions that combine vision, force and tactile perception with the goal to detect, selectively measure, model and interpret complex environments. Understanding of the scene can support the operation of collaborative mobile robots, or a robot arm/hand assembly to further selectively probe, interact with, and manipulate objects. The objective will be achieved by: i) expanding our formal opportunistic 3D imaging methods to tackle sensors' mobility issues; ii) leveraging state-of-the-art machine learning and visual attention methods to increase performance and robustness for the coordination of distributed robotic agents, and iii) developing formal methods to efficiently acquire and combine visual and tactile information in an efficient framework to support dexterous robotic interaction with deformable objects. This research will contribute fundamental knowledge related to efficient acquisition, representation and decisional methods for robotic applications. It offers the potential for significant breakthroughs in the way knowledge is acquired by unmanned systems, over a range of real-world scenarios where robots can intervene. The technology will be developed and validated on actual sensing and robotic platforms, in various contexts of application including defence/security, collaborative mobile robots, dexterous manipulation of deformable objects, and advanced manufacturing.