Telerobotics Simulation in Mixed Reality

空间计算、机器学习和远程操作领域最先进的研究成果的融合，推动了领域、工业和医疗机器人的变革。潜在的积极影响最能在对人类过于危险的场景中观察到，在这些场景中，机器人化身是不可替代的。深海就是这样一种情景。在这份报告中，我们回顾了围绕无人水下航行器(UUV)、遥控和计算机视觉等几个主题的文献，以确定研究差距。综合考虑几个方面的不足，提出了一种混合现实模拟遥操作系统。在描述用于在水下环境中执行深度估计、目标检测和分割以及混合现实实验的同步立体视觉系统之前，对计算机视觉和立体视觉进行了初步的研究。还讨论了未来在水箱中进行可变照明和浑浊测试的工作，以及混合现实视听辅助设备的上下文感知功能。探索无人[水下]航行器遥控的现状和挑战。开发计算机视觉和/或图像处理技术，以提高水下机器人及其机械手在作业过程中的效率和感知能力。以数据为中心的水下机器人在其环境中的仿真建模，以实现机器人和场景数据的高效混合现实反馈。目标：1.a.与行业利益攸关方接触，收集水下机器人操作员的反馈。进行学术文献调查，以了解移动机器人遥操作的最新技术和当前的挑战。探索水下机器人市场，了解水下机器人的控制和传感能力。针对不同的应用案例/场景，对水下机器人的计算机视觉和水下图像处理进行学术文献调研。探索现有的计算机视觉技术和框架。水下机器人管道方案的实施(S).所开发技术在给定场景下的测试性能(S).E.开发可视化技术以支持人工操作员。对远程机器人和环境进行以数据为中心的建模和仿真的学术文献调查。研究目标3的进一步目标将在稍后阶段确定，并将取决于研究目标1和2的结果。

The convergence of state-of-the-art research in spatial computing, machine learning, and teleoperation proposes to transform field, industrial, and medical robotics. The potential positive impact is most observed in scenarios too dangerous to humans, where robotic avatars are irreplaceable. One such scenario is the deep sea. In this report we review literature in several topics surrounding unmanned underwater vehicles (UUVs), teleoperation, and computer vision to identify research gaps. Several gaps are considered, and a mixed reality simulated teleoperation system is proposed. Preliminary computer vision and stereo vision studies are investigated before a synchronised stereo vision system is described for performing depth estimation, object detection and segmentation, and mixed reality experiments in an underwater environment. Future work on variable lighting and turbidity tests in the tank, and context-awareness features of the mixed reality audiovisual aids are also discussed.RESEARCH AIMS1. Explore state of the art and challenges in remote-control of unmanned [underwater] vehicles.2. Develop a computer vision and/or image processing technique for enhancing efficiency and perception of an underwater vehicle and its manipulators during operation.3. Data-centric simulation modelling of an underwater vehicle in its environment for efficient mixed reality feedback of robot and scene data. OBJECTIVES1.a. Engage with industry stakeholders to collect feedback from human operators of underwater vehicles.b. Perform an academic literature survey to understand the state of the art and current challenges in mobile robot teleoperation.c. Explore underwater vehicles market to understand control and sensing capabilities.2.a. Perform academic literature survey for computer vision and underwater image processing for underwater vehicles for different applied use cases/scenarios.b. Explore existing computer vision techniques and frameworks.c. Implement pipeline for underwater vehicle scenario(s).d. Test performance of the developed technique for given scenario(s).e. Develop visualisation technique to support human operator.3.a. Perform academic literature survey for data-centric modelling and simulation of remote robots and environments.b. Further objectives of research aim 3 will be identified at a later stage and will depend on the outcomes of research aims 1 and 2.