Engineering Robust 3D Representations from Robotic Visual Sensors for Navigation & Scene Analysis

无论我们是走路还是开车，我们都在不断地在脑海中绘制地图，以确定路径/道路、地标、物体和其他人类代理的位置以及它们之间的关系。这些心理地图和模型使我们能够导航到特定的位置，开车去上班，修剪草坪或打扫房屋。当我们推理或计划某些活动时，如修理篱笆或粉刷房屋，我们也会在脑海中构建地图。为机器人构建这样一个心理地图的能力是推动我们研究的动力。人类主要通过眼睛来做到这一点。如果我们可以为一个带摄像头的机器人做同样的事情呢？机器人将能够理解和推理世界，从而执行任务。心理地图只是时间的快照。该地图还必须包括区分移动实体与静态地标和占位符。根据手头的任务，详细程度会有所不同。静态组件的完整3D重建与分析、重新设计或可能的3D打印对象和环境相关。用于构建3D地图的两种视觉技术包括SFM（运动结构）和SLAM（同步定位和绘图）。这两种方法非常相似，区别在于SFM通常是离线的，而SLAM是在线的。这两种方法都包括一个前端，它检测感兴趣的特征，并使用摄影测量在视图之间将这些数据点关联起来。这两种方法的后端都是最小化重投影误差的优化方法。根据所使用的传感器、环境的复杂程度和所需的性能，这两个过程可能容易或困难。自动驾驶汽车得益于提供精确环境测量的激光雷达传感器和提供位置信息的全球定位系统。然而，对于许多现实世界的应用程序，仅仅依靠可视化SLAM/SFM并不健壮。SLAM/SFM解决方案可能非常脆弱，当相机在尖锐的角落时，例如在办公楼中，姿势跟踪失败并且无法注册视图。有一个严重依赖特征点数据关联工业清洗机只有一个相机系统；（二）绘制道路、桥梁等市政基础设施监测历史地图；和其他人。一个健壮的SLAM/SFM解决方案可以帮助解决这些应用程序和其他应用程序。**

Whether we are walking or driving, we are constantly making mental maps to determine where the pathway/road, landmarks, objects and other human agents are situated and their relationships to each other. These mental maps and models are what allow us to navigate to a particular location, drive to work, mow our lawns or clean our homes. We also build mental maps when we reason or plan certain activities like repairing a fence or painting a home. Transferring the ability to build such a mental map for a robot is what drives our research. Humans are able to chiefly do this with our eyes. What if we could do the same for a robot with a camera? The robot would then be able to understand and reason in the world so as to perform a task. A mental map is just a snapshot in time. This map has to also include differentiating moving entities from static landmarks and placeholders. Depending on the task at hand, the level of detail will vary. A complete 3D reconstruction of the static components is relevant for analysis, re-engineering or possibly 3D printing the objects and environments. Two visual techniques that are used to build 3D maps include SFM (Structure from Motion) and SLAM (Simultaneous Localization And Mapping). The two methods are very similar, the differentiating factor being that SFM is typically off-line while SLAM is online. Both methods include a front end which detects features of interest and uses photogrammetry to associate these data points between views. The back-end for both methods is an optimization method that minimizes re-projection errors. Both processes may be easy or difficult depending on the sensors being used, the level of complexity in the environment and the required performance. Autonomous automobiles benefit from using a LIDAR sensor which provides precise environmental measurements and a GPS which provides location information. However, for many real world applications, just relying on visual SLAM/SFM is not robust. SLAM/SFM solutions can be very brittle when the camera makes sharp corners such as in an office building where the tracking of pose fails and views cannot be registered. There is a heavy reliance on the feature point data association industrial cleaners with only a camera system; (2) building historical maps for municipal infrastructure monitoring such as for roads, bridges; and others. A robust SLAM/SFM solution can help solve these applications and others. **