RI: Small: Indoor Visual Navigation and Recognition for the Blind Using a Motion Sensing Input Device

在这个项目中，目标是开发一个计算机视觉系统，以帮助视障人士在室内环境中导航。视觉系统的一个新特征是它使用运动感测输入设备来收集场景的图像和深度图作为输入。通过学习结合深度和强度信息的联合收割机，可以学习强大的特征，其相对于现有的场景理解算法（其仅依赖于强度信息）给出显著的改进。深度信息的可用性还允许恢复房间几何形状，并允许在对象的位置上构建新类型的3D先验，这在现有方法中是不可能的。视觉系统的输出将通过一些可能的方法传达给视障人士：（i）手杖上的触觉手柄;（ii）嵌入致动器的可穿戴垫;（iii）BrainPort传感器，该传感器具有放置在舌头上的微型致动器阵列。该项目的预期成果是：（i）具有深度图和密集标签的室内场景的大型数据集;（ii）具有深度图和密集标签的智能传感器。（ii）融合深度和强度信息以帮助理解场景的新的开放源代码算法，以及（iii）为视力受损者提供的基于视觉的辅助设备原型。该项目旨在帮助美国约250万失明或部分失明的人。

In this project the goal is to develop a computer vision system to assist visually impaired people when navigating in indoor environments. A novel feature of the vision system is that it uses a motion sensing input device to gather both an image and depth map of the scene as input. By learning joint representations that combine both depth and intensity information, powerful features can be learned that give a dramatic improvement over existing scene understanding algorithms (which rely on intensity information alone). The availability of depth information also allows the recovery of the room geometry and permits the construction of new types of 3D priors on the locations of objects, not currently possible with existing approaches. The output of the vision system will be communicated to the visually impaired person via a number of possible methods: (i) a tactile hand-grip on a cane; (ii) a wearable pad embedded with actuators and (iii) the BrainPort sensor which has an array of tiny actuators that are placed on the tongue.The expected results of the project are: (i) a large dataset of indoor scenes with depth maps and dense labels; (ii) new open-source algorithms for fusing depth and intensity information to aid scene understanding and (iii) a prototype vision-based assistive device for visually impaired people. The project aims to assist the approximately 2.5 million people in the US are blind or partially sighted.