CDI-Type II: Collaborative Research: Perception of Scene Layout by Machines and Visually Impaired Users

该项目研究了对象检测，空间场景构建和自然语言空间描述的计算方法，这些空间描述来自实时视觉图像，以描述原型室内空间（例如，房间、办公室等）。这项研究的主要应用是为盲人或视力受损的用户提供有关其周围环境的空间信息，否则可能难以从非视觉感知中获得。这些知识将有助于开发准确的环境认知模型，并支持在日常任务中更好地执行空间行为。这项工作的第二个动机是有助于提高计算机和机器人的空间能力。计算机和机器人同样对图像“视而不见”，除非它们被提供了一些“看到”和理解图像的方法。目前，没有机器人系统能够基于图像序列可靠地执行空间信息的高级处理，例如，在房间中找到空椅子，这不仅意味着在图像中找到空椅子，而且还定位房间中的椅子，并执行到达椅子的动作。本研究的指导原则是，更好地理解人类从视觉图像和空间意识概念中获取的空间知识，也可以应用于减少自主系统信息处理的模糊性和不确定性。这项工作的一个核心贡献是使视觉图像的空间信息内容提供给视障者，我们的老龄化社会的人口迅速增长。在一个示例场景中，盲人和她的导盲犬正在步行到她的医生的办公室，她之前没有去过该办公室。在办公室，她需要一些信息来执行一些基本任务，例如找到登机柜台、可用座位或浴室。现有的无障碍导航系统无法描述环境的空间参数，并帮助检测和定位该空间中的对象。我们的工作将提供实现这样一个系统的基础研究和元素。

The project investigates computational methods for object detection, spatial scene construction, and natural language spatial descriptions derived from real-time visual images to describe prototypical indoor spaces (e.g., rooms, offices, etc.). The primary application of this research is to provide blind or visually impaired users with spatial information about their surroundings that may otherwise be difficult to obtain from non-visual sensing. Such knowledge will assist in development of accurate cognitive models of the environment and will support better informed execution of spatial behaviors in everyday tasks. A second motivation for the work is to contribute to the improvement of spatial capacities for computers and robots. Computers and robots are similarly "blind" to images unless they have been provided some means to "see" and understand them. Currently, no robotic system is able to reliably perform high-level processing of spatial information on the basis of image sequences, e.g., to find an empty chair in a room, which not only means finding an empty chair in an image, but also localizing the chair in the room, and performing an action of reaching the chair. The guiding tenet of this research is that a better understanding of spatial knowledge acquisition from visual images and concepts of spatial awareness by humans can also be applied to reducing the ambiguity and uncertainty of information processing by autonomous systems. A central contribution of this work is to make the spatial information content of visual images available to the visually impaired, a rapidly growing demographic of our aging society. In an example scenario a blind person and her guide dog are walking to her doctor's office, an office which she has not previously visited. At the office she needs information for performing some essential tasks such as finding the check-in counter, available seating, or the bathroom. No existing accessible navigation systems are able to describe the spatial parameters of an environment and help detect and localize objects in that space. Our work will provide the underlying research and elements to realize such a system.