CDI-Type II: Collaborative Research: Perception of Scene Layout by Machines and Visually Impaired Users

该项目研究用于对象检测、空间场景构建和来自实时视觉图像的自然语言空间描述的计算方法，以描述典型的室内空间(例如，房间、办公室等)。这项研究的主要应用是为盲人或视障用户提供关于他们周围环境的空间信息，否则这些信息可能很难从非视觉感知中获得。这些知识将有助于开发准确的环境认知模型，并将支持在日常任务中更好地执行空间行为。这项工作的第二个动机是为提高计算机和机器人的空间能力做出贡献。计算机和机器人同样对图像视而不见，除非它们被提供了某种方式来“看”和理解它们。目前，没有机器人系统能够基于图像序列可靠地执行空间信息的高级处理，例如，寻找房间中的空椅子，这不仅意味着在图像中寻找空椅子，而且还定位房间中的椅子，并执行到达椅子的动作。这项研究的指导原则是，更好地理解从视觉图像中获取空间知识和人类空间意识的概念，也可以用来减少自主系统信息处理的模糊性和不确定性。这项工作的一个主要贡献是向视障人士提供视觉图像的空间信息内容，视障人士是我们老龄化社会中迅速增长的人口群体。在一个示例场景中，一个盲人和她的导盲犬正走向她的医生办公室，这是一个她以前从未去过的办公室。在办公室，她需要一些信息来执行一些基本的任务，比如寻找办理登机手续的柜台、空座位或洗手间。现有的可访问导航系统都不能描述环境的空间参数，并帮助检测和定位该空间中的对象。我们的工作将为实现这样一个系统提供基础研究和要素。

The project investigates computational methods for object detection, spatial scene construction, and natural language spatial descriptions derived from real-time visual images to describe prototypical indoor spaces (e.g., rooms, offices, etc.). The primary application of this research is to provide blind or visually impaired users with spatial information about their surroundings that may otherwise be difficult to obtain from non-visual sensing. Such knowledge will assist in development of accurate cognitive models of the environment and will support better informed execution of spatial behaviors in everyday tasks. A second motivation for the work is to contribute to the improvement of spatial capacities for computers and robots. Computers and robots are similarly "blind" to images unless they have been provided some means to "see" and understand them. Currently, no robotic system is able to reliably perform high-level processing of spatial information on the basis of image sequences, e.g., to find an empty chair in a room, which not only means finding an empty chair in an image, but also localizing the chair in the room, and performing an action of reaching the chair. The guiding tenet of this research is that a better understanding of spatial knowledge acquisition from visual images and concepts of spatial awareness by humans can also be applied to reducing the ambiguity and uncertainty of information processing by autonomous systems. A central contribution of this work is to make the spatial information content of visual images available to the visually impaired, a rapidly growing demographic of our aging society. In an example scenario a blind person and her guide dog are walking to her doctor's office, an office which she has not previously visited. At the office she needs information for performing some essential tasks such as finding the check-in counter, available seating, or the bathroom. No existing accessible navigation systems are able to describe the spatial parameters of an environment and help detect and localize objects in that space. Our work will provide the underlying research and elements to realize such a system.