A System for Wearable Audio Navigation Integrating Advanced Localization and Auditory Display

对于美国数以百万计的视障人士来说，空间定向和导航是导致丧失行动能力和减少参与社区活动的主要问题，更不用说那些试图完全行动的人面临的严重安全问题。因此，迫切需要一种辅助工具，提供我们其他人认为理所当然的方向和导航信息以及空间线索。该项目的目标是通过利用地理信息系统(地理信息系统)数据库开发和维护、实时跟踪以及心理声学和音频演示等领域的技术和开发专门知识来实现这一目标。PI和他的团队将开发一种无缝的、空间化的音频呈现系统，通过该系统，人们可以获得在熟悉和陌生的室外和室内环境中成功和安全移动所需的额外方位提示和导航信息。为此，将为视障行人开发一个地理信息系统数据库，以确定可到达的步行路径，并提供路线规划的手段。该数据库将用系统用户经常使用的路径补充传统的调查地图数据，包括与这些路径有关的信息注释，例如路径沿线容易识别的地标和要避开的危险障碍物。要实现有效的演示，需要实时跟踪人的“头部姿势”(头部在空间中的位置和方向)。这将通过将来自GPS和惯性传感的数据与高频头部姿态数据相结合来实现。将开发新的概率定位算法，以利用基于地图的先验和用户意图的历史建模来提高室外GPS的位置精度。可视化三维重建将用于创建室内和室外数据库，以支持这一目的，并消除对室内传感基础设施的需求。基于这个3D数据库，将开发一种新型的基于视觉的定位传感器来提供头部姿势。在开发音频演示系统时，将研究非语音声音的使用，以此作为语音的替代方案，以便在直接的环境空间意识和定位比关于该环境的特定语义信息更令人关注的情况下使用。显然，当手头的任务是有效和安全地通过环境到达特定目的地时，被告知每一个门口和经过的物体的名称更多的是分散注意力，而不是帮助。然而，了解沿途门口和物体的存在和相对位置是监测进展和保持空间方向的一种有用的手段。PI的解决方案是开发类似于国际标牌的通用特征声音，这些声音很容易识别并与特定类别的物体相关联(例如，为入口关闭门的声音)。为了避免遮盖耳朵，将使用骨传导耳机，并开发与骨相关的传输功能，以呈现有效的声音立体成像。将进行实验，以确定哪些声音最直观、最容易定位，哪些种类和数量的不同声音可以很容易地学习，以及如何产生这些声音，以便它们不会分散人们对重要的自然声音提示的注意力。无缝运行的系统集成将通过开发一个总体操作系统来实现，该操作系统将前两个系统中的每一个系统作为模块，必须从该模块中融合数据以供第三个系统显示。该操作系统还将响应通过用户输入发出的控制命令，从而可以选择所呈现的信息的类型、数量、细节和半径。广泛的影响：与听觉呈现的感知和认知解释有关的基本问题将在本研究过程中得到解答；这些结果将适用于更广泛的音频显示技术领域。此外，还将开发快速和实时有效地展示各种空间信息的方法；这一知识将适用于必须通过非视觉渠道迅速吸收大量这类信息的任何领域。总体而言，这项研究将带来更有效的听觉呈现和信息传递系统，不仅可以供视障人士使用，而且可以随时随地需要及时的态势感知，从更好的空中交通管制员系统到在复杂情况下更快地吸收感知。事实证明，这种系统对于军事人员、消防员、喷气式飞机飞行员、急救人员等来说是非常宝贵的，对于任何需要以增强而不是阻碍其他感官过程的方式获取关于他或她的周围环境的更多信息的人来说，都是非常宝贵的。

For the millions of visually impaired people in the United States, over a million of whom have no usable vision, spatial orientation and navigation are a major problem leading to loss of mobility and reduced participation in community activities, not to mention serious safety concerns for those who do attempt to be fully mobile. There is thus a critical need for an aid that would provide the orientation and navigation information and spatial cues the rest of us take for granted. The goal of this project is to achieve this objective by bringing to bear technologies and development expertise in the fields of geographic information system (GIS) database development and maintenance, real-time tracking, and psycho-acoustics and audio presentation. The PI and his team will develop a seamless, spatialized audio presentation system with which a person can obtain the additional orientation cues and navigation information needed to move about successfully and safely in familiar and unfamiliar outdoor and indoor environments. To these ends, a GIS database for visually impaired pedestrians will be developed to define accessible walking paths and provide a means of route planning. This database will complement conventional, surveyed map data with paths frequented by users of the system, including annotations of information relevant to these paths such as easily recognized landmarks along the path and hazardous obstacles to avoid. Achieving an effective presentation will require real-time tracking of the person's "head pose" (location and orientation of the head in space). This will be accomplished by integrating data from GPS and inertial sensing with high frequency head pose data. Novel probabilistic localization algorithms will be developed to improve the positional accuracy of outdoor GPS using map-based priors and historical modeling of user intent. Visual 3D reconstruction will be used to create indoor and outdoor databases to support this purpose and obviate the need for an indoor sensing infrastructure. A novel vision-based localization sensor will be developed to provide head pose based on this 3D database. In developing an audio presentation system the use of non-speech sounds will be investigated as an alternative to speech for situations where immediate environmental spatial awareness and orientation is more of a concern than particular semantic information about that environment. Clearly, when the task at hand is to move through the environment effectively and safely to a particular destination, being told the name of every doorway and object being passed is more of a distraction than an aid. However, an awareness of the presence and relative location of doorways and objects along the way is useful as a means of monitoring progress and maintaining spatial orientation. The PI's solution is to develop generic characteristic sounds, analogous to international signage, which are easily recognized and associated with particular classes of objects (e.g., the sound of a closing door for an entrance). To avoid covering the ears, bone conduction headsets will be employed, and a bone-related transfer function developed to present effective stereo imaging of sounds. Experiments will be conducted to determine what sounds are most intuitive and easily localized, what variety and number of different sounds can be easily learned, and how to produce such sounds so they do not distract attention from important natural sound cues. System integration for seamless operation will be accomplished by developing an overarching operating system that treats each of the first two systems as modules from which data must be fused for presentation by the third system. This operating system will also respond to control commands made through user input, whereby selections may be made regarding the type, amount, detail and radius of information being presented.Broader Impacts: Fundamental questions relating to perception and the cognitive interpretation of auditory presentations will be answered in the course of this research; these results will be applicable to a much broader field of audio display technologies. In addition, methods for effectively and efficiently presenting a wide range of spatial information quickly and in real time will be developed; this knowledge will be applicable to any field where large amounts of such information must be assimilated quickly though non-visual channels. Overall, this research will lead to more effective auditory presentation and information delivery systems that can be used not only by people who are visually impaired, but wherever and whenever timely situational awareness is needed, from better systems for air traffic controllers to faster assimilation of awareness in complex situations. Such systems could prove invaluable to military personnel, firefighters, jet pilots, first responders, etc., and in general to anyone who needs to acquire more information about his or her surroundings in a manner that augments rather than impedes other sensory processes.