Environmental Localization Mapping and Guidance for Visual Prosthesis Users

项目摘要 大约有130万年龄在40岁及以上的美国人在法律上是盲人，其中大多数是因为疾病 老年性黄斑变性的发病年龄较晚，如青光眼和老年性黄斑变性。Second Sight 开发了世界上第一个FDA批准的视网膜植入物，Argus II，旨在恢复一些 视网膜色素变性（RP）患者的功能性视力。 在这个智能设备的时代，通用导航技术，如GPS地图应用程序， 智能手机可以提供方向以帮助引导盲人用户从点A到点B。然而，在这方面， 这些导航辅助工具几乎不能使盲人用户形成对 周围环境，不适合在室内导航，并且不做任何有助于避免 流动性的障碍。另一方面，Argus II为盲人用户提供有限的视觉 其周围环境的表示，提高了用户定位自己的能力， 穿越障碍，但缺乏高层次导航和语义解释的功能， 四周拟议的研究旨在通过以下方式解决Argus II的这些局限性： 最先进的刺激定位和映射（SLAM）与对象的协同作用 识别技术。在过去的三年里，JHU/APL与第二个 为Argus II开发类似的先进视觉功能，包括 通过立体视觉进行物体识别和障碍物检测的能力。 该建议是由假设驱动的，即视网膜假体的用户的导航可以是 通过结合SLAM和对象识别技术， 环境信息通过视网膜假体和听觉反馈。SLAM使 视觉假体系统，用于构建用户环境的地图并在其中定位用户 那张地图然后，该系统通过适当的通信提供对象位置和导航提示。 使用户能够在精神上形成环境的自我中心地图的感觉模态。 我们建议开发和测试一个视觉假体系统，1）构建一个地图， 不熟悉的环境，并使用SLAM技术2）自动定位用户 使用对象识别来识别导航相关的对象和地标，以及3）提供 用于导航、避障和物体/地标识别的感觉反馈。

Project Summary About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight has developed the world's first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP). In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. The Argus II, on the other hand, provides blind users with a limited visual representation of their surroundings that improves users' ability to orient themselves and traverse obstacles, yet lacks features for high-level navigation and semantic interpretation of the surroundings. The proposed research aims to address these limitations of the Argus II through a synergy of state-of-the-art stimultaneous localization and mapping (SLAM) and object recognition technologies. For the past three years, JHU/APL has collaborated with Second Sight to develop similar advanced vision-based capabilities for the Argus II, including capabilities for object recognition and obstacle detection by stereo vision. This proposal is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating SLAM and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. SLAM enables the visual prosthesis system to construct a map of the user's environment and locate the user within that map. The system then provides object location and navigational cues via appropriate sensory modalities enabling the user to mentally form an egocentric map of the environment. We propose to develop and test a visual prosthesis system which 1) constructs a map of unfamiliar environments and localizes the user using SLAM technology 2) automatically identifies navigationally-relevant objects and landmarks using object recognition and 3) provides sensory feedback for navigation, obstacle avoidance, and object/landmark identification.