Environmental Localization Mapping and Guidance for Visual Prosthesis Users

项目摘要 约有130万年龄在40岁及以上的美国人在法律上失明，其中大部分是因为疾病 发病年龄较晚，如青光眼和老年性黄斑变性。第二眼 已经开发出世界上第一个FDA批准的视网膜植入物Argus II，旨在修复一些 视网膜色素变性(RP)患者的功能性视力。 在这个智能设备时代，通用导航技术，如GPS地图应用程序，用于 智能手机可以提供方向帮助引导盲人用户从A点到B点。然而， 这些导航辅助工具几乎无法使盲人用户形成以自我为中心的理解 周围环境，不适合在室内航行，也不会帮助避免 机动性障碍。另一方面，Argus II为盲人用户提供了有限的视觉 他们周围环境的表现，提高了用户自我定位和 穿越障碍，但缺乏高级导航和语义解释的功能 周围的环境。拟议的研究旨在通过一个 最先进的刺激定位和测绘(SLAM)与对象的协同 识别技术。在过去的三年里，JHU/APL与Second SIGH为Argus II开发类似的基于视觉的高级功能，包括 通过立体视觉进行目标识别和障碍物检测。 这一提议是由一种假设推动的，即视网膜假体用户的导航可以 将SLAM和对象识别技术结合在一起，大大改进了传输 通过视网膜假体和听觉反馈获得环境信息。SLAM使 视觉假体系统，用于构建用户环境的地图并在其中定位用户 那张地图。然后，系统通过适当的方式提供对象位置和导航提示 使用户能够在精神上形成以自我为中心的环境地图的感觉形式。 我们计划开发和测试一种视觉假体系统，它1)构建了一张地图 不熟悉的环境，并使用SLAM技术自动本地化用户2) 使用对象识别识别与导航相关的对象和地标，并3)提供 用于导航、避障和物体/地标识别的感官反馈。

Project Summary About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight has developed the world's first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP). In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. The Argus II, on the other hand, provides blind users with a limited visual representation of their surroundings that improves users' ability to orient themselves and traverse obstacles, yet lacks features for high-level navigation and semantic interpretation of the surroundings. The proposed research aims to address these limitations of the Argus II through a synergy of state-of-the-art stimultaneous localization and mapping (SLAM) and object recognition technologies. For the past three years, JHU/APL has collaborated with Second Sight to develop similar advanced vision-based capabilities for the Argus II, including capabilities for object recognition and obstacle detection by stereo vision. This proposal is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating SLAM and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. SLAM enables the visual prosthesis system to construct a map of the user's environment and locate the user within that map. The system then provides object location and navigational cues via appropriate sensory modalities enabling the user to mentally form an egocentric map of the environment. We propose to develop and test a visual prosthesis system which 1) constructs a map of unfamiliar environments and localizes the user using SLAM technology 2) automatically identifies navigationally-relevant objects and landmarks using object recognition and 3) provides sensory feedback for navigation, obstacle avoidance, and object/landmark identification.