Environmental Localization Mapping and Guidance for Visual Prosthesis Users

视觉假体使用者的环境定位测绘和指导 项目摘要 约有130万年龄在40岁及以上的美国人在法律上失明，其中大部分是因为疾病 发病年龄较晚，如青光眼和老年性黄斑变性。第二眼 开发了世界上第一个FDA批准的视网膜植入物，Argus II，旨在修复一些 视网膜色素变性(RP)患者的功能性视力。 在这个智能设备时代，通用导航技术，如GPS地图应用程序，用于 智能手机可以提供方向帮助引导盲人用户从A点到B点。然而， 这些导航辅助工具几乎无法使盲人用户形成以自我为中心的理解 周围环境，不适合在室内航行，也不能帮助避开障碍物 为流动性干杯。另一方面，像Argus II这样的假眼设备为盲人用户提供了 对其周围环境的有限视觉表示，提高了用户的自我定位能力 和穿越障碍，但缺乏高级导航和语义解释的功能 周围的环境。拟议的研究旨在通过以下方面的协同作用来解决这些限制 最先进的同时定位和测绘(SLAM)和目标识别 技术。 这一提议是由一种假设推动的，即视网膜假体用户的导航可以 将SLAM和对象识别技术结合在一起，大大改进了传输 通过视网膜假体和听觉反馈获得环境信息。SLAM使 视觉假体系统，用于构建用户环境的地图并在其中定位用户 那张地图。然后，该系统通过适当的感官提供物体位置和导航提示 使用户能够在精神上形成以自我为中心的环境地图的形态。我们有 开发了一种先进的视觉假体系统，它1)构建了一张陌生的地图 使用SLAM技术的环境和本地化用户2)自动识别 环境的结构语义，以确定与导航相关的障碍和 功能和3)提供导航、避障和感官反馈 物体/地标识别。该项目的重点是完成人体受试者测试 这些能力。

Environmental Localization Mapping and Guidance for Visual Prosthesis Users Project Summary About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight developed the world’s first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP). In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. Prosthetic vision devices like the Argus II, on the other hand, provides blind users with a limited visual representation of their surroundings that improve users’ ability to orient themselves and traverse obstacles, yet lack features for high-level navigation and semantic interpretation of the surroundings. The proposed research aims to address these limitations through a synergy of state-of-the-art simultaneous localization and mapping (SLAM) and object recognition technologies. This proposal is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating SLAM and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. SLAM enables the visual prosthesis system to construct a map of the user’s environment and locate the user within that map. The system then provides object location and navigational cues via appropriate sensory modalities enabling the user to mentally form an egocentric map of the environment. We have developed an advanced visual prosthesis system which 1) constructs a map of unfamiliar environments and localizes the user using SLAM technology 2) automatically identifies the structural semantics of the surroundings to determine navigationally-relevant obstacles and features and 3) provides sensory feedback for navigation, obstacle avoidance, and object/landmark identification. This project focuses on the completion of human subject testing of these capabilities.