Augmented reality powers a cognitive assistant for the blind.
Augmented reality powers a cognitive assistant for the blind.
复制标题
DOI：
10.7554/elife.37841
复制
发表时间：
2018-11-27
期刊：
影响因子：
7.7
通讯作者：
Meister M
中科院分区：
文献类型：
--
作者：
Liu Y;Stiles NR;Meister M
为了恢复盲人的视力，已经探索了几种将原始图像传递到大脑的修复方法。到目前为止，这些方案都存在带宽不足的问题。另一种方法是在认知水平上恢复视力，绕过传递感官数据的需要。可穿戴计算机捕获视频和其他数据，提取重要的场景知识，并以紧凑的形式传达给用户。在这里，我们使用增强现实为这样的设备实现了一个直观的用户界面：环境中的每个对象都有一个声音，并根据命令与用户进行通信。通过最少的训练，该系统支持视觉认知的许多方面：避障，场景理解，空间记忆的形成和回忆，导航。盲人受试者可以在第一次尝试时穿过不熟悉的多层建筑。为了促进这一领域的进一步发展，我们开发了一个开源环境，用于视觉辅助设备的标准化基准测试。
To restore vision for the blind, several prosthetic approaches have been explored that convey raw images to the brain. So far, these schemes all suffer from a lack of bandwidth. An alternate approach would restore vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer captures video and other data, extracts important scene knowledge, and conveys that to the user in compact form. Here, we implement an intuitive user interface for such a device using augmented reality: each object in the environment has a voice and communicates with the user on command. With minimal training, this system supports many aspects of visual cognition: obstacle avoidance, scene understanding, formation and recall of spatial memories, navigation. Blind subjects can traverse an unfamiliar multi-story building on their first attempt. To spur further development in this domain, we developed an open-source environment for standardized benchmarking of visual assistive devices.