Point and Listen: Augmented Reality Interfaces for the Visually Impaired

指向并聆听：为视障人士提供的增强现实界面

• 批准号: 10540115
• 负责人: JAMES M COUGHLAN
• 金额: $ 42.94万
• 依托单位: SMITH-KETTLEWELL EYE RESEARCH INSTITUTE
• 依托单位国家: 美国
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-03-10 至 2026-02-28
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10540115
• 关键词: 3-Dimensional; American; Architecture; Augmented Reality; Blindness; Cellular Phone; Computer software; Development; Devices; Documentation; Economics; Employment; Ensure; Evaluation; Fingers; Goals; Grant; Health; Hearing; Home; Image Enhancement; Label; Location; Maps; Modeling; Output; Performance; Persons; Reader; Research; Research Personnel; Schools; Science, Technology, Engineering and Mathematics; System; Tactile; Touch sensation; Vision; Visual; Visual impairment; Work; Workplace; base; blind; braille; design; devices for the visually impaired; experience; haptics; improved; interest; open source; prototype; smartphone Application; three-dimensional modeling

Point and Listen: Augmented Reality Interfaces for the Visually Impaired Summary CamIO (“Camera Input-Output”) is a project designed to make physical objects, including documents, maps and 3D objects such as architectural models, fully accessible to people who are blind or visually impaired. It makes these objects accessible by means of an audio-based Augmented Reality (AR) interface: the user points to a specific location on an object of interest and hears audio information about this location read aloud (or sees an enhanced image of the selected location if the user has low vision). Compared with other approaches to making objects accessible to people who are blind or visually impaired, CamIO offers several advantages: (a) there is no need to modify or augment existing objects (e.g., with braille labels or special touch-sensitive buttons), requiring only a smartphone and optional wearable camera device; (b) CamIO is accessible even to those who are not fluent braille readers; and (c) it permits natural exploration of the object with all fingers to enable audio-haptic interactions with the object. Building on our past work on the prototype CamIO iPhone app, we plan to develop powerful enhancements and new functionality, including finger tracking as an alternative to the use of a pointing stylus, and support for wearable options such as the camera-enabled Envision Glasses that will offer a far superior user interface to what is possible with a stand-alone smartphone app. New authoring capabilities will make it easy for blind and visually impaired users and sighted assistants to create audio labels for objects. We will also pursue two exciting applications of CamIO in depth: using CamIO to make tactile maps fully interactive and easier to understand, and the development of interactive installations such as a 3D scale model of a large playground that helps visually impaired playground visitors familiarize themselves with the playground’s layout and features. Finally, people who are blind or visually impaired – the target users of the CamIO system – will be involved in all aspects of the proposed research, to maximize the impact of the research effort.

指向并聆听：视觉增强现实界面 受损 概括 CamIO（“相机输入输出”）是一个旨在制作物理对象的项目，包括文档、 地图和 3D 对象（例如建筑模型），盲人或视觉障碍人士完全可以访问 受损。它使这些对象可以通过基于音频的增强现实 (AR) 来访问 界面：用户指向感兴趣的对象上的特定位置并听到音频信息 大声朗读有关此位置的信息（或者如果用户的记忆力较低，则可以看到所选位置的增强图像） 想象）。与让盲人或盲人能够接触到物体的其他方法相比 对于视障人士来说，CamIO 具有以下几个优点：(a) 无需修改或增强现有的 物体（例如，带有盲文标签或特殊的触摸感应按钮），只需要智能手机和 可选配可穿戴摄像设备； (b) 即使盲文不流利的人也可以使用 CamIO 读者； (c) 它允许用所有手指自然地探索物体，以实现音频-触觉 与物体的相互作用。 基于我们过去在 CamIO iPhone 应用程序原型上的工作，我们计划开发功能强大的 增强功能和新功能，包括手指跟踪作为使用指向的替代方法 手写笔，并支持可穿戴选项，例如支持摄像头的 Envision 眼镜，该眼镜将提供 远优于独立智能手机应用程序的用户界面。新创作 功能将使盲人和视障用户以及视力正常的助手可以轻松创建音频 对象的标签。我们还将深入研究 CamIO 的两个令人兴奋的应用：使用 CamIO 使触觉地图具有充分的交互性和更容易理解，以及交互性的发展 装置，例如大型游乐场的 3D 比例模型，可帮助视障游乐场 游客熟悉游乐场的布局和特色。最后，盲人 或视障人士——CamIO 系统的目标用户——将参与该系统的各个方面 提出的研究，以最大限度地发挥研究工作的影响。