Point and Listen: Augmented Reality Interfaces for the Visually Impaired

“点和听：视觉障碍者的增强现实界面” 项目摘要 这是一个行政补充，以加强开放科学应用软件工具的父母 R 01格兰特，“点和听：增强现实接口的视觉障碍。”家长的关注点 grant是CamIO（“Camera Input-Output”）项目，旨在使物理对象，包括文档， 地图和3D对象，如建筑模型和设备，完全可供盲人或 视力受损者（英属维尔京群岛）。它通过基于音频的增强现实技术使这些对象变得可访问 (AR)界面：用户指向感兴趣的对象上的特定位置，并听到有关此的音频标签 位置大声朗读。 在父母补助金方面取得了实质性进展，包括几乎完成了永久性安装 CamIO在帕洛阿尔托的魔法桥游乐场制作了一张游乐场的3D青铜地图， BVI访客可使用。然而，该项目将受益于更广泛地采用和试验， CamIO的AR方法由无障碍社区的其他研究人员和开发人员提供。为此我们 提出一个补充项目，以增强CamIO软件，以降低研究人员的障碍， 将该软件纳入自己的项目中或对其进行试验。具体而言，我们计划进行两项增强， CamIO软件：（a）重组软件，目标是使其他人更容易纳入 他们自己的可访问性项目;以及（B）创建SKIstream，一个快速原型和调试工具，用于 CamIO和类似的无障碍项目。 通过对CamIO软件的这些改进，我们设想博物馆、医疗机构、 设施和大学开发自己的CamIO自定义版本，以制作3D地图和其他模型 完全无障碍。我们注意到，CamIO不需要物理修改就可以访问模型，前提是 相机可以查看用户将与之交互的模型表面;这意味着几乎任何3D模型都可以 可以被访问。其他研究人员可以尝试修改CamIO接口， 进行用户研究，以评估这些修改的可用性和有效性。最后，由于 增强的CamIO软件和SKIstream工具将作为开源代码免费共享，其他研究人员 可以为代码贡献自己的增强功能，包括改进和全新的功能。

“Point and Listen: Augmented Reality Interfaces for the Visually Impaired” Project Summary This is an Administrative Supplement to Enhance Software Tools for Open Science application for the parent R01 grant, “Point and Listen: Augmented Reality Interfaces for the Visually Impaired.” The focus of the parent grant is the CamIO (“Camera Input-Output”) project, intended to make physical objects, including documents, maps and 3D objects such as architectural models and appliances, fully accessible to people who are blind or visually impaired (BVI). It makes these objects accessible by means of an audio-based Augmented Reality (AR) interface: the user points to a specific location on an object of interest and hears an audio label about this location read aloud. Substantial progress has been made on the parent grant, including a nearly completed permanent installation of CamIO at the Magical Bridge Playground in Palo Alto to make a 3D bronze map of the playground fully accessible to BVI visitors. However, the project would benefit from wider adoption of, and experimentation with, CamIO’s AR approach by other researchers and developers in the accessibility community. To this end, we propose a Supplement project to enhance the CamIO software to lower the barriers to researchers who wish to incorporate the software in their own projects or experiment with it. Specifically, we plan two enhancements to the CamIO software: (a) restructure the software with the goal of making it easier for others to incorporate in their own accessibility projects; and (b) create SKIstream, a rapid prototyping and debugging tool for use with CamIO and similar accessibility projects. With these enhancements to the CamIO software, we envisage institutions such as museums, medical facilities and universities developing their own custom versions of CamIO to make 3D maps and other models fully accessible. We note that CamIO requires no physical modifications to make a model accessible, provided a camera can view the surface of the model that users will interact with; this means that virtually any 3D model can be made accessible. Other researchers can experiment with modifications to the CamIO interface and conduct user studies to evaluate the usability and effectiveness of these modifications. Finally, since the enhanced CamIO software and SKIstream tool will be shared freely as open source code, other researchers can contribute their own enhancements to the code, including improvements and entirely new functionality.