Point and Listen: Augmented Reality Interfaces for the Visually Impaired

点击翻译按钮获取中文摘要

“Point and Listen: Augmented Reality Interfaces for the Visually Impaired” Project Summary This is an Administrative Supplement to Enhance Software Tools for Open Science application for the parent R01 grant, “Point and Listen: Augmented Reality Interfaces for the Visually Impaired.” The focus of the parent grant is the CamIO (“Camera Input-Output”) project, intended to make physical objects, including documents, maps and 3D objects such as architectural models and appliances, fully accessible to people who are blind or visually impaired (BVI). It makes these objects accessible by means of an audio-based Augmented Reality (AR) interface: the user points to a specific location on an object of interest and hears an audio label about this location read aloud. Substantial progress has been made on the parent grant, including a nearly completed permanent installation of CamIO at the Magical Bridge Playground in Palo Alto to make a 3D bronze map of the playground fully accessible to BVI visitors. However, the project would benefit from wider adoption of, and experimentation with, CamIO’s AR approach by other researchers and developers in the accessibility community. To this end, we propose a Supplement project to enhance the CamIO software to lower the barriers to researchers who wish to incorporate the software in their own projects or experiment with it. Specifically, we plan two enhancements to the CamIO software: (a) restructure the software with the goal of making it easier for others to incorporate in their own accessibility projects; and (b) create SKIstream, a rapid prototyping and debugging tool for use with CamIO and similar accessibility projects. With these enhancements to the CamIO software, we envisage institutions such as museums, medical facilities and universities developing their own custom versions of CamIO to make 3D maps and other models fully accessible. We note that CamIO requires no physical modifications to make a model accessible, provided a camera can view the surface of the model that users will interact with; this means that virtually any 3D model can be made accessible. Other researchers can experiment with modifications to the CamIO interface and conduct user studies to evaluate the usability and effectiveness of these modifications. Finally, since the enhanced CamIO software and SKIstream tool will be shared freely as open source code, other researchers can contribute their own enhancements to the code, including improvements and entirely new functionality.