HCC: Small: STAAR: Spatial Touch Audio Annotator and Reader for Individuals with Blindness or Severe Visual Impairment

PI在这项研究中的目标是在最先进的平台（苹果iPad）上开发工具，使盲人或患有严重视觉障碍的人（IBSVI）能够访问文本信息。PI的方法是使用压花屏幕覆盖，为大声朗读的文本提供空间和触觉关联，并利用目标人群的空间认知和记忆资源，以便在文档中导航并根据需要对其进行注释。PI认为，从印刷媒体发明以来，信息已经被制定和优化，以供具有主导视觉能力的生物（人）消费。在IBSVI社区获取和传递信息的背景下，这种视觉空间偏差尚未得到很好的理解或研究；大多数技术信息辅助工具将信息作为顺序的听觉流输送给他们，从而避免了使用更广泛的空间认知资源。在这个项目中，PI将开发一个空间触摸音频注释器和阅读器（STAAR）测试平台，以探索一种多模式替代方案，使用户能够通过在板状设备上的触摸位置和文本到语音的音频渲染来融合空间布局和信息内容。STAAR将通过iPad表面的触觉覆盖图案实现自定阅读速度，该图案将提供触觉地标，以帮助用户浏览“页面”。STAAR将渲染触摸到的文本块。使用触摸手势来实现情境化高亮显示和笔记记录也将被研究。PI将研究目标人群在阅读和一段时间间隔后的回忆中如何使用空间策略和探索来重新发现和重新获取信息。更广泛的影响：以苹果iPad为代表的新一代平板设备可能会扩大IBSVI社区与大多数人口之间的可访问性差距。通过支持使用空间认知和记忆资源进行阅读和情境化注释，PI希望改善这种地方性的参与障碍。项目成果将有助于我们理解空间在信息设计和表达中的作用，以及获取这些信息所需的空间认知和记忆资源。他们将通过STAAR系统为移动计算领域做出贡献，特别是为IBSVI社区，但最终是为一般人，STAAR系统将被设计和实现，作为一个便携式设备，具有最先进的交互形式因素。在STAAR中，触觉和语言的多模态融合将对诸如导航辅助设备和为盲人和视力正常的人提供服务的系统的设计产生影响。此外，该项目将为本科生和研究生提供独特的跨学科教育和学习机会。在这个项目中开发的所有软件将放在开源。

The PI's goal in this research is to develop tools on a state-of-the-art platform (the Apple iPad) that will afford access to textual information for individuals who are blind or who suffer from severe visual impairments (IBSVI). The PI's approach is to use an embossed screen overlay to provide spatial and tactile correlates to text read aloud, and to engage the spatial cognition and memory resources of the target population for navigating through a document and annotating it if/as desired. The PI argues that from the invention of print media forward, information has been formulated and optimized for consumption by beings (people) with a dominant visual capability. This visuo-spatial bias is not well-understood or studied in the context of information access by and delivery to the IBSVI community; most technological information aids funnel information to them as sequential aural streams, obviating the use of broader spatial cognitive resources. In this project the PI will develop a Spatial Touch Audio Annotator and Reader (STAAR) testbed to explore a multimodal alternative that enables the user to fuse spatial layout and informational content through touch location on a slate-type device and audio rendering of text to speech, respectively. STAAR will enable self-paced reading using a tactile overlay pattern on an iPad surface, which will be designed to provide tactile landmarks to help the user navigate the "page." STAAR will render the text chunk touched audibly. The use of touch gestures to enable contextualized highlighting and note-taking will also be investigated. The PI will study how the target population may employ spatial strategies and exploration to re-find and re-access information both in the act of reading and for recall after some time interval. Broader Impacts: A new generation of slate-type devices exemplified by the Apple iPad threatens to widen the accessibility gap between the IBSVI community and the majority of the population. By supporting the use of spatial cognitive and memory resource for both reading and contextualized annotation, the PI hopes to ameliorate this endemic barrier to participation. Project outcomes will contribute to our understanding of the role of space in information design and representation, and the spatial cognition and memory resources needed for uptake of such information. And they will contribute to the domain of mobile computing, for the IBSVI community in particular but ultimately for the population in general, through the STAAR system, which will be designed and implemented from the ground up as a portable device on a state-of-art interaction form-factor. The multimodal fusion of haptics and speech in STAAR will have implications for the designs of such things as navigational aids and service delivery systems for the non-sighted as well as the sighted. The project will in addition provide unique cross-disciplinary educational and learning opportunities for undergraduate and graduate students. All software developed in this project will be placed in open source.