HCC: Small: STAAR: Spatial Touch Audio Annotator and Reader for Individuals with Blindness or Severe Visual Impairment

HCC：小型：STAAR：适用于失明或严重视力障碍人士的空间触摸音频注释器和阅读器

• 批准号: 1117854
• 负责人: Francis Quek
• 金额: $ 49.88万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2014-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1117854&HistoricalAwards=false
• 关键词: HCC; Small; STAAR; Spatial; Touch

The PI's goal in this research is to develop tools on a state-of-the-art platform (the Apple iPad) that will afford access to textual information for individuals who are blind or who suffer from severe visual impairments (IBSVI). The PI's approach is to use an embossed screen overlay to provide spatial and tactile correlates to text read aloud, and to engage the spatial cognition and memory resources of the target population for navigating through a document and annotating it if/as desired. The PI argues that from the invention of print media forward, information has been formulated and optimized for consumption by beings (people) with a dominant visual capability. This visuo-spatial bias is not well-understood or studied in the context of information access by and delivery to the IBSVI community; most technological information aids funnel information to them as sequential aural streams, obviating the use of broader spatial cognitive resources. In this project the PI will develop a Spatial Touch Audio Annotator and Reader (STAAR) testbed to explore a multimodal alternative that enables the user to fuse spatial layout and informational content through touch location on a slate-type device and audio rendering of text to speech, respectively. STAAR will enable self-paced reading using a tactile overlay pattern on an iPad surface, which will be designed to provide tactile landmarks to help the user navigate the "page." STAAR will render the text chunk touched audibly. The use of touch gestures to enable contextualized highlighting and note-taking will also be investigated. The PI will study how the target population may employ spatial strategies and exploration to re-find and re-access information both in the act of reading and for recall after some time interval. Broader Impacts: A new generation of slate-type devices exemplified by the Apple iPad threatens to widen the accessibility gap between the IBSVI community and the majority of the population. By supporting the use of spatial cognitive and memory resource for both reading and contextualized annotation, the PI hopes to ameliorate this endemic barrier to participation. Project outcomes will contribute to our understanding of the role of space in information design and representation, and the spatial cognition and memory resources needed for uptake of such information. And they will contribute to the domain of mobile computing, for the IBSVI community in particular but ultimately for the population in general, through the STAAR system, which will be designed and implemented from the ground up as a portable device on a state-of-art interaction form-factor. The multimodal fusion of haptics and speech in STAAR will have implications for the designs of such things as navigational aids and service delivery systems for the non-sighted as well as the sighted. The project will in addition provide unique cross-disciplinary educational and learning opportunities for undergraduate and graduate students. All software developed in this project will be placed in open source.

PI在这项研究中的目标是在最先进的平台（Apple iPad）上开发工具，为盲人或患有严重视觉障碍（IBSVI）的个人提供文本信息。PI的方法是使用浮雕屏幕覆盖来提供与朗读文本的空间和触觉相关性，并利用目标人群的空间认知和记忆资源来浏览文档并根据需要对其进行注释。 PI认为，从印刷媒体的发明开始，信息已经被制定和优化，以供具有主导视觉能力的生物（人）消费。 这种视觉空间偏见没有得到很好的理解或研究的背景下，信息访问和交付给IBSVI社区;大多数技术信息艾滋病漏斗信息给他们作为连续的听觉流，避免使用更广泛的空间认知资源。 在这个项目中，PI将开发一个空间触摸音频注释器和阅读器（STAAR）测试平台，以探索一种多模式的替代方案，使用户能够分别通过在平板型设备上的触摸位置和文本到语音的音频渲染来融合空间布局和信息内容。 STAAR将在iPad表面上使用触觉覆盖图案实现自定进度的阅读，该图案旨在提供触觉地标，以帮助用户导航“页面”。“STAAR将使触摸的文本块发出声音。 使用触摸手势，使上下文突出显示和笔记也将被调查。 PI将研究目标人群如何在阅读和一段时间后的回忆中使用空间策略和探索来重新发现和重新获取信息。 更广泛的影响：以苹果iPad为代表的新一代平板式设备有可能扩大IBSVI社区与大多数人口之间的可访问性差距。 通过支持使用空间认知和记忆资源进行阅读和语境化注释，PI希望改善这种参与的地方性障碍。 项目成果将有助于我们理解空间在信息设计和表达中的作用，以及吸收这些信息所需的空间认知和记忆资源。 他们将通过STAAR系统为移动的计算领域做出贡献，特别是为IBSVI社区，但最终为一般人群做出贡献，STAAR系统将作为一种便携式设备在最先进的交互形式因素上从头开始设计和实现。 STAAR中触觉和语音的多模态融合将对诸如为非盲人和视力正常者设计的导航辅助设备和服务提供系统等产生影响。 该项目还将为本科生和研究生提供独特的跨学科教育和学习机会。 在这个项目中开发的所有软件将被放置在开源。