CAREER: A New Interaction Model for Eyes-Free Exploration of Touch Screens

职业：一种用于裸眼探索触摸屏的新交互模型

• 批准号: 1652907
• 负责人: Shaun Kane
• 金额: $ 55万
• 依托单位: University of Colorado at Boulder
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-03-01 至 2023-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1652907&HistoricalAwards=false
• 关键词: CAREER; New; Interaction; Model; Eyes

Touch-based user interfaces, which have become ubiquitous in personal computers and mobile devices, typically offer only limited accessibility to blind and visually impaired users because they rely heavily upon visual interaction and provide little if any audio or haptic feedback. This creates an accessibility barrier that affects millions of people in the United States. The problem is compounded by the interaction model supported by most touch interfaces, which assumes that a blind user will explore via a single finger and interact with only one on-screen item at a time; as a consequence, blind users cannot easily browse or skim through large amounts of information or interact with on-screen content using gestures. These problems are further exacerbated by a lack of coordination between the touch input framework and the speech output engine. This research will substantially improve the accessibility and usability of touch-based interfaces for blind users through the creation of a new model that supports robust eyes-free interaction with touch screen applications and tangible user interfaces, which will significantly improve touch exploration of complex data by blind users and also by sighted people (e.g. to avoid distractions while driving). Students with disabilities will be recruited as members of the research team. Additional broad impact will be achieved through educational activities that engage computing students in the design of accessible user interfaces; to this end, modules on universal design of eyes-free user interfaces for courses on user-centered design and tangible computing will be created. Outreach activities will include a series of inclusive design workshops in which persons with disabilities will be invited to participate.The research will proceed in several stages. First, to identify benchmarks and metrics for natural eyes-free exploration, formative studies will be conducted with blind individuals and teachers of the visually impaired. A new interaction model that supports eyes-free exploration of, and interaction with, touch screen user interfaces will then be developed that supports the hand poses, movements, gestures and navigation strategies preferred by blind users, and will be implemented as an input framework that can be embedded in existing touch-based applications. As part of this work, new gesture recognizers and a new text-to-speech engine will be developed and evaluated. Finally, authoring tools that enable scientists, artists, teachers, and others to adapt their existing data to support non-visual touch interaction with both touch screens and 3D-printed tactile graphics will be developed through iterative testing with blind and sighted users, and their teachers, and through the public deployment of reference applications including an interactive map for learning public transit routes in a large urban area and a tool for exploring interactive museum exhibits.

在个人计算机和移动的设备中已经变得普遍存在的基于触摸的用户界面通常仅向盲人和视力受损的用户提供有限的可访问性，因为它们严重依赖于视觉交互并且提供很少（如果有的话）的音频或触觉反馈。 这造成了一个影响美国数百万人的可访问性障碍。 大多数触摸界面支持的交互模型使问题变得更加复杂，该模型假设盲人用户将通过单个手指探索并一次仅与屏幕上的一个项目交互;因此，盲人用户无法轻松浏览或浏览大量信息或使用手势与屏幕上的内容进行交互。 这些问题由于触摸输入框架和语音输出引擎之间缺乏协调而进一步恶化。 这项研究将通过创建一个新的模型，支持与触摸屏应用程序和有形用户界面的强大的裸眼交互，大大提高盲人用户基于触摸的界面的可访问性和可用性，这将显着改善盲人用户和视力正常的人对复杂数据的触摸探索（例如，避免驾驶时分心）。 残疾学生将被招募为研究团队的成员。 将通过教育活动，使计算机专业的学生参与无障碍用户界面的设计，实现更广泛的影响;为此，将为以用户为中心的设计和有形计算课程创建关于裸眼用户界面通用设计的模块。 外联活动将包括一系列包容性设计讲习班，邀请残疾人参加，研究将分几个阶段进行。 首先，为确定无眼自然探索的基准和指标，将与盲人和视障教师进行形成性研究。 一个新的交互模型，支持眼睛自由的探索，并与触摸屏用户界面的交互，然后将开发支持盲人用户首选的手部姿势，动作，手势和导航策略，并将被实现为一个输入框架，可以嵌入到现有的触摸应用程序。 作为这项工作的一部分，将开发和评估新的手势识别器和新的文本到语音引擎。 最后，通过对盲人和视力正常的用户及其教师进行迭代测试，将开发出使科学家、艺术家、教师和其他人能够调整其现有数据以支持与触摸屏和3D打印触觉图形进行非视觉触摸交互的创作工具。并通过公共部署参考应用程序，包括用于了解大城市地区公共交通路线的交互式地图和探索互动博物馆展品的工具。

项目成果

Sense and Accessibility: Understanding People with Physical Disabilities’ Experiences with Sensing Systems

感知与可达性：了解身体残疾人士的感知系统体验

• DOI: 10.1145/3373625.3416990
• 发表时间: 2020
• 期刊: Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '20
• 作者: Kane, Shaun K.;Guo, Anhong;Morris, Meredith Ringel
• 通讯作者: Morris, Meredith Ringel

The Haptic Video Player: Using Mobile Robots to Create Tangible Video Annotations

触觉视频播放器：使用移动机器人创建有形视频注释

• DOI: 10.1145/3279778.3279805
• 发表时间: 2018
• 期刊: Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces (ISS 2018
• 作者: Guinness, Darren;Muehlbradt, Annika;Szafir, Daniel;Kane, Shaun K.
• 通讯作者: Kane, Shaun K.

What's in an ALT Tag? Exploring Caption Content Priorities through Collaborative Captioning

ALT 标签中有什么内容？

• DOI: 10.1145/3507659
• 发表时间: 2022
• 期刊: ACM Transactions on Accessible Computing
• 影响因子: 2.4
• 作者: Muehlbradt, Annika;Kane, Shaun K.
• 通讯作者: Kane, Shaun K.

BrailleBlocks: Computational Braille Toys for Collaborative Learning

BrailleBlocks：用于协作学习的计算盲文玩具

• DOI: 10.1145/3313831.3376295
• 发表时间: 2020
• 期刊: CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
• 作者: Gadiraju, Vinitha;Muehlbradt, Annika;Kane, Shaun K.
• 通讯作者: Kane, Shaun K.

EyeDescribe: Combining Eye Gaze and Speech to Automatically Create Accessible Touch Screen Artwork

EyeDescribe：结合眼睛注视和语音自动创建可访问的触摸屏艺术品

• DOI: 10.1145/3343055.3359722
• 发表时间: 2019
• 期刊: ISS '19: Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces
• 作者: Reinholt, Kyle;Guinness, Darren;Kane, Shaun K.
• 通讯作者: Kane, Shaun K.