CAREER: A New Interaction Model for Eyes-Free Exploration of Touch Screens

基于触摸的用户界面已经在个人电脑和移动设备中变得无处不在，通常只为盲人和视障用户提供有限的可访问性，因为它们严重依赖视觉交互，几乎不提供音频或触觉反馈。这造成了一个影响美国数百万人的无障碍。大多数触摸界面支持的交互模式使问题变得更加复杂，该模式假设盲人用户将通过一根手指进行探索，一次只与一个屏幕上的项目交互；因此，盲人用户无法轻松浏览或浏览大量信息，也无法使用手势与屏幕上的内容交互。触摸输入框架和语音输出引擎之间缺乏协调，进一步加剧了这些问题。这项研究将通过创建一种新的模型来支持与触摸屏应用程序和有形用户界面的强大无眼交互，从而显著改善盲人用户和有视力的人对复杂数据的触摸探索(例如，避免开车时分心)，从而大幅提高盲人用户和视力正常用户的触摸式界面的可访问性和可用性。残疾学生将被招募为研究小组的成员。通过让计算机专业学生参与设计无障碍用户界面的教育活动，将产生更广泛的影响；为此目的，将为以用户为中心的设计和有形计算课程创建无眼睛用户界面通用设计单元。外联活动将包括一系列包容性设计讲习班，将邀请残疾人参加。研究将分几个阶段进行。首先，为了确定自然无眼探索的基准和指标，将对盲人和视障人士的教师进行形成性研究。然后，将开发一种新的交互模型，支持对触摸屏用户界面的无眼探索和交互，支持盲人用户喜欢的手势、动作、手势和导航策略，并将作为输入框架实现，可以嵌入现有的基于触摸的应用程序中。作为这项工作的一部分，将开发和评估新的手势识别器和新的文本到语音转换引擎。最后，将开发创作工具，使科学家、艺术家、教师和其他人能够调整他们现有的数据，以支持与触摸屏和3D打印触觉图形的非视觉触摸交互，方法是对盲人和视力正常的用户及其教师进行迭代测试，并公开部署参考应用程序，包括用于学习大城市地区公共交通路线的交互式地图和用于探索交互式博物馆展品的工具。

Touch-based user interfaces, which have become ubiquitous in personal computers and mobile devices, typically offer only limited accessibility to blind and visually impaired users because they rely heavily upon visual interaction and provide little if any audio or haptic feedback. This creates an accessibility barrier that affects millions of people in the United States. The problem is compounded by the interaction model supported by most touch interfaces, which assumes that a blind user will explore via a single finger and interact with only one on-screen item at a time; as a consequence, blind users cannot easily browse or skim through large amounts of information or interact with on-screen content using gestures. These problems are further exacerbated by a lack of coordination between the touch input framework and the speech output engine. This research will substantially improve the accessibility and usability of touch-based interfaces for blind users through the creation of a new model that supports robust eyes-free interaction with touch screen applications and tangible user interfaces, which will significantly improve touch exploration of complex data by blind users and also by sighted people (e.g. to avoid distractions while driving). Students with disabilities will be recruited as members of the research team. Additional broad impact will be achieved through educational activities that engage computing students in the design of accessible user interfaces; to this end, modules on universal design of eyes-free user interfaces for courses on user-centered design and tangible computing will be created. Outreach activities will include a series of inclusive design workshops in which persons with disabilities will be invited to participate.The research will proceed in several stages. First, to identify benchmarks and metrics for natural eyes-free exploration, formative studies will be conducted with blind individuals and teachers of the visually impaired. A new interaction model that supports eyes-free exploration of, and interaction with, touch screen user interfaces will then be developed that supports the hand poses, movements, gestures and navigation strategies preferred by blind users, and will be implemented as an input framework that can be embedded in existing touch-based applications. As part of this work, new gesture recognizers and a new text-to-speech engine will be developed and evaluated. Finally, authoring tools that enable scientists, artists, teachers, and others to adapt their existing data to support non-visual touch interaction with both touch screens and 3D-printed tactile graphics will be developed through iterative testing with blind and sighted users, and their teachers, and through the public deployment of reference applications including an interactive map for learning public transit routes in a large urban area and a tool for exploring interactive museum exhibits.