Feel-IT: Audio-Tactile Computer Accessibility

Feel-IT：音频触觉计算机辅助功能

• 批准号: 1936027
• 负责人: Yevgen Borodin
• 金额: $ 21.09万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-15 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1936027&HistoricalAwards=false
• 关键词: Feel; Audio; Tactile; Computer; Accessibility

To access computers, blind people have to use screen readers and/or Braille displays, which make the content of the computer screen accessible, but require much time and patience because users have to search the screen sequentially for relevant content and controls. The goal of this Project is to develop Feel-IT, a portable hardware-device that will enable blind people to interact with computers much faster by using an audio-tactile (hearing and feeling) interface. Blind people will be able to use portable tactile Feel-IT gloves to “feel” the content of the computer screen while moving their hands on the surface of a regular desk. Combining the tactile and auditory sources of feedback will enable Feel-IT users to process information on the computer screen significantly faster. It will also reduce the information overload because they will no longer have to listen to all content that is on the screen. Tactile feedback will make computers more accessible for blind people, thus enabling them to participate fully in our digital society. The transformative idea of Feel-IT is that it will be a single device that will make all screens (including non-touch screens) of any size accessible with audio-tactile feedback. Feel-IT will be evaluated with blind computer users in real-life scenarios. Feel-IT will enable the study of the strategies that blind people employ for audio-tactile interaction with computer, improve our understanding of the optimal audio-tactile interface design, and provide a design blueprint for future assistive technology devices. This Project will also provide accessibility research training to undergraduate and graduate students, especially those with vision impairments.The goal of this project is to develop ‘Feel-It,” a new human-computer interface that will improve the ability of blind and low-vision people to interact with computers. The interface will compensate the users’ inability to see the screen content by enabling them to both hear the content and feel its tactile representation with both hands. The portable hardware-device will connect to any computer (touchscreen or not) or smart phone device via Bluetooth, and allow users to interact with their devices by putting their hands in/on a pair of haptic “gloves” and moving them on any flat surface, e.g., a desk or a table. The Research Plan is organized under three objectives. OBJECTIVE 1 is to design and develop the Feel-IT gloves to support high-resolution tactile feedback and portability. Features of the glove include: TULAs (Tiny Ultrasonic Linear Actuators) that are placed on the distal phalanxes of fingers and act similarly to Braille Cells but enable increased resolution and require less voltage/bulk and wires; infrared LED lights on each finger that will enable a camera to track finger position and orientation; a stabilizer that enables more comfortable placement of the hand on the glove and greater control over movements; and buttons to click links or zoom in and out of any segment. OBJECTIVE 2 is to design audio-tactile interfaces that enable general exploration of the computer screen and enable standard office apps. The design will build on a current Interface Manager and Finger Tracker that supports one glove and only enables tactile interaction with web pages. User Interface (UI) Automation and/or standard vision segmentation algorithms will be used to obtain information about the computer screen content and convert it to tactile representation. The interface manager, which will be limited to MS Windows, will represent. in tactile form, open windows, app icons, menu elements, and the content of app windows. The system will enable the user to “zoom” into any window, feel its components, listen to the content, and then zoom out. OBJECTIVE 3 is to conduct human subject research in laboratory and in-situ settings. Eighty low vision and blind subjects will be recruited to conduct user studies on a typical screen with multiple icons, windows, menus, etc., and with typical office applications, e.g., word processors, spreadsheets, etc. Performance measurse include time to perform task, effort to perform task, and failures. Satisfaction measures will be determined by an After Scenario Questionnaire, a System Usability Scale (1-10) and a System Preference Questionnaire. Expected research outcomes include: (1) Design and software implementation of novel haptic interfaces for computer access; (2) Understanding tactile behaviors and strategies employed by blind and low-vision people; and (3) Insight into how dynamic haptic feedback can help improve computer interaction.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

为了访问计算机，盲人必须使用屏幕阅读器和/或盲文显示器，这使得计算机屏幕的内容可以访问，但需要很多时间和耐心，因为用户必须顺序地搜索屏幕以查找相关内容和控件。该项目的目标是开发Feel-IT，一种便携式硬件设备，使盲人能够通过使用听觉触觉（听觉和感觉）界面更快地与计算机交互。盲人将能够使用便携式触觉Feel-IT手套来“感觉”电脑屏幕上的内容，同时在普通桌面上移动他们的手。结合触觉和听觉反馈源将使Feel-IT用户能够更快地处理计算机屏幕上的信息。它还将减少信息过载，因为他们将不再需要收听屏幕上的所有内容。触觉反馈将使盲人更容易使用计算机，从而使他们能够充分参与我们的数字社会。Feel-IT的变革性想法是，它将是一个单一的设备，将使任何尺寸的所有屏幕（包括非触摸屏）都可以通过听觉触觉反馈访问。Feel-IT将在现实生活场景中与盲人计算机用户一起进行评估。Feel-IT将使盲人与计算机进行听觉触觉交互的策略研究成为可能，提高我们对最佳听觉触觉界面设计的理解，并为未来的辅助技术设备提供设计蓝图。该项目还将为本科生和研究生提供无障碍研究培训，特别是那些有视力障碍的学生。该项目的目标是开发"Feel-It"，这是一种新的人机界面，将提高盲人和低视力人士与计算机互动的能力。该界面将通过使用户能够听到内容并用双手感受其触觉表示来补偿用户无法看到屏幕内容。 便携式硬件设备将通过蓝牙连接到任何计算机（触摸屏或非触摸屏）或智能电话设备，并允许用户通过将他们的手放在一对触觉"手套"中/上并在任何平面上移动它们来与他们的设备交互，例如，一张桌子或桌子。 研究计划有三个目标。 目标1是设计和开发Feel-IT手套，以支持高分辨率触觉反馈和便携性。手套的特点包括：图拉斯（微型超声波线性致动器）放置在手指的远端指骨上，作用类似于盲文单元，但可以提高分辨率，需要更少的电压/体积和电线;每个手指上的红外LED灯，使摄像头能够跟踪手指位置和方向;稳定器，使手能够更舒适地放置在手套上并更好地控制动作;以及用于单击链接或放大和缩小任何片段的按钮。 目标2是设计能够对计算机屏幕进行一般探索并启用标准办公应用程序的听觉触觉界面。该设计将建立在目前的界面管理器和手指跟踪器，支持一个手套，只使触觉与网页互动。 用户界面（UI）自动化和/或标准视觉分割算法将用于获取有关计算机屏幕内容的信息，并将其转换为触觉表示。界面管理器，这将是有限的MS Windows，将代表。以触觉的形式，打开窗口，应用程序图标，菜单元素和应用程序窗口的内容。该系统将使用户能够"放大"到任何窗口，感觉其组成部分，听内容，然后缩小。 目标3是在实验室和现场环境中进行人类受试者研究。将招募80名低视力和盲人受试者，在具有多个图标、窗口、菜单等的典型屏幕上进行用户研究，并且对于典型的办公应用，例如，文字处理器、电子表格等。性能测量包括执行任务的时间、执行任务的努力和失败。 将通过情景后问卷、系统可用性量表（1 - 10）和系统偏好问卷确定满意度指标。 预期的研究成果包括：（1）设计和软件实现的新型触觉接口的计算机访问;（2）理解触觉行为和策略，盲人和低视力的人使用;以及（3）深入了解动态触觉反馈如何帮助改善计算机交互。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准。

项目成果

Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users

通过为盲人用户提供统一的覆盖来驯服用户界面的异构性

• DOI: 10.1145/3503252.3531317
• 发表时间: 2022
• 期刊: Adaptation and Personalization (UMAP'22
• 作者: Uckun, Utku;Tumkur Suresh, Rohan;Ferdous, Md Javedul;Bi, Xiaojun;Ramakrishnan, I.V.;Ashok, Vikas
• 通讯作者: Ashok, Vikas