Feel-IT: Audio-Tactile Computer Accessibility

为了使用计算机，盲人必须使用屏幕阅读器和/或盲文显示器，这使得计算机屏幕上的内容可以访问，但需要花费大量时间和耐心，因为用户必须依次搜索屏幕以查找相关内容和控件。该项目的目标是开发Feel-IT，这是一种便携式硬件设备，通过使用听觉和感觉界面，使盲人能够更快地与计算机交互。盲人将能够使用便携式触觉触觉手套来“感受”电脑屏幕上的内容，同时在普通桌面上移动他们的手。触觉和听觉反馈的结合将使Feel-IT用户在电脑屏幕上处理信息的速度大大加快。它还将减少信息过载，因为他们将不再需要听屏幕上的所有内容。触觉反馈将使盲人更容易使用电脑，从而使他们能够充分参与我们的数字社会。Feel-IT的革命性理念是，它将是一个单一的设备，它将使所有屏幕（包括非触摸屏）的任何尺寸都可以通过听觉-触觉反馈。Feel-IT将在真实场景中与盲人电脑用户一起进行评估。Feel-IT将研究盲人与计算机进行听觉触觉交互的策略，提高我们对最佳听觉触觉界面设计的理解，并为未来的辅助技术设备提供设计蓝图。该项目还将为本科生和研究生提供无障碍研究培训，特别是那些有视力障碍的学生。这个项目的目标是开发“Feel-It”，一种新的人机界面，将提高盲人和低视力人群与电脑互动的能力。该界面将弥补用户无法看到屏幕内容的缺陷，使他们既能听到内容，又能用双手感受到它的触觉表现。这种便携式硬件设备可以通过蓝牙连接到任何电脑（触摸屏或非触摸屏）或智能手机设备上，用户可以把手放在一副触觉“手套”上，在任何平面上（例如书桌或桌子）移动它们，从而与他们的设备进行交互。研究计划有三个目标。目标1是设计和开发Feel-IT手套，以支持高分辨率的触觉反馈和便携性。该手套的特点包括：图拉（微型超声线性致动器），放置在手指的远端指骨上，作用类似于盲文细胞，但可以提高分辨率，需要更少的电压/体积和电线；每个手指上的红外LED灯将使摄像头能够跟踪手指的位置和方向；一个稳定器，使手在手套上更舒适的位置和更大的运动控制；和按钮点击链接或放大和缩小任何部分。目标2是设计音频触觉界面，使计算机屏幕的一般探索和启用标准的办公应用程序。该设计将建立在当前的界面管理器和手指跟踪器的基础上，支持一只手套，只能与网页进行触觉交互。用户界面（UI）自动化和/或标准视觉分割算法将用于获取有关计算机屏幕内容的信息并将其转换为触觉表示。界面管理器，这将仅限于MS Windows，将表示。以触觉的形式，打开窗口、应用图标、菜单元素和应用窗口的内容。该系统将使用户能够“放大”任何窗口，感受其组成部分，听取内容，然后缩小。目标3是在实验室和现场环境中进行人体受试者研究。我们将招募80名低视力和盲人受试者，在一个典型的屏幕上进行用户研究，屏幕上有多个图标、窗口、菜单等，并使用典型的办公应用程序，如文字处理器、电子表格等。性能度量包括执行任务的时间、执行任务的努力和失败。满意度测量将由场景后问卷、系统可用性量表（1-10）和系统偏好问卷确定。预期的研究成果包括：(1)新型计算机触觉接口的设计与软件实现；(2)了解盲人和低视力人群的触觉行为和策略；(3)了解动态触觉反馈如何帮助改善计算机交互。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

To access computers, blind people have to use screen readers and/or Braille displays, which make the content of the computer screen accessible, but require much time and patience because users have to search the screen sequentially for relevant content and controls. The goal of this Project is to develop Feel-IT, a portable hardware-device that will enable blind people to interact with computers much faster by using an audio-tactile (hearing and feeling) interface. Blind people will be able to use portable tactile Feel-IT gloves to “feel” the content of the computer screen while moving their hands on the surface of a regular desk. Combining the tactile and auditory sources of feedback will enable Feel-IT users to process information on the computer screen significantly faster. It will also reduce the information overload because they will no longer have to listen to all content that is on the screen. Tactile feedback will make computers more accessible for blind people, thus enabling them to participate fully in our digital society. The transformative idea of Feel-IT is that it will be a single device that will make all screens (including non-touch screens) of any size accessible with audio-tactile feedback. Feel-IT will be evaluated with blind computer users in real-life scenarios. Feel-IT will enable the study of the strategies that blind people employ for audio-tactile interaction with computer, improve our understanding of the optimal audio-tactile interface design, and provide a design blueprint for future assistive technology devices. This Project will also provide accessibility research training to undergraduate and graduate students, especially those with vision impairments.The goal of this project is to develop ‘Feel-It,” a new human-computer interface that will improve the ability of blind and low-vision people to interact with computers. The interface will compensate the users’ inability to see the screen content by enabling them to both hear the content and feel its tactile representation with both hands. The portable hardware-device will connect to any computer (touchscreen or not) or smart phone device via Bluetooth, and allow users to interact with their devices by putting their hands in/on a pair of haptic “gloves” and moving them on any flat surface, e.g., a desk or a table. The Research Plan is organized under three objectives. OBJECTIVE 1 is to design and develop the Feel-IT gloves to support high-resolution tactile feedback and portability. Features of the glove include: TULAs (Tiny Ultrasonic Linear Actuators) that are placed on the distal phalanxes of fingers and act similarly to Braille Cells but enable increased resolution and require less voltage/bulk and wires; infrared LED lights on each finger that will enable a camera to track finger position and orientation; a stabilizer that enables more comfortable placement of the hand on the glove and greater control over movements; and buttons to click links or zoom in and out of any segment. OBJECTIVE 2 is to design audio-tactile interfaces that enable general exploration of the computer screen and enable standard office apps. The design will build on a current Interface Manager and Finger Tracker that supports one glove and only enables tactile interaction with web pages. User Interface (UI) Automation and/or standard vision segmentation algorithms will be used to obtain information about the computer screen content and convert it to tactile representation. The interface manager, which will be limited to MS Windows, will represent. in tactile form, open windows, app icons, menu elements, and the content of app windows. The system will enable the user to “zoom” into any window, feel its components, listen to the content, and then zoom out. OBJECTIVE 3 is to conduct human subject research in laboratory and in-situ settings. Eighty low vision and blind subjects will be recruited to conduct user studies on a typical screen with multiple icons, windows, menus, etc., and with typical office applications, e.g., word processors, spreadsheets, etc. Performance measurse include time to perform task, effort to perform task, and failures. Satisfaction measures will be determined by an After Scenario Questionnaire, a System Usability Scale (1-10) and a System Preference Questionnaire. Expected research outcomes include: (1) Design and software implementation of novel haptic interfaces for computer access; (2) Understanding tactile behaviors and strategies employed by blind and low-vision people; and (3) Insight into how dynamic haptic feedback can help improve computer interaction.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.