SBIR Phase I: Ushering in a New Paradigm of Real-Time Tactile Graphics for Visually Impaired Students

该SBIR第一阶段项目旨在为科学、技术、工程和数学(STEM)教育中的盲人和视障(BVI)学生创建一种新形式的触觉(触摸)图形。为了说明这一点，可以考虑在传统的数学课堂中学习，而不能看到黑板上显示的图形。这是英属维尔京群岛学生在STEM课堂上每天面临的挑战，目前可用的技术需要密集的人工准备，无法实时呈现，导致学生学习延迟，而且成本高昂。该项目通过开发软件来解决这些挑战，该软件可以将视觉图像转换为可以在商业触摸屏上感觉到(通过振动)和听到(通过声音)的图像。这项工作植根于了解人们如何通过触摸来改善STEM学习体验，直接关系到NSF的使命。该项目将把美国课堂推向学习和包容的新高度，为各种学习风格的学生创造机会，让他们进入STEM课堂并在STEM课堂和职业中取得成功。这将通过创造一支称职的、多样化的美国STEM劳动力队伍来产生经济影响，并将美国定位为包容性、可获得的教育体验的明显领导者。这项工作的创新在于通过触摸屏上的振动和声音来传达视觉图形，这样最终用户可以在不需要看到图像的情况下解释图像中的信息。为了迎接这一重大挑战，该项目解决了基于触摸屏的触觉图形识别和跟踪线条的根本障碍。线条或线条的组合构成了大多数图形表示的基础。然而，由于触摸屏是平坦的，没有可用于表示线条的物理边缘或深度变化，这使得这些表面上的线条跟踪(以及随后的图形信息解释)具有难以置信的挑战性。在第一阶段项目完成后，这项工作将(1)展示通过触摸屏上的振动和声音来促进更有效的直线跟随所需的算法和非视觉表现的可行性，以及(2)在三个课堂环境中演示试点软件，以帮助英属维尔京群岛的学生和他们的老师了解如何将该软件整合到传统的教育环境中。该项目将有助于推动触摸屏超越视觉输入/输出设备，并将为个人如何通过振动感知、解释和学习提供新的线索，同时创造当今被认为是触觉图形的范式转变。

This SBIR Phase I project aims to create a new form of tactile (touch) graphics for blind and visually impaired (BVI) students in Science, Technology, Engineering, and Math (STEM) education. To illustrate, consider learning in a traditional mathematics classroom without being able to see the graphics presented on the board. This is the challenge that BVI students face on a daily basis in STEM classrooms, and the current technologies available require intensive manual preparation, cannot be presented in real-time causing a delay in student learning, and are expensive. This project addresses these challenges by developing software that translates visual images into images that can be felt (through vibrations) and heard (through sound) on commercially available touchscreens. This work, rooted in understanding how people see through touch toward bettering the STEM learning experience, directly relates to NSF's mission. This project will push U.S. classrooms into new heights of learning and inclusion that will create opportunities for students of all learning styles to access, participate, and succeed in STEM classrooms and professions. This will have an economic impact through the creation of a competent, diverse U.S. STEM workforce and will position the U.S. as a clear leader in inclusive, accessible educational experiences.The innovation in this work is in conveying visual graphics via vibrations and sounds on touchscreens, such that an end user could interpret information from the image without needing to see it. Toward meeting this grand challenge, this project addresses a fundamental barrier to touchscreen-based tactile graphics identifying and following lines. Lines or combinations of lines form the basis for most graphic representations. Because touchscreens are flat, however, there are no physical edges or depth changes that can be used to represent lines, making line following on these surfaces (and subsequently interpretation of graphical information) incredibly challenging. Upon completion of this Phase I project, this work will have (1) demonstrated the feasibility of the algorithms and nonvisual representations needed to promote more efficient line following via vibrations and sounds on touchscreens and (2) demonstrated pilot software in three classroom settings with BVI students and their teachers to understand how the software could be integrated into traditional educational settings. This project will contribute to the movement of propelling touchscreens beyond visual input/output devices and will shed new light on how individuals perceive, interpret, and learn through vibrations, while creating a paradigm shift in what is considered today as tactile graphics.