SBIR Phase II: Closing the Digital Divide: Real-Time Multisensory Learning for Special Education Students

该项目创建了一个基于触摸屏的教育软件，将视觉教育内容转换为可访问的多感官内容，供有特殊需要的学生使用，特别是那些失明和视力障碍的学生。考虑一下当今教育领域的挑战：学校越来越多地采用数字化工具，旨在为主流学生创造更具互动性的个性化体验，但同时，也在努力适应其多样化的学生群体，特别是美国600万接受特殊教育的学生。这个问题在科学、技术、工程和数学（STEM）中被夸大了，其中内容通常是复杂的和可视的。这个项目解决了这些挑战，建立在我们已经知道的人类信息处理和触觉接口的基础上，创建软件，将高度可视化的内容自动转换为可以在课堂上实时看到，听到和感觉到的内容。该项目支持NSF？的使命，确保所有学生的包容性是在美国教室数字化转型的最前沿。这项工作的社会影响克服了阻碍有特殊需要的学生成为STEM教育经验和最终许多STEM专业的独立和积极贡献者的几个障碍。ViTAL预计在运营三年内为纳税人带来直接的财务投资回报，创造收入和新的就业机会，并计划逐年增加这一增长。该项目的创新之处在于创建了将视觉内容有效转换为多模态内容的方法和算法，适当地对非视觉感官通道进行下采样，同时有效地传达最有意义的信息。这种转换目前还不存在，这导致创建无障碍图形的管理费用很高，是特殊教育的一个主要痛点，特别是对视力障碍或失明的人来说。此外，由于与视觉相比人类触摸的有限带宽以及图形信息中呈现的特征的复杂性，将图形内容从视觉转换到多模态（视觉、听觉和触觉）空间不是简单的转换。当考虑到不同平台（Android和iOS）可以提供的不同类型的触觉反馈时，这一挑战被夸大了。该项目解决了这些挑战，创建了一个简化的解决方案，用于在K-12教室的商业平台上实时生成可访问的多感官内容。该项目的成果将（1）创建所需的算法，以提供从视觉到多模态空间的自动转换，（2）建立一个教师“仪表板”，以简化拟议的软件？的整合在课堂上，（3）扩大软件的iOS市场，同时发现新的触觉效果，利用苹果？独特的Taptic反馈。开发完成后，该软件将与合作学校进行beta测试，然后在Android和iOS市场上发布。

This project creates an educational, touchscreen-based software that translates visual educational content into accessible, multisensory content for students with special needs, and particularly those with blindness and visual impairments. Consider the challenge in the educational landscape today: schools are increasingly adopting digital tools oriented toward creating a more interactive, personalized experience for mainstream students, but at the same time, are struggling to accommodate their diverse student population, particularly, the 6 million students in special education in the U.S. This problem is exaggerated in Science, Technology, Engineering, and Math (STEM), where content is often complex and visual. This project addresses these challenges, building on what we already know about human information processing and haptic interfaces to create software that will automatically convert highly visual content into content that can be seen, heard, and felt in real-time in class. This project supports NSF?s mission ensuring that inclusion of all students is at the forefront of the digital transformation of U.S. classrooms. The societal impacts of this work overcome several barriers impeding students with special needs from being independent and active contributors in the STEM educational experience and ultimately, many STEM professions. ViTAL projects a direct, financial return on investment for taxpayers within three years of operations, generating both revenue and new jobs, with plans to multiply this growth year over year. The innovation in this project is the creation of methods and algorithms for effective translation of visual content into multimodal content, appropriately down sampled for the nonvisual sensory channels yet effective in conveying the most meaningful information. Such a conversion currently does not exist, which results in high overhead costs to create accessible graphics and is a major pain point in special education, particularly for individuals with visual impairments or blindness. Further, translating graphical content from the visual to the multimodal (visual, auditory, and haptic) space is not a straightforward conversion due to the limited bandwidth of human touch compared to vision and the complexity of features presented in graphical information. This challenge is exaggerated when accounting for the varying types of haptic feedback that can be provided from varying platforms (Android and iOS). This project addresses these challenges, creating a streamlined solution for generating accessible, multisensory content, in real-time, on commercial platforms for K-12 classrooms. The outcomes of this project will (1) create the algorithms needed to provide an automated conversion from the visual to multimodal space, (2) establish a teacher "dashboard" to streamline the proposed software?s integration in the classroom, and (3) expand the software to the iOS market while simultaneously uncovering novel haptic effects that leverage Apple?s unique Taptic feedback. Upon completion of the development, the software will be beta tested with partnering schools, and then released on both Android and iOS markets.