SBIR Phase II: Closing the Digital Divide: Real-Time Multisensory Learning for Special Education Students

这个项目创建了一个教育的，基于触摸屏的软件，将视觉教育内容转换为有特殊需要的学生，特别是那些失明和视觉障碍的学生，可以访问的，多感官的内容。考虑到当今教育领域的挑战：学校越来越多地采用数字工具，旨在为主流学生创造更具互动性和个性化的体验，但与此同时，学校也在努力适应多样化的学生群体，特别是美国600万接受特殊教育的学生。这个问题在科学、技术、工程和数学（STEM）领域被夸大了，因为这些学科的内容通常是复杂和可视化的。这个项目解决了这些挑战，建立在我们已经知道的人类信息处理和触觉界面的基础上，创建了一个软件，可以自动将高度视觉化的内容转换为可以在课堂上实时看到、听到和感受到的内容。本项目支持NSF？我们的使命是确保将所有学生纳入美国课堂数字化转型的最前沿。这项工作的社会影响克服了一些障碍，阻碍有特殊需要的学生在STEM教育经历中成为独立和积极的贡献者，并最终成为许多STEM职业。重要的项目在三年内为纳税人的投资带来直接的经济回报，创造收入和新的就业机会，并计划每年成倍增长。这个项目的创新之处在于创造了将视觉内容有效地翻译成多模态内容的方法和算法，适当地减少了非视觉感官通道的采样，但有效地传达了最有意义的信息。这种转换目前还不存在，这导致创建可访问图形的开销很高，并且是特殊教育的主要痛点，特别是对于有视觉障碍或失明的个人。此外，将图形内容从视觉转换到多模态（视觉、听觉和触觉）空间并不是一个简单的转换，因为与视觉相比，人类触觉的带宽有限，而且图形信息中呈现的特征很复杂。考虑到不同平台（游戏邦注：如Android和iOS）所提供的不同类型的触觉反馈，这一挑战被夸大了。该项目解决了这些挑战，在K-12教室的商业平台上创建了一个简化的解决方案，用于实时生成可访问的多感官内容。该项目的结果将(1)创建提供从视觉到多模态空间的自动转换所需的算法，(2)建立教师“仪表板”以简化拟议的软件？(3)将该软件扩展到iOS市场，同时发现利用Apple？独特的触觉反馈。开发完成后，该软件将与合作学校进行beta测试，然后在Android和iOS市场上发布。

This project creates an educational, touchscreen-based software that translates visual educational content into accessible, multisensory content for students with special needs, and particularly those with blindness and visual impairments. Consider the challenge in the educational landscape today: schools are increasingly adopting digital tools oriented toward creating a more interactive, personalized experience for mainstream students, but at the same time, are struggling to accommodate their diverse student population, particularly, the 6 million students in special education in the U.S. This problem is exaggerated in Science, Technology, Engineering, and Math (STEM), where content is often complex and visual. This project addresses these challenges, building on what we already know about human information processing and haptic interfaces to create software that will automatically convert highly visual content into content that can be seen, heard, and felt in real-time in class. This project supports NSF?s mission ensuring that inclusion of all students is at the forefront of the digital transformation of U.S. classrooms. The societal impacts of this work overcome several barriers impeding students with special needs from being independent and active contributors in the STEM educational experience and ultimately, many STEM professions. ViTAL projects a direct, financial return on investment for taxpayers within three years of operations, generating both revenue and new jobs, with plans to multiply this growth year over year. The innovation in this project is the creation of methods and algorithms for effective translation of visual content into multimodal content, appropriately down sampled for the nonvisual sensory channels yet effective in conveying the most meaningful information. Such a conversion currently does not exist, which results in high overhead costs to create accessible graphics and is a major pain point in special education, particularly for individuals with visual impairments or blindness. Further, translating graphical content from the visual to the multimodal (visual, auditory, and haptic) space is not a straightforward conversion due to the limited bandwidth of human touch compared to vision and the complexity of features presented in graphical information. This challenge is exaggerated when accounting for the varying types of haptic feedback that can be provided from varying platforms (Android and iOS). This project addresses these challenges, creating a streamlined solution for generating accessible, multisensory content, in real-time, on commercial platforms for K-12 classrooms. The outcomes of this project will (1) create the algorithms needed to provide an automated conversion from the visual to multimodal space, (2) establish a teacher "dashboard" to streamline the proposed software?s integration in the classroom, and (3) expand the software to the iOS market while simultaneously uncovering novel haptic effects that leverage Apple?s unique Taptic feedback. Upon completion of the development, the software will be beta tested with partnering schools, and then released on both Android and iOS markets.