A Remote Multimodal Learning Environment to Increase Graphical Information Access for Blind and Visually Impaired Students

盲人或视障(BVI)学生在课堂或工作场所访问复杂的STEM图形信息时存在许多限制。这个长期存在的问题是由于依赖于难以获得和过时的学习材料，需要昂贵的专门设备，以及坚持过时的教育服务模式而产生的。为了解决这些问题，该项目将调查创新远程学习系统的开发和评估，该系统基于使用多个感觉通道来战略性地呈现来自听觉、语言、触觉和增强的视觉感知的信息。这项研究将专注于多通道信息呈现和感知的优化，根据其独特的信息处理特性来分离感觉输出，以传达不同类型的刺激。第一个项目的目标是提高英属维尔京群岛学生的STEM教学质量，方法是确定促进STEM图形和空间信息的非视觉知识获取的感知动机学习支持(学习目标)。第二个项目的目标是通过创建一个创新的远程多模式界面来传达视觉信息的概念含义，从而增加对图形和空间STEM内容的获取(技术目标)。该项目的成果将有助于非视觉学习和多感官处理的理论，并为盲人和视力正常的用户开发更高效、直观和可用的多模式界面提供一条明确的转换途径。结果的应用将有助于解决英属维尔京群岛人在STEM相关学科中代表性严重不足的问题，以及这一人口群体70%的失业率，为传播非视觉STEM图形材料提供一个新的、低成本和可访问的技术平台。研究人员将回答以下相互关联的问题：1)为了最大限度地提高STEM图形材料的感知突显性、可学习性、解释力和表现力，不同的通道输出所传达的最佳信息内容是什么？在实验室中进行优化后，2)优化的多模式学习系统在远程部署环境中向英属维尔京群岛学习者传输STEM图形材料方面的表现如何；以及3)与传统的BVI教学方法相比，远程学习系统是否提高了对STEM图形内容的理解水平？将收集和分析有关优化过程和远程技术系统的定量和定性数据，包括速度/精度方面的用户响应指标、用户体验数据和适用于英属维尔京群岛学生的STEM图形评估工具。研究的第一阶段将研究多模式信息处理，以建立信息传递和非视觉图形学习效率的最佳实践，通过实验比较以不同形式呈现的三个核心STEM图形主题的图形信息：图形、图表和地图。第二阶段的实验将调查远程学习系统的功效，以及评估用户在图形化STEM学习测量和关键可用性和满意度指标上的表现。该项目有更广泛的目标，即提高BVI学习者和其他可能受益于远程多模式学习环境的残疾或非残疾学生的独立性，并开发一种新工具，用于支持BVI人类受试者的大规模研究和辅助技术评估，从而显著提高科学家招募BVI用户并与其合作的能力，这比目前基于实验室的研究要多得多。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

There are many limitations for students who are blind or visually impaired (BVI) in accessing complex STEM graphical information in the classroom or workplace. This longstanding problem arises due to reliance on inaccessible and outdated learning materials, the need for costly specialized devices, and an adherence to an outdated educational service model. To address these issues, this project will investigate the development and evaluation of an innovative remote learning system based on the use of multiple sensory channels to strategically present information from auditory, linguistic, touch, and enhanced visual sensing. The research will focus specifically on the optimization of multimodal information presentation and perception, separating sensory output based on its unique information processing characteristics for conveying different types of stimuli. The first project goal is to increase the quality of STEM instruction for BVI students by determining perceptually motivated learning supports that promote non-visual knowledge acquisition of STEM graphical and spatial information (learning goal). The second project goal is to increase access to graphical and spatial STEM content through creation of an innovative remote multimodal interface for communicating the conceptual meaning of visual information (technology goal). The project outcomes will contribute to theories of non-visual learning and multisensory processing, and a clear translational path to development of more efficient, intuitive, and usable multimodal interfaces for both blind and sighted users. The application of the results will help to address the severe under-representation of BVI individuals in STEM-related disciplines, and the 70% unemployment rate of this demographic, by providing a new, low-cost, and accessible technology platform for communicating non-visual graphical STEM materials. The researchers will answer the following inter-connected questions: 1) What is the best information content to be conveyed by different modal outputs for maximizing perceptual saliency, learnability, interpretation, and representation of STEM graphical materials? Once optimized in the lab, 2) How well does the optimized multimodal learning system perform in a remote deployment environment in conveying graphical STEM materials to BVI learners; and 3) Does the remote learning system increase the level of comprehension of STEM graphical content as compared to traditional BVI instructional methods? Both quantitative and qualitative data about the optimization process and the remote technology system will be collected and analyzed including user response metrics on speed/accuracy, user experience data, and STEM graphical assessment instruments adapted for BVI students. The first phase of the research will investigate multimodal information processing in order to establish best practices for information delivery and non-visual graphical learning efficiency with experiments comparing graphical information presented in different modalities for three core STEM graphical themes: graphs, diagrams, and maps. The second phase of experiments will investigate the remote learning system's efficacy as well as evaluating user performance on graphical STEM learning measures and key usability and satisfaction metrics. This project has the broader goals of increasing independence for BVI learners and other students with or without disabilities who might benefit from a remote multimodal learning environment, and the development of a new tool for supporting large-scale research and assistive technology evaluation with BVI human subjects, thereby dramatically increasing scientists' ability to recruit and work with a much larger population of BVI users than is currently possible from lab-based studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.