CHS: Small: Reading, Doing, and Sharing Mathematical Expressions for the Blind: A Multimodal Approach

点击翻译按钮获取中文摘要

Reading, doing, and sharing mathematics are cognitively intensive tasks that require significant amounts of learning and preparation. These skills form an indispensable foundation for participation in all STEM fields and most careers. So the lack of mathematical literacy severely disadvantages individuals who are blind or who have a severe visual impairment (IBSVI). By the time they reach middle school, IBSVI are typically 1-3 years behind their seeing counterparts in mathematics, and the gap continues to widen in subsequent years. The key challenge to accessing mathematical expressions is that they are laid out spatially, and this spatial notation needs to be accessed interactively. This project will develop and study solutions to support reading, doing, and sharing of mathematics on modern tablet and handheld devices for IBSVI by balancing across multiple modes of low-vision access, interactive sound, tactile and haptic support, spatial landmarking, audio reading, interaction methodologies and the structure of the mathematical expression. This will open the door to personalized solutions that honor the ranges of visual abilities (e.g., degree of tunnel vision, low-light capacities, tactile abilities) of IBSVI. The research will investigate such multimodal reading, doing and sharing of mathematics expressions starting in later elementary and middle school.The research begins with the proposition that information representation is designed for the sighted. The non-linearity and hyper-dimensionality of mathematical expressions are encoded spatially and are intended for visuo-spatial access and exploration. Furthermore, reading and doing of mathematics is cognitively intensive, requiring years of learning to build literacy. The project will address this chicken-and-egg problem where literacy is needed to access mathematical material, and mathematical material is needed to build literacy. This problem is compounded by the range of visual and tactile abilities of IBSVI, who have a range of central vision acuity, light and color sensitivity, degree of tunnel and tactile ability. The project will develop a 6-element model comprising coarse visual cues, tactile-landmark overlay grid configuration, audio/sonification feedback and reading, interaction methodologies, dynamic haptic feedback; and structural organization of the mathematical content. The model will be balanced for the range of mathematical access needs and IBSVI visual/tactile abilities, and will curate a solution space for supporting reading, doing, and sharing of mathematics. Reading, doing, and sharing will be treated in tandem for students at the upper elementary and middle school levels at the Texas School of the Blind and Visually Impaired (TSBVI), where teachers will work with project researchers in an iterative design, implement, and test process to address the authentic mathematics needs of their students. Students will have sustained access to the research products so that researchers can gain insight into how IBSVI learn 'and adapt to technologies, avoiding the 'one-shot-syndrome' that often prevents research approaches from moving beyond insights from first use scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.