CHS: Small: Reading, Doing, and Sharing Mathematical Expressions for the Blind: A Multimodal Approach

阅读、做和分享数学是认知密集型的任务，需要大量的学习和准备。这些技能构成了参与所有STEM领域和大多数职业的不可或缺的基础。因此，缺乏数学素养对盲人或有严重视力障碍(IBSVI)的人来说是非常不利的。到他们上中学的时候，IBSVI的数学成绩通常比看得见的同龄人落后1-3年，而且在接下来的几年里差距还会继续拉大。访问数学表达式的关键挑战是它们是在空间上布局的，并且需要交互地访问该空间符号。该项目将为IBSVI开发和研究支持在现代平板电脑和手持设备上阅读、做和分享数学的解决方案，通过平衡弱视访问、交互声音、触觉和触觉支持、空间标志、音频阅读、交互方法和数学表达式的结构等多种模式。这将为个性化解决方案打开大门，这些解决方案符合IBSVI的各种视觉能力(例如隧道视觉程度、微光能力、触觉能力)。本研究从小学后期和中学后期开始，对数学表达的多模式阅读、做和分享进行了研究。数学表达式的非线性性和高维性在空间上被编码，旨在用于视觉空间的访问和探索。此外，阅读和做数学是认知密集型的，需要多年的学习来培养识字能力。该项目将解决这个先有鸡还是先有蛋的问题，即需要识字才能获得数学材料，需要数学材料来建立识字能力。IBSVI的视觉和触觉能力的范围使这个问题变得更加复杂，他们具有中心视觉敏锐度、光和颜色敏感度、隧道程度和触觉能力。该项目将开发一个6元素模型，包括粗略的视觉提示、触觉地标覆盖网格配置、音频/发声反馈和阅读、交互方法、动态触觉反馈以及数学内容的结构组织。该模型将平衡数学获取需求和IBSVI视觉/触觉能力的范围，并将为支持阅读、做和共享数学而策划一个解决方案空间。在德克萨斯盲人和视障人士学校(TSBVI)，阅读、做和分享将在小学和中学的高年级学生中同时进行，教师将在迭代的设计、实施和测试过程中与项目研究人员合作，满足学生真实的数学需求。学生将能够持续接触到研究产品，以便研究人员能够深入了解IBSVI如何学习和适应技术，避免经常阻止研究方法超越首次使用情景的洞察力的“一次性综合症”。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Reading, doing, and sharing mathematics are cognitively intensive tasks that require significant amounts of learning and preparation. These skills form an indispensable foundation for participation in all STEM fields and most careers. So the lack of mathematical literacy severely disadvantages individuals who are blind or who have a severe visual impairment (IBSVI). By the time they reach middle school, IBSVI are typically 1-3 years behind their seeing counterparts in mathematics, and the gap continues to widen in subsequent years. The key challenge to accessing mathematical expressions is that they are laid out spatially, and this spatial notation needs to be accessed interactively. This project will develop and study solutions to support reading, doing, and sharing of mathematics on modern tablet and handheld devices for IBSVI by balancing across multiple modes of low-vision access, interactive sound, tactile and haptic support, spatial landmarking, audio reading, interaction methodologies and the structure of the mathematical expression. This will open the door to personalized solutions that honor the ranges of visual abilities (e.g., degree of tunnel vision, low-light capacities, tactile abilities) of IBSVI. The research will investigate such multimodal reading, doing and sharing of mathematics expressions starting in later elementary and middle school.The research begins with the proposition that information representation is designed for the sighted. The non-linearity and hyper-dimensionality of mathematical expressions are encoded spatially and are intended for visuo-spatial access and exploration. Furthermore, reading and doing of mathematics is cognitively intensive, requiring years of learning to build literacy. The project will address this chicken-and-egg problem where literacy is needed to access mathematical material, and mathematical material is needed to build literacy. This problem is compounded by the range of visual and tactile abilities of IBSVI, who have a range of central vision acuity, light and color sensitivity, degree of tunnel and tactile ability. The project will develop a 6-element model comprising coarse visual cues, tactile-landmark overlay grid configuration, audio/sonification feedback and reading, interaction methodologies, dynamic haptic feedback; and structural organization of the mathematical content. The model will be balanced for the range of mathematical access needs and IBSVI visual/tactile abilities, and will curate a solution space for supporting reading, doing, and sharing of mathematics. Reading, doing, and sharing will be treated in tandem for students at the upper elementary and middle school levels at the Texas School of the Blind and Visually Impaired (TSBVI), where teachers will work with project researchers in an iterative design, implement, and test process to address the authentic mathematics needs of their students. Students will have sustained access to the research products so that researchers can gain insight into how IBSVI learn 'and adapt to technologies, avoiding the 'one-shot-syndrome' that often prevents research approaches from moving beyond insights from first use scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.