CHS: Small: Reading, Doing, and Sharing Mathematical Expressions for the Blind: A Multimodal Approach

阅读、做数学和分享数学是需要大量学习和准备的认知密集型任务。这些技能是参与所有STEM领域和大多数职业不可或缺的基础。因此，缺乏数学知识对盲人或严重视力障碍（IBSVI）的人来说是非常不利的。当他们进入中学时，IBSVI学生在数学方面通常比有视力的同龄人落后1-3年，并且在随后的几年里差距会继续扩大。访问数学表达式的关键挑战是它们是在空间上布局的，并且需要交互式地访问这种空间符号。该项目将开发和研究解决方案，通过平衡低视力访问、交互式声音、触觉和触觉支持、空间地标、音频阅读、交互方法和数学表达结构等多种模式，支持在IBSVI的现代平板电脑和手持设备上阅读、做数学和分享数学。这将为尊重IBSVI的视觉能力范围（如隧道视觉程度、弱光能力、触觉能力）的个性化解决方案打开大门。本研究将探讨从小学和中学后期开始的数学表达式的多模态阅读、做和分享。研究开始于一个命题，即信息表示是为有视力的人设计的。数学表达式的非线性和超维性是空间编码的，目的是为了视觉空间的访问和探索。此外，阅读和做数学是认知密集型的，需要多年的学习来建立读写能力。该项目将解决这个先有鸡还是先有蛋的问题，即需要有读写能力来获取数学资料，又需要有数学资料来建立读写能力。IBSVI的视觉和触觉能力的范围使这个问题更加复杂，他们有一系列的中央视觉灵敏度，光和颜色敏感性，隧道度和触觉能力。该项目将开发一个6元素模型，包括粗糙的视觉线索、触觉地标覆盖网格配置、音频/超声反馈和阅读、交互方法、动态触觉反馈；以及数学内容的结构组织。该模型将平衡数学获取需求和IBSVI视觉/触觉能力的范围，并将策划一个支持数学阅读、实践和分享的解决方案空间。阅读、实践和分享将在德克萨斯州盲人和视障儿童学校（TSBVI）的小学高年级和初中阶段同步进行，教师将与项目研究人员合作，在迭代设计、实施和测试过程中解决学生的真实数学需求。学生将能够持续使用研究产品，以便研究人员能够深入了解IBSVI如何学习和适应技术，避免“一次性综合症”，这种综合症经常阻碍研究方法超越首次使用场景的洞察力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Reading, doing, and sharing mathematics are cognitively intensive tasks that require significant amounts of learning and preparation. These skills form an indispensable foundation for participation in all STEM fields and most careers. So the lack of mathematical literacy severely disadvantages individuals who are blind or who have a severe visual impairment (IBSVI). By the time they reach middle school, IBSVI are typically 1-3 years behind their seeing counterparts in mathematics, and the gap continues to widen in subsequent years. The key challenge to accessing mathematical expressions is that they are laid out spatially, and this spatial notation needs to be accessed interactively. This project will develop and study solutions to support reading, doing, and sharing of mathematics on modern tablet and handheld devices for IBSVI by balancing across multiple modes of low-vision access, interactive sound, tactile and haptic support, spatial landmarking, audio reading, interaction methodologies and the structure of the mathematical expression. This will open the door to personalized solutions that honor the ranges of visual abilities (e.g., degree of tunnel vision, low-light capacities, tactile abilities) of IBSVI. The research will investigate such multimodal reading, doing and sharing of mathematics expressions starting in later elementary and middle school.The research begins with the proposition that information representation is designed for the sighted. The non-linearity and hyper-dimensionality of mathematical expressions are encoded spatially and are intended for visuo-spatial access and exploration. Furthermore, reading and doing of mathematics is cognitively intensive, requiring years of learning to build literacy. The project will address this chicken-and-egg problem where literacy is needed to access mathematical material, and mathematical material is needed to build literacy. This problem is compounded by the range of visual and tactile abilities of IBSVI, who have a range of central vision acuity, light and color sensitivity, degree of tunnel and tactile ability. The project will develop a 6-element model comprising coarse visual cues, tactile-landmark overlay grid configuration, audio/sonification feedback and reading, interaction methodologies, dynamic haptic feedback; and structural organization of the mathematical content. The model will be balanced for the range of mathematical access needs and IBSVI visual/tactile abilities, and will curate a solution space for supporting reading, doing, and sharing of mathematics. Reading, doing, and sharing will be treated in tandem for students at the upper elementary and middle school levels at the Texas School of the Blind and Visually Impaired (TSBVI), where teachers will work with project researchers in an iterative design, implement, and test process to address the authentic mathematics needs of their students. Students will have sustained access to the research products so that researchers can gain insight into how IBSVI learn 'and adapt to technologies, avoiding the 'one-shot-syndrome' that often prevents research approaches from moving beyond insights from first use scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.