RAPD: Analytical and Exploratory Approaches to Communicate Mathematics to Visually Impaired Students

主要研究者：Pontelli，Enrico提案编号：人类视觉系统的特点是大的空间带宽和并发处理能力，使个人能够从复杂的视觉布局中的单个组件，项目组和模式中学习。视觉信息处理的使能能力在许多领域中是至关重要的，理解具有数学内容的文档就是这样的领域之一。数学的传统表示本质上是空间和多维的（例如，矩阵，分数），并且它经常结合分析表示（例如，公式，定义）与直观的图形描述（例如，这给视力残疾的人造成了根本障碍;他们缺乏视觉通道，这使他们无法访问数学内容的传统表示，即使是以数字格式编码的（例如，在线数学教科书）。视力残疾学生使用的传统辅助技术（例如，盲文显示器、屏幕阅读器）本质上是线性的，去除了上下文参考，并对读者的工作记忆施加了非常高的负荷。实际上，数学的可及性已经成为信息和学习机会的数字障碍。这在教育方面尤其严重，在教育方面，强调远距离教育和在线材料的同时，没有适当地采用无障碍和普遍使用的原则。 该项目的目标是开发一套知识和工具，使视力残疾学生能够获得具有重要数学内容的在线教育材料。这项工作的目的是使视障学生能够有效地接触和学习数学，并遵循与有视力学生相同的认知过程。特别是，我们的目标是建立一个全面的框架，提供在分析层面上的数学内容（例如，公式、定理、证明），在探索/直觉水平（例如，内容探索、对图表的访问），以及在实践层面（例如，问题解决，练习）。这项工作的指导原则是理解和形式化存在于数学文档中的底层语义结构，从其基本组件（例如，单独的公式、图的组件）以及向更复杂的组织（例如，逻辑步骤、证明、定理、图表）。该语义结构用于指导多模态呈现方案，其提供呈现之间的自动调谐平衡（即，对内容的连续访问）和导航（即，用户引导的内容遍历）、基于用户模型的呈现适配、以及听觉和触觉呈现的组合。触觉演示依赖于一种新的硬件设备（触觉手指）的设计和开发作为本项目的一部分。我们提出的框架也将使视障学生交错的阅读”数学与“实践”，通过一个方便的工作簿，直接互动的阅读过程。我们将采用不同层次的评估，让视障学生在设计初期参与，并依靠正式的认知模型来评估进度。该项目汇集了一批经验丰富的研究人员，他们在应用认知/感知研究、辅助技术软件和硬件研究以及可用性设计和评估等领域拥有专业知识。研究人员以前曾合作过，例如，Tran和Pontelli进行了NSF赞助的关于视障学生的网络可访问性的研究。智力优势：（1）观察和分析视障学生与以电子形式呈现的数学内容的人机交互，（2）开发提取数学内容的语义表示的方法;（3）数学概念图形表达的触觉装置和控制软件的设计：（4）数学内容听觉表达和导航方法的研究;（4）将触觉和听觉呈现与数学导航相结合，由用户模型驱动，并与可访问的数学概念实践相结合;（5）对视力正常和视力受损的学生进行可用性评估。（1）促进教学、培训和学习，重点关注视力残疾的多样性;（2）扩大视力残疾学生的代表性，使他们能够访问和浏览具有大量数学内容的文件（例如，数学教科书），从而增加他们在所有领域的教育机会，特别是在科学和工程领域;（3）通过提高可用性专家在教育环境中评估触觉和听觉界面的能力，加强教育基础设施。最终，社会将受益于教育的更大包容性，改善视力残疾学生的培训机会，并普遍使这些用户更广泛地获得信息。所有PI都积极参与各种举措，以促进多样化学生群体的教育机会-例如，NMSU是一个少数民族和西班牙裔服务机构，和学生从传统上代表性不足的群体将参与研究和教育活动.关键词：数学的可及性，触觉导航，视觉障碍，可用性评估方法

PI: Pontelli, EnricoProposal Number: 0754525The human visual system is characterized by a large spatial bandwidth and concurrent processing capabilities, enabling individuals to learn from individual components, groups of items, and patterns in complex visual layouts. The enabling capabilities of visual information processing are vital in many areas the understanding of documents with mathematical content is one of such areas. The traditional representation of mathematics is inherently spatial and multi-dimensional (e.g., matrices, fractions), and it often combines analytical presentation (e.g., formulae, definitions) with intuitive graphical descriptions (e.g., diagrams).This creates a fundamental obstacle for people with visual disabilities; their lack of visual access prevents them from accessing traditional representations of mathematical content, even when encoded in digital format (e.g., on-line mathematics textbooks). The traditional assistive technologies employed by students with visual disabilities (e.g., Braille displays, screen readers) are inherently linear, removing contextual references and imposing very high load on the reader's working memory. Effectively, accessibility of mathematics has become a digital barriers to information and learning opportunities. This is particularly severe in the educational context, where the emphasis on distance education and on-line material has not been properly matched by the adoption of accessibility and universal usability principles. The goal of this project is to develop a body of knowledge and tools that will enable students with visual disabilities to access on-line educational material with significant mathematical content. The aim of this work is to enable visually impaired students to access and learn mathematics with a level of efficiency and following cognitive processes that are at par with those used by sighted students. In particular, we aim at creating a comprehensive framework which provides access to mathematical content at the analytical level (e.g., formulae, theorems, proofs), at the exploratory/intuitive level (e.g., content exploration, access to diagrams and charts), and at the practical level (e.g., problem solving, exercises).The guiding principle of this work is to understand and formalize the underlying semantic structure present in mathematical documents, starting from its basic components (e.g., individual formulae, components of a graph) and moving towards more complex organizations (e.g., logical steps, proofs, theorems, diagrams). Thesemantic structure is employed to guide multi-modal presentation schemes, which provide an automatically tuned balance between presentation (i.e., continuous access to the content) and navigation (i.e., user-directed traversal of the content), presentation adaptation based on user models, and combination of aural and haptic presentation. Haptic presentation relies on a novel hardware device (the haptic finger) to be designed and developed as part of this project. The framework we propose will also enable visually impaired students to interleave \reading" mathematics with \practicing" it, through an accessible workbook, which directly interacts with the reading process. Different levels of evaluation will be applied, involving visually impaired students early on in the design, and relying on formal cognitive models to assess progress.The project assembles a team of experienced researchers with expertise in the areas of applied cognitive/perceptual research, assistive technology software and hardware research, and usability design and evaluation. The researchers have collaborated previously, e.g., Tran and Pontelli conducted NSF-sponsored research in accessibility of the web for visually-impaired students.Intellectual Merit: (1) Observation and analysis of human-computer interaction by students with visual disabilities with mathematical content presented in electronic form, (2) Development of methodologies to extract semantic representations of mathematical content; (3) Design of a haptic device and control software for the presentation of graphical representations of mathematical concepts; (4) Study of approaches to auralpresentation and navigation of mathematical content; (4) Integration of haptic and aural presentation and navigation of mathematics, driven by user models, and composed with workspaces for accessible practice of mathematical concepts; (5) Usability evaluation with sighted and visually-impaired students.Broader Impact: (1) promote teaching, training, and learning, with focus on diversity due to visual disabilities; (2) broaden the representation of students with visual disabilities in their ability to access and navigate documents with heavy mathematical content (e.g., math textbooks), thereby increasing their educational opportunities in all areas, but especially in the sciences and engineering; (3) enhance infrastructure for education by improving the ability of usability experts to evaluate haptic and auditory interfaces in an educational setting. Ultimately, society will benefit from the greater inclusion in education and improved training opportunities for students with visual disabilities and generally by making information more widely accessible to these users. All PIs are actively involved in initiatives to advance the educational opportunitiesfor a diverse student population - e.g., NMSU is a Minority and Hispanic-serving Institution, and students from traditionally underrepresented groups will be involved in the research and educational activities.Keywords: accessibility of mathematics, haptic navigation, visual disabilities, usability evaluation methods