HCC: Small: AccessMath: Improving Mathematics Lectures for Low Vision Students through Integrated Video, Note-Taking and Search

获得数学教育对于促进科学，工程，技术和许多职业的职业生涯以及提供日常生活中所需的数学素养至关重要。 PI在该项目中的目标是增加低视力学生（视力受损但不失明）在课堂上实时访问数学材料的机会，这是目前最缺乏的，因为这些学生必须主要依靠辅助技术和人类笔记员在课堂会议后提供课程材料。 PI的方法是将计算机科学、数学和可访问性方面的专业知识结合起来，为iPad提供一个有用的解决方案。 该原型应用程序名为“Mimio Math”，它将提供可缩放的相机视图以及在白板上书写的笔记视图（由Mimio捕获），这些视图可以通过放大和对比度调整功能进行更改。 该系统将支持在讲座期间咨询和搜索课程内容，包括教科书、讲义、白板上的笔记以及视频中的音频（例如，在本课程的记录讲座中找到单词“特征值”）。 将采用新颖的笔记记录设施，其中笔记被表示在笔记“卡”上，笔记“卡”可以被附加到当前视图中的特定位置（例如，在白板视图中的一个点处），然后手动地移动、调整大小、旋转、堆叠和“弹”出视图，从而提供用于与在讲课期间创建的笔记非正式地交互的自然手段。 笔记可以创建为空白卡片，也可以创建为包含白板数据视图、视频摘录或现有笔记的卡片。 注释也用于构造搜索查询。 线性代数（特别是矩阵）是该项目的试点范围，因为在传递矩阵，矩阵中的各个项目，以下推导以及完成非平凡问题时所需的矩阵规模方面存在许多挑战。 CNOMATH用户将拥有一台连接到服务器的iPad，该服务器提供视频源以及识别和检索服务。Math将帮助低视力学生在高中和大学的课堂内外进行数学教学，通过提供即时访问讲座材料，包括搜索白板笔记，讲义，教科书中的数学内容，和讲座视频（通过音频中的单词识别）。 该系统将使学生在记笔记和定制访问方面拥有自主权，这将大大优于目前的解决方案，并将减少或消除视障学生在课堂环境中落后于视力正常的同龄人的程度。 该项目还将推进数学符号识别和检索的最新技术，并将使用桌面上照片的隐喻创建一个新颖的笔记模型，该模型应适用于数学以外的领域。

Access to mathematics education is critical to facilitating careers in the sciences, engineering, technology, and in many professions, as well as to providing the mathematical literacy needed in everyday life. The PI's goal in this project is to increase real-time access in the classroom to mathematical material for low vision students (who are visually impaired but not blind), which is currently mostly lacking as these students must primarily rely on assistive technology and human note takers to provide access to course materials after the class meeting. The PI's approach is to bring together expertise in computer science, mathematics, and accessibility to provide a useful and usable solution to this problem for the iPad. Called AccessMath, the prototype application will provide zoom-able camera views as well as a view of notes written at the whiteboard (captured by a Mimio), which may be altered through magnification and contrast adjustment features. The system will support consulting and searching course content during lecture, including textbooks, handouts, notes written at the whiteboard, and audio in video (e.g., to find the word "eigenvalue" in recorded lectures for the course). A novel note-taking facility will be employed, where notes are represented on note "cards" that can be attached to a specific location in the current view (e.g., at a spot in the whiteboard view), then manually moved, resized, rotated, stacked, and "flicked" out of view, providing a natural means for interacting informally with notes created during lecture. Notes are created as blank cards, or as cards containing views of whiteboard data, video excerpts, or existing notes. Notes are also used in constructing search queries. Linear algebra (in particular matrices) is the pilot scope for the project, due to the many challenges in conveying a matrix, individual items within the matrix, following derivations, and the scale of the matrix needed when completing non-trivial problems. AccessMath users will have a single iPad which connects to a server providing video feeds and recognition and retrieval services.Broader Impacts: AccessMath will assist low-vision students with mathematical instruction in and out of class, at the high school and university levels, by providing immediate access to lecture materials, including searches for mathematical content in whiteboard notes, handouts, textbooks, and lecture videos (through word spotting in audio). The system will afford student ownership in note taking and customizing access, which will be significantly better than current solutions, and will reduce or eliminate the extent to which visually impaired students fall behind their sighted peers in the classroom setting. The project will also advance the state-of-the-art in recognition and retrieval for mathematical notation, and will create a novel note-taking model using the metaphor of photos on a tabletop, which should be applicable to domains other than mathematics.