Automated Tactilization of Graphical Images: Full Access to Math, Science, and Engineering for Blind Students

图形图像(折线图、条形图、图表、插图等)在所有教育级别的数学、科学和工程(MSE)教科书中普遍存在。但是，尽管研究表明，触觉感知是视障人士理解图形图像的最佳方式，但教科书中的图形图像通常不是这种格式。视障学生缺乏对教科书内容的充分接触，这阻碍了他们的学习、发展和在MSE职业生涯中的成功，而在MSE职业生涯中，残疾人的比例很低。该项目寻求解决这一问题，方法是开发创新方法，克服将图形图像及时转换为实际格式的障碍。将解决两个用户群体的需求：在低产出和高产出环境中将图形图像转换为实际格式的抄写员；以及在K-12和中学后教育水平的MSE班级中盲人阅读盲文的学生。为此，PI组建了一支跨学科团队，拥有图像处理、机器学习、信息检索、人机交互、实验设计和满足残疾学生需求的专业知识。PI和他的团队将设计和开发触觉图形助手(TG Assistant)，这是一套Adobe Photoshop和Illustrator的插件，将支持抄写员尽可能自动和智能地将图形图像转换为高质量的触觉形式，以便盲人学生复制和使用。将进行实证性研究，以更好地了解触觉图形的感知，并为设计预测模型以估计图像理解时间和理解准确性，应用机器学习技术按图像类型分类图像，以及设计图像处理算法以执行将图像转换为触觉形式的步骤(适合图像类型)提供信息。在TG助手的开发过程中将遵循以用户为中心的设计方法。项目的好处将通过三项概念验证活动记录下来，其中TG助手将用于向三名K-8、高中和高等教育水平的学生提供访问教科书图像的途径。更广泛的影响。实际接触图形图像将提高盲人学生的学习、成绩、记忆力和在MSE职业生涯中取得成功的潜力。项目成果将包括关于触觉图形的触觉感知的宝贵数据、新的图像处理和分类算法，以及将图形图像简化为触觉格式的可用工具。关于图形图像翻译的研究成果将被总结为一套指导方针，并分发给抄写员。PI将努力将指南和TG助手纳入转录员培训计划。研究小组包括来自代表性不足的群体(妇女、少数民族和残疾人)的个人。

Graphical images (line graphs, bar charts, diagrams, illustrations, etc.) are prevalent in math, science, and engineering (MSE) textbooks at all educational levels. But while studies have shown that tactual perception is the best modality for comprehension of graphical images by people who are visually impaired, the graphical images found in textbooks typically aren't available in this format. Visually impaired students' lack of full access to the contents of textbooks impedes their learning, development, and success in MSE careers, areas in which individuals with disabilities are underrepresented. This project seeks to address this problem, by developing innovative ways to overcome obstacles to the timely translation of graphical images into a tactual format. The needs of two user communities will be addressed: transcribers who translate graphical images into tactual formats within low- and high-production environments; and students who are in MSE classes at the K-12 and postsecondary education levels, are blind, and read Braille. To these ends, the PI has assembled an interdisciplinary team with expertise in image processing, machine learning, IR, HCI, experiment design, and addressing the needs of students with disabilities. The PI and his team will design and develop the Tactile Graphics Assistant (TG Assistant), a set of plug-ins for Adobe Photoshop and Illustrator, which will support transcribers in transforming, as automatically and intelligently as possible, graphical images into a high-quality tactual form that can be reproduced and then used by students who are blind. Empirical studies will be conducted to better understand the perception of tactile graphics, and to inform the design of prediction models to estimate image comprehension time and comprehension accuracy, the application of machine learning techniques to classify images by their type, and the design of image processing algorithms to carry out the steps (appropriate for the image type) to translate an image into a tactual form. A user-centered design approach will be followed during the development of the TG Assistant. Project benefits will be documented by three proof-of-concept activities, wherein the TG Assistant will be used to provide access to textbook images to three students at the K-8, high school, and postsecondary education levels. Broader Impact. Tactual access to graphical images will improve blind students' learning, performance, retention, and potential to succeed in MSE careers. Project outcomes will include invaluable data on the tactual perception of tactile graphics, new image processing and classification algorithms, and a usable tool to streamline the translation of graphical images into a tactual format. Research findings on the translation of graphical images will be summarized as a set of guidelines and distributed to transcribers. The PI will make an effort to incorporate the guidelines and TG Assistant into transcriber training programs. The research team includes individuals from under-represented groups (women, minority, and disabled individuals.