SGER: Toward the Automated Tactilization of Graphical Images

图形图像的触觉感知是一个具有挑战性的问题。 在这个项目中，PI和他的团队建议开发新的图像处理和理解技术，特别是与低带宽触觉感知相关的技术。 PI及其团队将设计和构建一个交互式触觉图像编辑器（TIE）的原型，该编辑器将使视力正常的用户能够尽可能自动地将图形图像转换为盲人可以触摸查看的高度可理解的盲文图形（凸点压印在纸上）。 重点是科学家、工程师和数学家（学生和专业人士）使用的教科书、研究论文和网页中出现的图形图像（图纸、图形、图表和图表）。 将制定标准，以确定如何使盲文图形尽可能易于理解。 这些信息将提供给图形图像转录员，使他们能够就图形图像的转录做出正确的决定，无论是手动还是在技术信息交换的帮助下。 为了实现他的目标，PI和他的团队将探索使用图像处理技术来自动化和加速将印刷或数字格式的图形图像转换为盲文图形。 他们将探索低层次图像处理技术的应用，如去噪，边缘检测，平滑和分辨率降低，以确定生产有效盲文图形的算法。 他们将通过一项实验研究来验证开发的视觉识别技术，在这项实验中，盲人参与者将评估衍生视觉识别技术产生的图像的可读性和可理解性。 他们将探索分割技术的应用，使他们能够在图像中定位需要特别注意的文本和其他对象（例如，必须提取文本并适当放置）。 他们将开发分类和学习算法，使他们能够确定正在处理的图像类型，因为不同类型的图像可能需要不同的处理步骤。 他们将开发一个软件原型，触觉图像编辑器（TIE），这将使他们能够在未来的研究中轻松应用图像处理算法，分割技术以及分类和学习算法。 他们将对盲文抄写员进行观察性研究，以更好地了解他们使用的过程，并为TIE界面和功能的未来设计提供信息。更广泛的影响：这项研究将使盲人科学家、工程师和数学家（学生和专业人士）更好、更及时地获得在教科书、研究论文和网络上越来越普遍的图形图像。 这项研究还将使人们更好地了解是什么使盲文图形最容易理解;这一信息将对图形图像转录器有用，无论他们是否使用TIE工具。 虽然研究的重点是科学的图形材料，TIE是潜在的有用的依赖于图形图像来传递信息的其他领域。 从事该项目的研究生和本科生将从其跨学科性质中受益。

Tactual perception of graphical images is a challenging problem. In this project, the PI and his team propose to develop new image processing and understanding techniques that specifically relate to low-bandwidth tactual perception. The PI and his team will design and build a prototype of an interactive Tactile Image Editor (TIE) that will enable sighted users to translate, as automatically as possible, graphical images into highly comprehensible Braille graphics (raised dots embossed on paper) that can be viewed tactually by blind persons. The focus is on graphical images (drawings, graphs, charts, and diagrams) that appear in textbooks, research papers, and Web pages that are used by scientists, engineers, and mathematicians (students and professionals). Criteria will be developed for what makes a Braille graphic as understandable as possible. That information will be provided to graphical image transcribers to enable them to make good decisions about the transcription of graphical images either manually or with the assistance of the TIE. To achieve his goals, the PI and his team will explore the use of image processing techniques to automate and expedite the conversion of graphical images that are in print or digital format into Braille graphics. They will explore application of low-level image processing techniques such as denoising, edge detection, smoothing, and resolution reduction, to determine heuristics for producing effective Braille graphics. They will validate developed heuristics with an experimental study, in which blind participants will evaluate the readability and comprehensibility of images produced by the derived heuristics. They will explore the application of segmentation techniques to enable them to locate text and other objects that need special attention within an image (for instance, text must be extracted and placed appropriately). They will develop classification and learning algorithms to enable them to determine the type of image being processed, because different kinds of images may require different processing steps. They will develop a software prototype, the Tactile Image Editor (TIE), which will enable them to easily apply the image processing heuristics, segmentation techniques, and classification and learning algorithms in future research. They will conduct an observational study of Braille transcribers, to better understand the processes they use and to inform the future design of the TIE's interface and functionality.Broader Impacts: This research will lead to better and more timely access by blind scientists, engineers, and mathematicians (students and professionals) to graphical images that are becoming more and more prevalent in textbooks, research papers, and on the Web. The research will also lead to a better understanding of what makes Braille graphics most understandable; this information will be useful for graphical image transcribers, whether they use the TIE tool or not. Although the focus of the research is on scientific graphical material, the TIE is potentially useful in other areas that rely on graphical images to covey information. The students, both graduate and undergraduate, who work on the project will benefit from its cross-disciplinary nature.