SGER: Toward the Automated Tactilization of Graphical Images

图形图像的触觉感知是一个具有挑战性的问题。在这个项目中，PI和他的团队提议开发专门与低带宽触觉相关的新的图像处理和理解技术。PI和他的团队将设计和建造一个交互式触觉图像编辑器(TIE)的原型，使视力正常的用户能够尽可能自动地将图形图像转换为高度可理解的盲文图形(凸起的点在纸上浮雕)，盲人可以实际查看。重点是科学家、工程师和数学家(学生和专业人员)使用的教科书、研究论文和网页中出现的图形图像(图形、图形、图表和图表)。将制定使盲文图形尽可能容易理解的标准。这些信息将被提供给图形图像转录人员，使他们能够手动或在TIE的协助下就图形图像的转录做出正确的决定。为了实现他的目标，PI和他的团队将探索使用图像处理技术来自动化和加快将印刷或数字格式的图形图像转换为盲文图形的过程。他们将探索低级图像处理技术的应用，如去噪、边缘检测、平滑和分辨率降低，以确定产生有效盲文图形的启发式方法。他们将通过一项实验研究来验证开发的启发式方法，在实验研究中，盲人参与者将评估由派生的启发式方法产生的图像的可读性和可理解性。他们将探索分割技术的应用，使他们能够在图像中定位文本和其他需要特别注意的对象(例如，文本必须被提取并适当放置)。他们将开发分类和学习算法，使他们能够确定正在处理的图像的类型，因为不同类型的图像可能需要不同的处理步骤。他们将开发一个软件原型--触觉图像编辑器(TIE)，这将使他们能够在未来的研究中轻松地应用图像处理启发式、分割技术以及分类和学习算法。他们将对盲文抄写器进行观察性研究，以更好地了解他们使用的过程，并为TIE的界面和功能的未来设计提供信息。更广泛的影响：这项研究将使盲人科学家、工程师和数学家(学生和专业人员)更好、更及时地接触到在教科书、研究论文和网络上越来越普遍的图形图像。这项研究还将使人们更好地理解是什么让盲文图形最容易理解；这些信息将对图形图像转录者有用，无论他们是否使用Tie工具。尽管这项研究的重点是科学的图形材料，但这种领带在其他依赖图形图像来收集信息的领域也有潜在的用处。参与该项目的学生，无论是研究生还是本科生，都将受益于它的跨学科性质。

Tactual perception of graphical images is a challenging problem. In this project, the PI and his team propose to develop new image processing and understanding techniques that specifically relate to low-bandwidth tactual perception. The PI and his team will design and build a prototype of an interactive Tactile Image Editor (TIE) that will enable sighted users to translate, as automatically as possible, graphical images into highly comprehensible Braille graphics (raised dots embossed on paper) that can be viewed tactually by blind persons. The focus is on graphical images (drawings, graphs, charts, and diagrams) that appear in textbooks, research papers, and Web pages that are used by scientists, engineers, and mathematicians (students and professionals). Criteria will be developed for what makes a Braille graphic as understandable as possible. That information will be provided to graphical image transcribers to enable them to make good decisions about the transcription of graphical images either manually or with the assistance of the TIE. To achieve his goals, the PI and his team will explore the use of image processing techniques to automate and expedite the conversion of graphical images that are in print or digital format into Braille graphics. They will explore application of low-level image processing techniques such as denoising, edge detection, smoothing, and resolution reduction, to determine heuristics for producing effective Braille graphics. They will validate developed heuristics with an experimental study, in which blind participants will evaluate the readability and comprehensibility of images produced by the derived heuristics. They will explore the application of segmentation techniques to enable them to locate text and other objects that need special attention within an image (for instance, text must be extracted and placed appropriately). They will develop classification and learning algorithms to enable them to determine the type of image being processed, because different kinds of images may require different processing steps. They will develop a software prototype, the Tactile Image Editor (TIE), which will enable them to easily apply the image processing heuristics, segmentation techniques, and classification and learning algorithms in future research. They will conduct an observational study of Braille transcribers, to better understand the processes they use and to inform the future design of the TIE's interface and functionality.Broader Impacts: This research will lead to better and more timely access by blind scientists, engineers, and mathematicians (students and professionals) to graphical images that are becoming more and more prevalent in textbooks, research papers, and on the Web. The research will also lead to a better understanding of what makes Braille graphics most understandable; this information will be useful for graphical image transcribers, whether they use the TIE tool or not. Although the focus of the research is on scientific graphical material, the TIE is potentially useful in other areas that rely on graphical images to covey information. The students, both graduate and undergraduate, who work on the project will benefit from its cross-disciplinary nature.