III: Small: Extracting Data and Structure from Charts and Graphs for Analysis, Reuse and Indexing

图表和图表是普遍存在的数据表示形式，出现在科学论文、教科书、报告、新闻文章和网页中。这些可视化利用人类的视觉处理来有效地传达大量的量化信息，并说明数据中的趋势和差异。但是，尽管人们可以很容易地从图表和图形中解释数据，但机器不能直接访问这些数据。今天，海量的信息被锁定在数据可视化中。这项提议将开发从这种可视化中提取数据和结构的技术，从而使数据能够在现有图表和图表集合中进一步分析、重复使用和建立新的索引形式。其中一些应用程序将专门设计，以改善视障用户的可视化可访问性。这些工具将为知识整合和共享提供一种新颖的计算基础设施，并影响包括科学家、记者、经济学家、社会科学家和教育工作者在内的广泛用户。具体地说，这项提案涉及三个主要目标。首先，它开发用于解释可视化的计算模型，以提取底层数据、图形标记和将数据与标记属性相关联的映射。这一方法将受到最近关于人类对可视化的感知和认知的研究的启发。其目的是建立通用的计算模型，能够准确地从可视化中提取数据，并模仿人们从可视化中解码信息的方式。其次，它支持开发一套应用程序，以实现可视化和数据的分析和再利用。第三，它在互联网规模上应用了自动化可视化解释技术，并开发了一个搜索引擎，根据可视化的底层数据和图形结构对其进行索引。搜索引擎将通过促进浏览和检索目前被锁定在计算上无法访问的可视化的数据来加速数据驱动的分析和发现。项目网站将包括关于该项目的信息，并提供查阅所产生的出版物、软件和数据集的途径。

Charts and graphs are ubiquitous forms of data representations, appearing in scientific papers, textbooks, reports, news articles and webpages. These visualizations leverage human visual processing to efficiently convey large amounts of quantitative information, and to illustrate trends and differences in the data. But, while people can easily interpret data from charts and graphs, machines cannot directly access this data. Today, a vast trove of information is locked inside data visualizations. This proposal will develop techniques for extracting data and structure from such visualizations and thereby enable data further analysis, reuse and new forms of indexing across the collection of existing charts and graphs. Some of the applications will be specifically designed to improve the accessibility of visualizations for visual impaired users. The tools will provide a novel computational infrastructure for knowledge integration and sharing and impact a broad range of users including scientists, journalists, economists, social scientists, and educators. Specifically this proposal addresses three main goals. First, it develops computational models for interpreting visualizations to extract the underlying data, graphical marks, and mappings that relate the data to mark attributes. The approach will be informed by recent work on human perception and cognition of visualizations. The aim is to build generalized computational models that can accurately extract data from visualizations and also mimic the way people decode information from visualizations. Second, it supports development of a suite of applications that enable analysis and repurposing of visualizations and data. Third, it applies automated visualization interpretation techniques at Internet scale and develops a search engine that indexes visualizations based on their underlying data and graphical structure. The search engine will accelerate data-driven analysis and discovery by facilitating browsing and retrieval of data that is currently locked in computationally inaccessible visualizations. The project website will include information on the project and provide access to resulting publications, software and datasets.