Data-driven modeling for understanding ancient documents from multimodal images

使用多光谱(MS)和太赫兹(THz)对文档进行无损分析代表了文档处理方面的一项重大技术进步，使我们图书馆和博物馆中存储的许多古代文档栩栩如生。对于这项发现拨款，我们专注于多模式历史文档图像的处理和理解。这项研究计划的总体目标是开发一种全面的数据驱动的无损分析方法，并有效地理解退化文档中包含的可视信息。此外，我们的目标是为处理通过传统数字化技术获取的大量文档这一具有挑战性的问题做出贡献。其短期目标是：OBJ 1)为MS/THz文档图像分析设计统一的图像分解框架；OBJ 2)设计数值退化隐藏方法，以便提供干净的数字化文档；OBJ 3)开发一种数据驱动的方法，用于理解视觉信息和发现大量古代文档中的潜在关系。*长期愿景是探索电磁频谱的不同非侵入性范围，并提供一个新的表示空间，其中灰色、彩色、MS和THz空间以统一的表示无缝映射。本届总干事的成果将是朝着这一目标迈出的第一步。长期目标将是推动太赫兹时间域(THz-TD)模式采用多层结构处理，这将有助于在不打开书籍和包装手稿的情况下分析闭合的书籍和手稿。从一层到另一层的内容破译是一个挑战，从处理步骤到THz图像的最佳频率采样和降维。*我们提出了一个多层次的建模，允许从根本上理解同一文档中墨水和纸张之间的相互作用以及外部因素，以及理解大型集合中的内容。这与统一的图像分析框架相结合，将提供一种全面的方法来理解具有重要文化意义的古代手稿。它将使学生体验到连接计算视觉和物理的实际和多学科问题。在新的THz-TD图像模式和与应用无关的数据驱动建模的支持下，拟议的研究将进一步推动这一愿景，并推动从自然科学到人文科学的多个学科的研究。因此，拟议的研究将导致数字文化遗产知识的显著进步；显著节省成本；并在太赫兹-TD文件图像模式的新时代带来创新。扩大与新用户群体的联系，开发和推广新的研究方法，管理新技术和数据，以及提高对数据在优化研究结果方面的潜力的认识，将确保加拿大的研究在世界范围内具有竞争力。

Nondestructive analysis of documents using multi-spectral (MS) and terahertz (THz) represents a major technical advance in document processing for bringing to life the many ancient documents that are stored in our libraries and museums. For this Discovery Grant, we focus on multimodal historical document image processing and understanding. The overall objective of this research program is to develop a holistic data-driven approach for nondestructive analysis and to efficiently understanding the visual information contained in degraded documents. In addition, we aim to contribute the challenging problem of processing large collections of documents acquired by traditional digitization techniques. The short-term objectives are: OBJ 1) Design a unified image decomposition framework for MS/THz document image analysis; OBJ 2) Devise numerical degradation concealing methods in order to provide clean digitized documents; OBJ 3) Develop a data-driven approach for understanding visual information and discovering latent relations in large collections of ancient documents.***The long-term vision is to explore different non-invasive ranges of the electromagnetic spectrum and contribute a new representation space where gray, color, MS and THz spaces are mapped seamlessly in a unified representation. The outcome of this DG will be the first step towards that goal. The long-term objective will be to propel the THz time domain (THz-TD) modality to embrace multilayer structure processing, which will facilitate the analysis of closed books and wrapped manuscripts without opening them. Deciphering content from one layer to another is challenging from the processing steps to the optimal frequency sampling and dimensionality reduction of the THz images.***We propose a multi-level modeling allowing a fundamental understanding of the interaction between ink and paper and external factors within the same document, as well as the understanding of the content within large collections. This, combined with a unified image analysis framework, will provide a holistic approach to understanding ancient, culturally important manuscripts. It will enable students to experience practical and multidisciplinary problems bridging computational vision and physics. Empowered by new THz-TD image modality and application-agnostic data-driven modeling, the proposed research will take this vision one step further and propel research forward in several disciplines, from the natural sciences to the humanities. Thus, the proposed research will lead to a significant knowledge advance for digital cultural heritage; to significant cost savings; and to innovation in the new era of THz-TD document image modality. Outreach to new user communities, the development and promotion of new research methodologies, stewardship of new technologies and data, and building awareness of the potential that data holds for optimizing research outcomes would ensure Canada's research worldwide competitiveness.**