Data-driven modeling for understanding ancient documents from multimodal images

利用多光谱（MS）和太赫兹（THz）对文件进行无损分析，代表了文件处理的一项重大技术进步，使保存在图书馆和博物馆的许多古代文件栩栩如生。对于这个发现基金，我们专注于多模式历史文档图像的处理和理解。这项研究计划的总体目标是开发一种全面的数据驱动方法，用于无损分析，并有效地理解退化文件中包含的视觉信息。此外，我们的目标是贡献具有挑战性的问题，处理通过传统的数字化技术获得的大量文件。短期目标：1)设计统一的MS/THz文档图像分解框架；OBJ 2)设计数值退化隐藏方法，提供干净的数字化文件；OBJ 3)开发一种数据驱动的方法，用于理解视觉信息和发现大量古代文献中潜在的关系。***长期愿景是探索电磁频谱的不同非侵入范围，并提供一个新的表示空间，其中灰色，颜色，MS和THz空间在统一表示中无缝映射。这次会议的结果将是实现这一目标的第一步。长期目标将是推动太赫兹时域（THz- td）模式采用多层结构处理，这将有助于分析封闭的书籍和包裹的手稿，而无需打开它们。从处理步骤到太赫兹图像的最佳频率采样和降维，将内容从一层解密到另一层是具有挑战性的。***我们提出了一个多层次的模型，允许对同一文档中墨水和纸张之间的相互作用以及外部因素的基本理解，以及对大型集合中的内容的理解。这与统一的图像分析框架相结合，将提供一种全面的方法来理解古代的、具有重要文化意义的手稿。它将使学生体验连接计算视觉和物理学的实际和多学科问题。通过新的太赫兹- td图像模式和与应用无关的数据驱动建模，拟议的研究将使这一愿景更进一步，并推动从自然科学到人文科学等多个学科的研究向前发展。因此，拟议的研究将导致数字文化遗产的重大知识进步；大幅节省成本；以及创新新时代的THz-TD文献图像形态。拓展新的用户群体，开发和推广新的研究方法，管理新技术和数据，以及建立对数据在优化研究成果方面的潜力的认识，将确保加拿大的研究在全球范围内具有竞争力

Nondestructive analysis of documents using multi-spectral (MS) and terahertz (THz) represents a major technical advance in document processing for bringing to life the many ancient documents that are stored in our libraries and museums. For this Discovery Grant, we focus on multimodal historical document image processing and understanding. The overall objective of this research program is to develop a holistic data-driven approach for nondestructive analysis and to efficiently understanding the visual information contained in degraded documents. In addition, we aim to contribute the challenging problem of processing large collections of documents acquired by traditional digitization techniques. The short-term objectives are: OBJ 1) Design a unified image decomposition framework for MS/THz document image analysis; OBJ 2) Devise numerical degradation concealing methods in order to provide clean digitized documents; OBJ 3) Develop a data-driven approach for understanding visual information and discovering latent relations in large collections of ancient documents.***The long-term vision is to explore different non-invasive ranges of the electromagnetic spectrum and contribute a new representation space where gray, color, MS and THz spaces are mapped seamlessly in a unified representation. The outcome of this DG will be the first step towards that goal. The long-term objective will be to propel the THz time domain (THz-TD) modality to embrace multilayer structure processing, which will facilitate the analysis of closed books and wrapped manuscripts without opening them. Deciphering content from one layer to another is challenging from the processing steps to the optimal frequency sampling and dimensionality reduction of the THz images.***We propose a multi-level modeling allowing a fundamental understanding of the interaction between ink and paper and external factors within the same document, as well as the understanding of the content within large collections. This, combined with a unified image analysis framework, will provide a holistic approach to understanding ancient, culturally important manuscripts. It will enable students to experience practical and multidisciplinary problems bridging computational vision and physics. Empowered by new THz-TD image modality and application-agnostic data-driven modeling, the proposed research will take this vision one step further and propel research forward in several disciplines, from the natural sciences to the humanities. Thus, the proposed research will lead to a significant knowledge advance for digital cultural heritage; to significant cost savings; and to innovation in the new era of THz-TD document image modality. Outreach to new user communities, the development and promotion of new research methodologies, stewardship of new technologies and data, and building awareness of the potential that data holds for optimizing research outcomes would ensure Canada's research worldwide competitiveness.**