Data-driven modeling for understanding ancient documents from multimodal images

使用多光谱（MS）和太赫兹（THz）对文档进行无损分析代表了文档处理的一项重大技术进步，可以使存储在我们图书馆和博物馆中的许多古老文档栩栩如生。对于这个发现补助金，我们专注于多模态历史文档图像处理和理解。该研究计划的总体目标是开发一种整体的数据驱动的方法进行无损分析，并有效地理解退化文档中包含的视觉信息。此外，我们的目标是促进处理传统数字化技术获得的大量文件的挑战性问题。短期目标是：目标1）为MS/THz文档图像分析设计一个统一的图像分解框架;目标2）设计数字退化隐藏方法，以提供干净的数字化文档;目标3）开发一种数据驱动的方法，用于理解视觉信息并发现大量古代文档中的潜在关系。长期愿景是探索电磁频谱的不同非侵入性范围，并贡献一个新的表示空间，其中灰色，彩色，MS和THz空间在统一表示中无缝映射。本次总干事会议的成果将是实现这一目标的第一步。长期目标将是推动太赫兹时域（THz-TD）模式接受多层结构处理，这将有助于在不打开的情况下分析封闭的书籍和包裹的手稿。从一层到另一层的解密内容从处理步骤到THz图像的最佳频率采样和降维都具有挑战性。*我们提出了一种多层次建模，可以从根本上了解同一文档中墨水和纸张与外部因素之间的相互作用，以及了解大型集合中的内容。结合统一的图像分析框架，这将为理解古代文化重要手稿提供一种整体方法。它将使学生能够体验连接计算视觉和物理的实际和多学科问题。在新的THz-TD图像模式和应用不可知的数据驱动建模的支持下，拟议的研究将进一步推动这一愿景，并推动从自然科学到人文科学的多个学科的研究。因此，拟议中的研究将导致数字文化遗产的知识进步;显着节省成本;并在THz-TD文档图像模式的新时代的创新。与新用户社区的联系，新研究方法的开发和推广，新技术和数据的管理，以及对数据优化研究成果的潜力的认识，将确保加拿大的研究在全球范围内的竞争力。

Nondestructive analysis of documents using multi-spectral (MS) and terahertz (THz) represents a major technical advance in document processing for bringing to life the many ancient documents that are stored in our libraries and museums. For this Discovery Grant, we focus on multimodal historical document image processing and understanding. The overall objective of this research program is to develop a holistic data-driven approach for nondestructive analysis and to efficiently understanding the visual information contained in degraded documents. In addition, we aim to contribute the challenging problem of processing large collections of documents acquired by traditional digitization techniques. The short-term objectives are: OBJ 1) Design a unified image decomposition framework for MS/THz document image analysis; OBJ 2) Devise numerical degradation concealing methods in order to provide clean digitized documents; OBJ 3) Develop a data-driven approach for understanding visual information and discovering latent relations in large collections of ancient documents.***The long-term vision is to explore different non-invasive ranges of the electromagnetic spectrum and contribute a new representation space where gray, color, MS and THz spaces are mapped seamlessly in a unified representation. The outcome of this DG will be the first step towards that goal. The long-term objective will be to propel the THz time domain (THz-TD) modality to embrace multilayer structure processing, which will facilitate the analysis of closed books and wrapped manuscripts without opening them. Deciphering content from one layer to another is challenging from the processing steps to the optimal frequency sampling and dimensionality reduction of the THz images.***We propose a multi-level modeling allowing a fundamental understanding of the interaction between ink and paper and external factors within the same document, as well as the understanding of the content within large collections. This, combined with a unified image analysis framework, will provide a holistic approach to understanding ancient, culturally important manuscripts. It will enable students to experience practical and multidisciplinary problems bridging computational vision and physics. Empowered by new THz-TD image modality and application-agnostic data-driven modeling, the proposed research will take this vision one step further and propel research forward in several disciplines, from the natural sciences to the humanities. Thus, the proposed research will lead to a significant knowledge advance for digital cultural heritage; to significant cost savings; and to innovation in the new era of THz-TD document image modality. Outreach to new user communities, the development and promotion of new research methodologies, stewardship of new technologies and data, and building awareness of the potential that data holds for optimizing research outcomes would ensure Canada's research worldwide competitiveness.**