Data-driven modeling for understanding ancient documents from multimodal images

点击翻译按钮获取中文摘要

Nondestructive analysis of documents using multi-spectral (MS) and terahertz (THz) represents a major technical advance in document processing for bringing to life the many ancient documents that are stored in our libraries and museums. For this Discovery Grant, we focus on multimodal historical document image processing and understanding. The overall objective of this research program is to develop a holistic data-driven approach for nondestructive analysis and to efficiently understanding the visual information contained in degraded documents. In addition, we aim to contribute the challenging problem of processing large collections of documents acquired by traditional digitization techniques. The short-term objectives are: OBJ 1) Design a unified image decomposition framework for MS/THz document image analysis; OBJ 2) Devise numerical degradation concealing methods in order to provide clean digitized documents; OBJ 3) Develop a data-driven approach for understanding visual information and discovering latent relations in large collections of ancient documents. The long-term vision is to explore different non-invasive ranges of the electromagnetic spectrum and contribute a new representation space where gray, color, MS and THz spaces are mapped seamlessly in a unified representation. The outcome of this DG will be the first step towards that goal. The long-term objective will be to propel the THz time domain (THz-TD) modality to embrace multilayer structure processing, which will facilitate the analysis of closed books and wrapped manuscripts without opening them. Deciphering content from one layer to another is challenging from the processing steps to the optimal frequency sampling and dimensionality reduction of the THz images. We propose a multi-level modeling allowing a fundamental understanding of the interaction between ink and paper and external factors within the same document, as well as the understanding of the content within large collections. This, combined with a unified image analysis framework, will provide a holistic approach to understanding ancient, culturally important manuscripts. It will enable students to experience practical and multidisciplinary problems bridging computational vision and physics. Empowered by new THz-TD image modality and application-agnostic data-driven modeling, the proposed research will take this vision one step further and propel research forward in several disciplines, from the natural sciences to the humanities. Thus, the proposed research will lead to a significant knowledge advance for digital cultural heritage; to significant cost savings; and to innovation in the new era of THz-TD document image modality. Outreach to new user communities, the development and promotion of new research methodologies, stewardship of new technologies and data, and building awareness of the potential that data holds for optimizing research outcomes would ensure Canada's research worldwide competitiveness.