Data-driven modeling for understanding ancient documents from multimodal images

用于从多模态图像理解古代文献的数据驱动建模

• 批准号: RGPIN-2019-05230
• 负责人: Cheriet, Mohamed
• 金额: $ 4.01万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=715426
• 关键词: Data; driven; modeling; understanding; ancient

Nondestructive analysis of documents using multi-spectral (MS) and terahertz (THz) represents a major technical advance in document processing for bringing to life the many ancient documents that are stored in our libraries and museums. For this Discovery Grant, we focus on multimodal historical document image processing and understanding. The overall objective of this research program is to develop a holistic data-driven approach for nondestructive analysis and to efficiently understanding the visual information contained in degraded documents. In addition, we aim to contribute the challenging problem of processing large collections of documents acquired by traditional digitization techniques. The short-term objectives are: OBJ 1) Design a unified image decomposition framework for MS/THz document image analysis; OBJ 2) Devise numerical degradation concealing methods in order to provide clean digitized documents; OBJ 3) Develop a data-driven approach for understanding visual information and discovering latent relations in large collections of ancient documents. The long-term vision is to explore different non-invasive ranges of the electromagnetic spectrum and contribute a new representation space where gray, color, MS and THz spaces are mapped seamlessly in a unified representation. The outcome of this DG will be the first step towards that goal. The long-term objective will be to propel the THz time domain (THz-TD) modality to embrace multilayer structure processing, which will facilitate the analysis of closed books and wrapped manuscripts without opening them. Deciphering content from one layer to another is challenging from the processing steps to the optimal frequency sampling and dimensionality reduction of the THz images. We propose a multi-level modeling allowing a fundamental understanding of the interaction between ink and paper and external factors within the same document, as well as the understanding of the content within large collections. This, combined with a unified image analysis framework, will provide a holistic approach to understanding ancient, culturally important manuscripts. It will enable students to experience practical and multidisciplinary problems bridging computational vision and physics. Empowered by new THz-TD image modality and application-agnostic data-driven modeling, the proposed research will take this vision one step further and propel research forward in several disciplines, from the natural sciences to the humanities. Thus, the proposed research will lead to a significant knowledge advance for digital cultural heritage; to significant cost savings; and to innovation in the new era of THz-TD document image modality. Outreach to new user communities, the development and promotion of new research methodologies, stewardship of new technologies and data, and building awareness of the potential that data holds for optimizing research outcomes would ensure Canada's research worldwide competitiveness.

利用多光谱（MS）和太赫兹（THz）对文件进行无损分析，代表了文件处理的一项重大技术进步，使保存在图书馆和博物馆的许多古代文件栩栩如生。对于这个发现基金，我们专注于多模式历史文档图像的处理和理解。这项研究计划的总体目标是开发一种全面的数据驱动方法，用于无损分析，并有效地理解退化文件中包含的视觉信息。此外，我们的目标是贡献具有挑战性的问题，处理通过传统的数字化技术获得的大量文件。短期目标：1)设计统一的MS/THz文档图像分解框架；OBJ 2)设计数值退化隐藏方法，提供干净的数字化文件；OBJ 3)开发一种数据驱动的方法，用于理解视觉信息和发现大量古代文献中潜在的关系。