Data-Driven Approach to Understanding Ancient Manuscripts

古代手稿是全球文化遗产的主要载体，目前正在世界各地对其进行密集的数字化，以确保其保存，并最终确保其内容的广泛获取。这一研究过程的关键是图像文件的易读性和获取现场文本的机会。已经提出和开发了几种最先进的方法和途径，以解决与处理这些手稿有关的挑战。然而，这涉及到大量的数据，而且人类专家反馈和参考数据的高成本和稀缺性要求以客观和易处理的方式开发涵盖所有这些方面的基本方法。在本研究中，我们提出了一种新的方法，它是一种数据驱动的、多层次的、自我维持的和基于学习的古代手稿计算模式分析框架，并利用现有的大量未处理的数据。与许多快进到特征向量分析的方法不同，我们提出的框架代表了对该任务的新视角，从问题的起点开始，即对象的定义。此外，它利用对对象之间关系的数据驱动挖掘来发现它们之间隐藏但持久的链接。这个问题主要从三个层面来解决。在图像的最低层，我们使用视觉对象的空间、光谱、稀疏和基于图形的表示来处理文档图像的自动、数据驱动的增强和恢复，重点是基于面片的表示的空间图形，该空间图形具有空间邻近性和局部相似性。在退化建模方面，我们的方法是对数据点流形进行聚类分析。迁移学习方法也将被用来从整体上发现、模拟和纠正看不见的退化。在第二个层次，即音译，我们使用主动学习框架中的有向图形模型、隐马尔可夫模型、无向随机场模型和基于块的表示的空间关系模型来识别手稿图像中的活文本，以努力大大减少对人类专家和参考数据的依赖。此外，还考虑了口语翻译中的跨语言适应方法，如州一级的转换映射，以便实现跨写作风格甚至跨书面语言的适应。最后，在网络分析的最高层面上，即对对象之间的关系进行网络分析(从补丁和文字到手稿和作者)，寻找链接手稿的社交网络。考虑到视觉语言处理(VLP)下的这种数据驱动的方法，我们希望它将为加拿大即将到来的数据管理计划中的新研究铺平道路。这个研究计划将导致使用连贯的、数据驱动的框架和易于处理的解决方案来处理和理解古代手稿的新范式。拟议方法的多层次结构使研究人员能够协作挖掘、建模和解释数字化手稿，所有这些都可以归功于数据驱动的方法，而到目前为止，该领域基本上没有数据驱动的方法。在终身学习理念的支持下，我们的研究使开发的方法和模型可以在不同收藏之间移植。它将推动图像处理、模式识别、机器学习和网络科学的范式，有可能对大量手稿产生影响，并利用不同的表现空间和指标，以及它们相关的关系和联系。

Ancient manuscripts constitute a primary carrier of cultural heritage globally, and they are currently being intensively digitized all over the world to ensure their preservation, and, ultimately, wide accessibility to their content. Critical to this research process are the legibility of the documents in image form and access to live texts. Several state-of-the-art methods and approaches have been proposed and developed to address the challenges associated with processing these manuscripts. However, there is a huge amount of data involved, and also the high cost and scarcity of human expert feedback and reference data call for the development of fundamental approaches that encompass all these aspects in an objective and tractable manner.In this research, we propose one such approach, which is a novel framework for the computational pattern analysis of ancient manuscripts that is data-driven, multilevel, self-sustaining and learning-based, and takes advantage of the large quantities of unprocessed data available. Unlike many approaches, which fast-forward to the analysis of feature vectors, our proposed framework represents a new perspective on the task, which starts from ground zero of the problem, which is the definition of objects. In addition, it leverages the data-driven mining of relations among objects to discover hidden but persistent links between them. The problem is addressed at three main levels. At the lowest level, that of images, we tackle the automatic, data-driven enhancement and restoration of document images using spatial, spectral, sparse and graph-based representations of visual objects with a focus on spatial graphs of patch-based representations empowered with spatial proximity and local similarities. In terms of degradation modeling, our approach is to perform a cluster analysis on the data point manifolds. Transfer learning approaches will be also used to discover, model and correct unseen degradation holistically. At the second level, which is transliteration, we use directed graphical models, HMMs, Undirected Random Fields and spatial relations models of patch-based representations in an active learning framework to recognize the live text in manuscript images, in an effort to drastically reduce dependency on human experts and on reference data that are rarely available. In addition, cross-lingual approaches to adaptation in spoken language translations, such as transform mapping at state level, are also considered, in order to allow adaptation across writing styles and even across written languages. Finally, at the highest level, that of network analysis of the relations among objects (from patches and words to manuscripts and writers), we search for ‘social networks’ linking manuscripts. Considering this data-driven approach under the heading of Visual Language Processing (VLP), we hope that it will pave the way for new research in Canada’s upcoming data stewardship plan.This research program will lead to novel paradigms for processing and understanding ancient manuscripts using coherent, data-driven frameworks with tractable solutions. The multi-level structure of the proposed approach enables researchers to collaboratively mine, model and interpret digitized manuscripts, all of which can be achieved thanks to data-driven approaches, which have been largely absent from the field up to now. Empowered by the concept of life-long learning, our research makes the methods and models developed transferable across collections. It will advance the paradigms of image processing, pattern recognition, machine learning and network science, with the potential to impact huge collections of manuscripts and leverage different representation spaces and metrics, and their associated relationships and links.