Data-Driven Approach to Understanding Ancient Manuscripts

古代手稿是全球文化遗产的主要载体，目前它们正在世界各地被密集数字化，以确保它们的保存，并最终确保其内容的广泛获取。该研究过程的关键是图像形式文档的易读性和对实时文本的访问。已经提出并开发了几种最先进的方法和方法来解决与处理这些手稿相关的挑战。然而，涉及的数据量巨大，而且人类专家反馈和参考数据的成本高且稀缺，因此需要开发以客观且易于处理的方式涵盖所有这些方面的基本方法。 在这项研究中，我们提出了一种这样的方法，这是一种用于古代手稿计算模式分析的新颖框架，该框架是数据驱动的、多层次的、自我维持的和基于学习的，并利用大量未处理的可用数据。与许多快进到特征向量分析的方法不同，我们提出的框架代表了任务的新视角，它从问题的零开始，即对象的定义。此外，它还利用数据驱动的对象之间关系挖掘来发现它们之间隐藏但持久的链接。该问题主要在三个层面上得到解决。在图像的最低级别，我们使用视觉对象的空间、光谱、稀疏和基于图形的表示来解决文档图像的自动、数据驱动的增强和恢复，重点关注具有空间邻近性和局部相似性的基于块的表示的空间图。在退化建模方面，我们的方法是对数据点流形进行聚类分析。迁移学习方法还将用于整体发现、建模和纠正看不见的退化。在第二个层面，即音译，我们在主动学习框架中使用有向图形模型、HMM、无向随机场和基于块表示的空间关系模型来识别手稿图像中的实时文本，以大大减少对人类专家和很少可用的参考数据的依赖。此外，还考虑了口语翻译中的跨语言适应方法，例如状态级别的变换映射，以便允许跨写作风格甚至跨书面语言的适应。最后，在最高层次上，即对象之间关系的网络分析（从补丁和单词到手稿和作者），我们搜索链接手稿的“社交网络”。考虑到视觉语言处理（VLP）标题下的这种数据驱动方法，我们希望它能为加拿大即将推出的数据管理计划的新研究铺平道路。 该研究项目将使用连贯的、数据驱动的框架和易于处理的解决方案，形成处理和理解古代手稿的新范式。所提出方法的多层次结构使研究人员能够协作挖掘、建模和解释数字化手稿，所有这些都可以通过数据驱动的方法来实现，而迄今为止该领域基本上还没有这种方法。在终身学习概念的支持下，我们的研究使开发的方法和模型可以跨馆藏转移。它将推进图像处理、模式识别、机器学习和网络科学的范式，有可能影响大量手稿收藏，并利用不同的表示空间和指标及其相关的关系和链接。

Ancient manuscripts constitute a primary carrier of cultural heritage globally, and they are currently being intensively digitized all over the world to ensure their preservation, and, ultimately, wide accessibility to their content. Critical to this research process are the legibility of the documents in image form and access to live texts. Several state-of-the-art methods and approaches have been proposed and developed to address the challenges associated with processing these manuscripts. However, there is a huge amount of data involved, and also the high cost and scarcity of human expert feedback and reference data call for the development of fundamental approaches that encompass all these aspects in an objective and tractable manner. In this research, we propose one such approach, which is a novel framework for the computational pattern analysis of ancient manuscripts that is data-driven, multilevel, self-sustaining and learning-based, and takes advantage of the large quantities of unprocessed data available. Unlike many approaches, which fast-forward to the analysis of feature vectors, our proposed framework represents a new perspective on the task, which starts from ground zero of the problem, which is the definition of objects. In addition, it leverages the data-driven mining of relations among objects to discover hidden but persistent links between them. The problem is addressed at three main levels. At the lowest level, that of images, we tackle the automatic, data-driven enhancement and restoration of document images using spatial, spectral, sparse and graph-based representations of visual objects with a focus on spatial graphs of patch-based representations empowered with spatial proximity and local similarities. In terms of degradation modeling, our approach is to perform a cluster analysis on the data point manifolds. Transfer learning approaches will be also used to discover, model and correct unseen degradation holistically. At the second level, which is transliteration, we use directed graphical models, HMMs, Undirected Random Fields and spatial relations models of patch-based representations in an active learning framework to recognize the live text in manuscript images, in an effort to drastically reduce dependency on human experts and on reference data that are rarely available. In addition, cross-lingual approaches to adaptation in spoken language translations, such as transform mapping at state level, are also considered, in order to allow adaptation across writing styles and even across written languages. Finally, at the highest level, that of network analysis of the relations among objects (from patches and words to manuscripts and writers), we search for ‘social networks’ linking manuscripts. Considering this data-driven approach under the heading of Visual Language Processing (VLP), we hope that it will pave the way for new research in Canada’s upcoming data stewardship plan. This research program will lead to novel paradigms for processing and understanding ancient manuscripts using coherent, data-driven frameworks with tractable solutions. The multi-level structure of the proposed approach enables researchers to collaboratively mine, model and interpret digitized manuscripts, all of which can be achieved thanks to data-driven approaches, which have been largely absent from the field up to now. Empowered by the concept of life-long learning, our research makes the methods and models developed transferable across collections. It will advance the paradigms of image processing, pattern recognition, machine learning and network science, with the potential to impact huge collections of manuscripts and leverage different representation spaces and metrics, and their associated relationships and links.