Data-Driven Approach to Understanding Ancient Manuscripts

古代手稿是全球文化遗产的主要载体，目前世界各地正在对它们进行大规模数字化，以确保它们得到保存，并最终使其内容广泛可及。对这个研究过程至关重要的是图像形式的文件的易读性和对实时文本的访问。已经提出并开发了几种最先进的方法和方法来解决与处理这些手稿相关的挑战。然而，涉及的数据量巨大，而且人类专家反馈和参考数据的高成本和稀缺要求开发以客观和可处理的方式涵盖所有这些方面的基本方法。在本研究中，我们提出了一种基于数据驱动、多层次、自我维持和基于学习的古代手稿计算模式分析新框架，并利用了大量可用的未处理数据。与许多快速推进到特征向量分析的方法不同，我们提出的框架代表了对任务的新视角，它从问题的起点开始，即对象的定义。此外，它利用数据驱动的对象之间关系挖掘来发现它们之间隐藏但持久的链接。这个问题在三个主要层面上得到解决。在图像的最低层次，我们使用视觉对象的空间、光谱、稀疏和基于图形的表示来处理文档图像的自动、数据驱动的增强和恢复，重点是基于空间接近性和局部相似性的基于补丁的表示的空间图。在退化建模方面，我们的方法是对数据点流形执行聚类分析。迁移学习方法也将用于整体地发现、建模和纠正看不见的退化。在第二个层面，即音译，我们在主动学习框架中使用有向图形模型、hmm、无向随机场和基于补丁的表示的空间关系模型来识别手稿图像中的实时文本，从而大大减少对人类专家和很少可用的参考数据的依赖。此外，还考虑了口语翻译中的跨语言适应方法，例如州一级的转换映射，以便允许跨写作风格甚至跨书面语言的适应。最后，在最高层次，即对象之间关系的网络分析（从补丁和文字到手稿和作家），我们搜索连接手稿的“社会网络”。考虑到这种数据驱动的方法在视觉语言处理（VLP）的标题下，我们希望它将为加拿大即将到来的数据管理计划的新研究铺平道路。该研究项目将为使用连贯的、数据驱动的框架和易于处理的解决方案来处理和理解古代手稿带来新的范例。该方法的多层次结构使研究人员能够协同挖掘、建模和解释数字化手稿，所有这些都可以通过数据驱动的方法实现，这在很大程度上是目前该领域所缺乏的。在终身学习概念的推动下，我们的研究使所开发的方法和模型可以在各个馆藏之间进行转移。它将推进图像处理、模式识别、机器学习和网络科学的范式，有可能影响大量手稿，并利用不同的表示空间和指标，以及它们之间的相关关系和联系。

Ancient manuscripts constitute a primary carrier of cultural heritage globally, and they are currently being intensively digitized all over the world to ensure their preservation, and, ultimately, wide accessibility to their content. Critical to this research process are the legibility of the documents in image form and access to live texts. Several state-of-the-art methods and approaches have been proposed and developed to address the challenges associated with processing these manuscripts. However, there is a huge amount of data involved, and also the high cost and scarcity of human expert feedback and reference data call for the development of fundamental approaches that encompass all these aspects in an objective and tractable manner.In this research, we propose one such approach, which is a novel framework for the computational pattern analysis of ancient manuscripts that is data-driven, multilevel, self-sustaining and learning-based, and takes advantage of the large quantities of unprocessed data available. Unlike many approaches, which fast-forward to the analysis of feature vectors, our proposed framework represents a new perspective on the task, which starts from ground zero of the problem, which is the definition of objects. In addition, it leverages the data-driven mining of relations among objects to discover hidden but persistent links between them. The problem is addressed at three main levels. At the lowest level, that of images, we tackle the automatic, data-driven enhancement and restoration of document images using spatial, spectral, sparse and graph-based representations of visual objects with a focus on spatial graphs of patch-based representations empowered with spatial proximity and local similarities. In terms of degradation modeling, our approach is to perform a cluster analysis on the data point manifolds. Transfer learning approaches will be also used to discover, model and correct unseen degradation holistically. At the second level, which is transliteration, we use directed graphical models, HMMs, Undirected Random Fields and spatial relations models of patch-based representations in an active learning framework to recognize the live text in manuscript images, in an effort to drastically reduce dependency on human experts and on reference data that are rarely available. In addition, cross-lingual approaches to adaptation in spoken language translations, such as transform mapping at state level, are also considered, in order to allow adaptation across writing styles and even across written languages. Finally, at the highest level, that of network analysis of the relations among objects (from patches and words to manuscripts and writers), we search for ‘social networks’ linking manuscripts. Considering this data-driven approach under the heading of Visual Language Processing (VLP), we hope that it will pave the way for new research in Canada’s upcoming data stewardship plan.This research program will lead to novel paradigms for processing and understanding ancient manuscripts using coherent, data-driven frameworks with tractable solutions. The multi-level structure of the proposed approach enables researchers to collaboratively mine, model and interpret digitized manuscripts, all of which can be achieved thanks to data-driven approaches, which have been largely absent from the field up to now. Empowered by the concept of life-long learning, our research makes the methods and models developed transferable across collections. It will advance the paradigms of image processing, pattern recognition, machine learning and network science, with the potential to impact huge collections of manuscripts and leverage different representation spaces and metrics, and their associated relationships and links.