Combining Image and Graph-based Neural Networks for Handwriting Recognition

这项提议旨在为分类或检索等文件分析任务开发模型、数据合成和培训方法，以便在不需要手动创建注释的情况下应用深度学习模型。特别是，考虑到历史文献或低资源语言，对标记数据的大量需求阻碍了基于学习的方法的应用。拟议项目中的一个关键因素是对笔迹结构性质的开发。除了视觉外观外，底层结构还可以表示为图形。几何深度学习的最新发展使得在模型设计层面上整合这一结构元素成为可能。此外，显式建模几何组件作为正则化的一种形式，增加了所开发模型的适应性和泛化能力。然后，在尽可能少的监督下执行组合视觉和几何信息的模型的训练。因此，将开发也包括结构表示的数据合成方法。这个项目通过开发显式建模手写几何成分的方法，扩展了基于深度学习的文档分析的最新技术。随之而来的培训数据需求的减少打开了一系列不同的应用领域和任务。

This proposal aims to develop models, data synthesis and training methods for document analysis tasks such as classification or retrieval that allow the application of deep learning models without the need for manually created annotations. Especially, considering historic documents or low resource language the heavy demand for labeled data hinders the application of learning-based methodology. A key factor in the proposed project is the exploitation of the structural nature of handwriting. In Addition to the visual appearance, the underlying structure can represented as a graph. Recent developments in geometric deep learning allow to integrate this structural element on the level of model design. Additionally, explicitly modeling the geometric component serves as a form of regularization, increasing adaptability and the generalization capabilities of the developed models. Training of a model combining visual and geometrical information is then performed with as little supervision as possible. Therefore, data synthesis approaches will be developed that also include structural representations. This project extends the state-of-the art in deep learning based document analysis by developing methods that explicitly model the geometric components of handwriting. The resulting reduction of training data demand opens up a diverse set of application areas and tasks.