Multimodal analytical imaging of Old Master Paintings: addressing the challenges of registration, mosaic construction and image resolution

古代大师画作的多模态分析成像：解决配准、马赛克构建和图像分辨率的挑战

• 批准号: 2296941
• 金额: --
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2019
• 资助国家: 英国
• 起止时间: 2019 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2296941
• 关键词: Multimodal; analytical; imaging; Old; Master

The cultural heritage sector is experiencing a digital revolution. It is now possible to scan entire paintings with a range of complementary imaging techniques and the National Gallery is one of the few institutions in the world with access to this equipment. It has a leading presence in the field of high-resolution digital imaging of paintings including macrophotography, imaging by X-radiography, HSI and MA-XRF scanning.These multimodal datasets contain a wealth of information which, when properly exploited, offer unprecedented insights into the creation and history of Old Master paintings. However, the generation of huge bodies of data using different imaging techniques poses new signal processing challenges that cannot be addressed with traditional supervised approaches.The expertise needed to adapt and apply such approaches does not exist within the heritage sector: collaboration with experts in image and signal processing is required. Through collaboration, such experts in turn would gain access to a wide range of multimodal data that presents some unique challenges and the possibility to demonstrate the potential of their approaches to a broad audience via the National Gallery's activities. ObjectivesCurrently, in the cultural heritage field, images are registered and/or mosaicked using overlap regions between frames normally manually using tools like Photoshop or custom software. The process needs a lot of user input and varies considerably in how successful it is. With a wider range of imaging modalities now being used which are acquired using different instruments, the limitations of the current approaches are becoming increasingly apparent. The issue is further complicated by the increasing use of spectroscopic imaging techniques (e.g. HSI or MAXRF) generating huge datacubes. This poses new signal processing challenges and require the creation of new signal and image processing tools to facilitate the processing and the interpretation of the acquired data.Research questionsThe main research question is how to develop new registration methods that can automatically extract features which are common to different modalities and which are resilient to variation in spatial resolution as well as other form of inconsistencies and to use them for rigid or semirigid registration. The main focus will be on registration and mosaicking of MA-XRF, HSI and visible images and datacubes but the wider applicability of the methods will also be explored.Methodology and OutcomesThe approach to be adopted will be to initially develop separate algorithms to address the challenges of automatic multimodal registration and mosaicking and of resolution enhancement for multimodal data/images. Attempts will then be made to solve the registration and resolution enhancement problem with a single approach.Beyond the usual outcomes such as the PhD thesis, academic publications, conference presentations and seminars, other tangible outcomes of the research include:- Development of a range of novel algorithms to facilitate the comparison and interpretation of multimodal data and images including algorithms for automatic (but human-in-the-loop based) registration and mosaicking and resolution matching of a wide range of multimodal images (and datacubes); - Production of a range of open-source, user-friendly software tools to allow art curators and other heritage end-users to use the methods developed in the thesis and based on these algorithms; - Development of a tool or graphical user interface to display overlaid image modalities and blend/morph from one modality to another to help visualise if features have been moved, changed or concealed; - Development of a researcher with valuable interdisciplinary research skills, comfortable working across traditional subject boundaries and able to liaise with a range of stakeholders and end-users.

文化遗产部门正在经历一场数字革命。现在可以使用一系列互补的成像技术扫描整幅画作，国家美术馆是世界上为数不多的能够使用这种设备的机构之一。它在高分辨率绘画数字成像领域处于领先地位，包括宏观摄影、X 射线成像、HSI 和 MA-XRF 扫描。这些多模态数据集包含丰富的信息，如果利用得当，可以为早期大师绘画的创作和历史提供前所未有的见解。然而，使用不同成像技术生成大量数据带来了传统监督方法无法解决的新信号处理挑战。遗产部门不存在适应和应用此类方法所需的专业知识：需要与图像和信号处理专家进行合作。通过合作，这些专家反过来将获得广泛的多模式数据，这些数据提出了一些独特的挑战，并有可能通过国家美术馆的活动向广大观众展示他们的方法的潜力。目标目前，在文化遗产领域，通常使用 Photoshop 或自定义软件等工具手动使用帧之间的重叠区域来注册和/或镶嵌图像。该过程需要大量用户输入，并且其成功程度差异很大。随着现在使用更广泛的成像模式（使用不同的仪器获取），当前方法的局限性变得越来越明显。由于越来越多地使用光谱成像技术（例如 HSI 或 MAXRF）生成巨大的数据立方体，该问题变得更加复杂。这提出了新的信号处理挑战，需要创建新的信号和图像处理工具，以促进对所获取数据的处理和解释。 研究问题主要研究问题是如何开发新的配准方法，该方法可以自动提取不同模态共有的特征，并且能够适应空间分辨率的变化以及其他形式的不一致，并将它们用于刚性或半刚性配准。主要重点将是 MA-XRF、HSI 以及可见图像和数据立方体的配准和镶嵌，但也将探索这些方法的更广泛适用性。方法和结果将采用的方法是首先开发单独的算法，以解决自动多模态配准和镶嵌以及多模态数据/图像分辨率增强的挑战。然后将尝试用单一方法解决配准和分辨率增强问题。除了博士论文、学术出版物、会议演讲和研讨会等通常成果之外，研究的其他有形成果包括： - 开发一系列新颖的算法，以促进多模态数据和图像的比较和解释，包括自动（但基于人在环）配准和识别的算法 各种多模态图像（和数据立方体）的镶嵌和分辨率匹配； - 生产一系列开源、用户友好的软件工具，允许艺术策展人和其他遗产最终用户使用论文中开发的基于这些算法的方法； - 开发工具或图形用户界面来显示重叠的图像模态以及从一种模态到另一种模态的混合/变形，以帮助可视化特征是否已被移动、更改或隐藏； - 培养具有宝贵的跨学科研究技能、能够轻松跨传统学科界限工作并能够与一系列利益相关者和最终用户联络的研究人员。