MultiHeart: Fusing CT and MRI with Space-Time Transformation Networks

PHD项目的总体目标是开发一个基于深度学习的多模式时空图谱，这将使不同模式之间的患者心脏解剖快速联合注册成为可能。具体目标是：自动记录患者内部和患者之间的CT和MRI检查。从CT预测运动异常从CT项目描述预测缺血区的范围冠心病是全球主要的死亡原因。冠状动脉CT血管造影(CCTA)和心脏磁共振(CMR)是常用的无创成像方法，用于评估患者是否需要进行有创手术[1]。然而，它们提供了互补的信息，导致需要结合不同模式的信息来改善诊断[2-4]。CCTA提供高分辨率的解剖学信息，可以评估冠状动脉狭窄(狭窄的动脉)，包括斑块、钙、狭窄百分比[5]和限流性狭窄[6]的特征。CMR提供有关心肌功能的信息，包括评估室壁运动异常(来自电影-CMR)、存活、缺血和瘢痕(来自灌注-CMR和延迟增强CMR)[1]。Cine-CMR的高时间分辨率可以观察到室壁运动异常，而灌注-CMR的高对比敏感度可以量化心肌灌注，以及识别微血管疾病[7]。这些信息补充了CCTA对冠状动脉疾病的评估，提供了更完整的图像，有助于指导治疗选择。CCTA和CMR检查的融合将通过对解剖和功能预测因素提供更深入和一致的评估，为现有的分析提供更多的临床实用价值。它还可以更好地识别哪些狭窄是严重的。心肌血流灌注成像可以改善患者特定的边界条件，允许通过冠脉树的总流量与整个心脏或区域水平的测量流量相匹配。这样的联合注册还将允许验证CCTA得出的缺血区范围的计算[8]，并允许更好地参数化CCTA得出的模拟血流模型[9]。除了这一应用，还可以学习联合CT-MRI时空运动图谱。这样的时空运动图谱可以利用电影-MRI中的高时间信息来仅根据CCTA来估计运动异常，而CCTA可能只有有限的时间分辨率(例如，整个心脏周期中的几帧)。本博士项目将专注于开发基于深度学习的多模式时空图谱，这也将使不同模式之间的患者心脏解剖快速联合配准成为可能。这项工作旨在以该领域最近的其他工作为基础，如Atlas-ISTN[10]。一种有效的多模式配准方法将进一步使解剖学和功能的统计总体模型的令人兴奋的应用，利用不同数据集中的补充信息，提供关于正常和病理变异的新见解。

Aim of the PhD Project The overall goal is to develop a deep-learning based multi-modal spatio-temporal atlas, which will enable rapid co-registration of patient cardiac anatomy between different modalities. Specific aims are:Automatically register CT and MRI exams within and between patients.Predict motion abnormalities from CTPredict extent of ischemic region from CTProject descriptionCoronary artery disease is the leading cause of death worldwide. Coronary computed tomography angiography (CCTA) and cardiac magnetic resonance (CMR) are non-invasive imaging methods which are commonly used to evaluate patients to test whether invasive procedures are necessary [1]. However, they provide complimentary information, leading to the need for combining information across modalities for improved diagnosis [2-4]. CCTA provides high-resolution anatomical information allowing for the evaluation of coronary artery stenosis (narrowed arteries), including the characterization of plaque, calcium, percent stenosis [5], and flow limiting stenoses [6]. CMR provides information about myocardial function, including assessment of wall motion abnormalities (from cine-CMR), viability, ischemia and scar (from perfusion-CMR and delayed-enhancement CMR) [1]. The high temporal resolution of cine-CMR allows for wall motion abnormalities to be observed, while the high contrast sensitivity of perfusion-CMR allows for the quantification of myocardial perfusion, as well as identification of microvascular disease [7]. This information complements the assessment of coronary artery disease from CCTA and offers a more complete picture, helping to guide treatment options.Fusion of CCTA and CMR examinations will give added clinical utility to existing analyses, by providing a deeper and concordant evaluation of anatomical and functional predictors. It would also enable better identification of which stenoses are significant. Perfusion imaging-derived myocardial blood flow could provide improved patient-specific boundary conditions, allowing total flow through the coronary tree to be matched with the measured flow at a whole-heart or territory level. Such co-registration would also allow for validation of a CCTA-derived computation of extent of ischaemic region [8] and allow better parametrization of a CCTA-derived simulated perfusion model [9]. In addition to this application, a joint CT-MRI spatio-temporal motion atlas could be learned. Such a spatio-temporal motion atlas could leverage the high temporal information in cine-MRI to inform the estimation of motion abnormalities from CCTA alone, where only limited temporal resolution (e.g., several frames across the cardiac cycle) might be available.This PhD project will focus on the development of a deep-learning based multi-modal spatio-temporal atlas, which will also enable rapid co-registration of patient cardiac anatomy between different modalities. This work aims to build on other recent work in this space, such as Atlas-ISTN [10]. An effective multi-modal registration approach would further enable exciting applications of statistical population modelling of anatomy and function, leveraging the complementary information in heterogeneous datasets providing novel insights about normal and pathological variations.