MultiHeart: Fusing CT and MRI with Space-Time Transformation Networks

总体目标是开发一个基于深度学习的多模态时空图谱，这将使不同模态之间的患者心脏解剖快速共同注册成为可能。具体目标是：自动登记患者内部和患者之间的CT和MRI检查。从ct预测运动异常从ct预测缺血区域的范围项目描述冠状动脉疾病是世界范围内死亡的主要原因。冠状动脉计算机断层血管造影（CCTA）和心脏磁共振（CMR）是非侵入性成像方法，通常用于评估患者是否需要进行侵入性手术。然而，它们提供了补充信息，导致需要将不同模式的信息结合起来以改进诊断[2-4]。CCTA提供高分辨率的解剖信息，用于评估冠状动脉狭窄（狭窄的动脉），包括斑块、钙、狭窄百分比[5]和限流狭窄[6]的特征。CMR提供心肌功能信息，包括评估壁运动异常（来自cine-CMR）、活力、缺血和疤痕（来自灌注-CMR和延迟增强CMR）[1]。cine-CMR的高时间分辨率可以观察到壁运动异常，而灌注- cmr的高对比度灵敏度可以量化心肌灌注，以及识别微血管疾病[7]。这一信息补充了CCTA对冠状动脉疾病的评估，并提供了更完整的图像，有助于指导治疗方案。CCTA和CMR检查的融合将为现有的分析提供更深入和一致的解剖和功能预测评估，从而增加临床实用性。它还可以更好地识别哪些狭窄是严重的。灌注成像衍生的心肌血流可以提供更好的患者特异性边界条件，允许通过冠状动脉树的总流量与整个心脏或区域水平的测量流量相匹配。这种共配准也将允许验证ccta衍生的缺血区域[8]范围计算，并允许更好地参数化ccta衍生的模拟灌注模型[9]。除此之外，还可以学习关节CT-MRI时空运动图谱。这样的时空运动图谱可以利用电影mri中的高时间信息来单独从CCTA中估计运动异常，其中只有有限的时间分辨率（例如，跨心脏周期的几个帧）可能可用。该博士项目将专注于基于深度学习的多模态时空图谱的开发，这也将使不同模态之间的患者心脏解剖快速共同注册成为可能。这项工作旨在建立在该领域最近的其他工作的基础上，例如Atlas-ISTN[10]。一种有效的多模态注册方法将进一步使解剖学和功能的统计群体建模的令人兴奋的应用成为可能，利用异构数据集中的互补信息，提供关于正常和病理变化的新见解。

Aim of the PhD Project The overall goal is to develop a deep-learning based multi-modal spatio-temporal atlas, which will enable rapid co-registration of patient cardiac anatomy between different modalities. Specific aims are:Automatically register CT and MRI exams within and between patients.Predict motion abnormalities from CTPredict extent of ischemic region from CTProject descriptionCoronary artery disease is the leading cause of death worldwide. Coronary computed tomography angiography (CCTA) and cardiac magnetic resonance (CMR) are non-invasive imaging methods which are commonly used to evaluate patients to test whether invasive procedures are necessary [1]. However, they provide complimentary information, leading to the need for combining information across modalities for improved diagnosis [2-4]. CCTA provides high-resolution anatomical information allowing for the evaluation of coronary artery stenosis (narrowed arteries), including the characterization of plaque, calcium, percent stenosis [5], and flow limiting stenoses [6]. CMR provides information about myocardial function, including assessment of wall motion abnormalities (from cine-CMR), viability, ischemia and scar (from perfusion-CMR and delayed-enhancement CMR) [1]. The high temporal resolution of cine-CMR allows for wall motion abnormalities to be observed, while the high contrast sensitivity of perfusion-CMR allows for the quantification of myocardial perfusion, as well as identification of microvascular disease [7]. This information complements the assessment of coronary artery disease from CCTA and offers a more complete picture, helping to guide treatment options.Fusion of CCTA and CMR examinations will give added clinical utility to existing analyses, by providing a deeper and concordant evaluation of anatomical and functional predictors. It would also enable better identification of which stenoses are significant. Perfusion imaging-derived myocardial blood flow could provide improved patient-specific boundary conditions, allowing total flow through the coronary tree to be matched with the measured flow at a whole-heart or territory level. Such co-registration would also allow for validation of a CCTA-derived computation of extent of ischaemic region [8] and allow better parametrization of a CCTA-derived simulated perfusion model [9]. In addition to this application, a joint CT-MRI spatio-temporal motion atlas could be learned. Such a spatio-temporal motion atlas could leverage the high temporal information in cine-MRI to inform the estimation of motion abnormalities from CCTA alone, where only limited temporal resolution (e.g., several frames across the cardiac cycle) might be available.This PhD project will focus on the development of a deep-learning based multi-modal spatio-temporal atlas, which will also enable rapid co-registration of patient cardiac anatomy between different modalities. This work aims to build on other recent work in this space, such as Atlas-ISTN [10]. An effective multi-modal registration approach would further enable exciting applications of statistical population modelling of anatomy and function, leveraging the complementary information in heterogeneous datasets providing novel insights about normal and pathological variations.