Upsampling of low-resolution/large-volume 3D tomographic images using generative adversarial neural networks applied to biological anthropology, medical imaging, and evolutionary biology

重要的是要记住，信息不是知识，知识不是智慧，智慧不是远见。但信息是实现这一切的第一步。”克拉克爵士如是说。在所有依赖成像的研究领域，大容量与高分辨率信息是一个固有的难题；你可以实现其中一个，也可以实现另一个，但不可能两者兼得。在保持大背景的同时提高分辨率的动机有很多，从对基本生物现象的全面分析，到维持成像方面的辐射安全标准，到保护具有文化价值的稀有标本的完整性。然而，外推不存在的空间信息一直是一个不适定问题，因为它的解决方案并不是唯一的，直到现在使用神经网络的深度学习应用。尽管关于使用超分辨率卷积神经网络（CNN）、循环神经网络（RNN）或生成对抗网络（GAN）升级单个图像的文献很多，但大多数算法都是为合成下采样的2D图像设计的（并在其上进行了验证）。在这里，我们融合了深度学习、断层x射线成像、生物矿化、生物人类学和牙科放射学方面的专业知识，设计并验证了一个开放式的上采样算法，该算法使用历史（古代木乃伊）、临床（牙科锥束计算机断层扫描）和基础科学（鸟蛋壳）样本的3D图像。我们项目的独特之处在于原始的多尺度3D图像集的可用性，这些图像集跨越多个分辨率/体积尺度，可以在3D中精确地叠加（注册），并且可以熟练地分割（在体素的基础上识别有意义的特征并将其分配给目标类）。图像是分层的，这意味着局部特征的邻域与特征本身一样重要；事实上，这是CNN操作的基础，它根据上下文（以及上下文的上下文）识别和标记特征。GANs包括构建人工特征的生成器算法，以及比较人工特征和真实特征的判别器算法：两者的迭代导致收敛，并构建逼真的人工空间信息。通过将GAN-CNN应用于低分辨率/大体积图像，使用高分辨率/小体积图像作为地面真值，我们将实现3D图像上采样×n。为了避免不可避免地增加数据大小（×n3），我们将实现一个并行分割算法，减少体素深度，因为最终目标是分割上采样图像。最后，我们将探索在三维图像上采样的限制使用注册系列3D图像的古代木乃伊，鸟蛋壳和人类颅面复合体，在多次放大获得。这项开放式研究将把生物人类学、动物学和临床放射学带入生物成像和3D图像分析的新阶段。

'It is vital to remember that information is not knowledge, that knowledge is not wisdom, and that wisdom is not foresight. But information is the first essential step to all of these', Sir Arthur C. Clarke.Large-volume versus high-resolution information is an inherent conundrum in all domains of research that rely on imaging; you can achieve either one, or the other, but never both. The incentives to increase resolution while keeping large context are many, from comprehensive analysis of basic biological phenomena, to maintaining radiation safety standards in imaging, to preserving the integrity of rare specimens of cultural value. However, extrapolating nonexisting spatial information has always been an ill-posed problem because its solution is not unique ' until now with the application of deep learning using neural networks. Although there is ample literature on upscaling single images using super-resolution convolutional neural networks (CNN), recurrent neural networks (RNN) or generative adversarial networks (GAN), most algorithms are designed for (and validated on) synthetically downsampled 2D images. Here we merge expertise in deep learning, tomographic X-ray imaging, biomineralization, biological anthropology, and dental radiology to design and validate an open-ended upsampling algorithm using 3D images of historical (ancient mummies), clinical (dental cone-beam computed tomography) and basic science (bird eggshell) samples. Unique to our project is the availability of original multi-scale 3D image sets that span multiple resolution/volume scales and which can be accurately superimposed (registered) in 3D, and can be expertly segmented (having meaningful features identified and assigned to the target class on a voxel basis). Images are hierarchical, meaning that the neighborhood of local features is as important as the features themselves; indeed, that is the basis of the CNN operation which identifies and labels features based on their context (and the context of context). GANs include a generator algorithm that constructs artificial features, and a discriminator algorithm that compares artificial and true features: iteration of the two leads to convergence, and to construction of realistic artificial spatial information. By applying GAN-CNN to a low-resolution/large-volume image using a high-resolution/low-volume image as ground truth, we will achieve 3D image upsampling ×n. To circumvent the inevitable increase of the data size (×n3) we will implement a parallel segmentation algorithm that reduces voxel depth, because the ultimate objective is the segmentation of upsampled images. Finally, we will explore the limit of image upsampling in 3D using registered series 3D images of ancient mummies, bird eggshells and human craniofacial complex, acquired at multiple magnifications. This open-ended study will bring biological anthropology, zoology and clinical radiology towards the new tier in bioimaging and 3D image analysis.