Upsampling of low-resolution/large-volume 3D tomographic images using generative adversarial neural networks applied to biological anthropology, medical imaging, and evolutionary biology

点击翻译按钮获取中文摘要

'It is vital to remember that information is not knowledge, that knowledge is not wisdom, and that wisdom is not foresight. But information is the first essential step to all of these', Sir Arthur C. Clarke.Large-volume versus high-resolution information is an inherent conundrum in all domains of research that rely on imaging; you can achieve either one, or the other, but never both. The incentives to increase resolution while keeping large context are many, from comprehensive analysis of basic biological phenomena, to maintaining radiation safety standards in imaging, to preserving the integrity of rare specimens of cultural value. However, extrapolating nonexisting spatial information has always been an ill-posed problem because its solution is not unique ' until now with the application of deep learning using neural networks. Although there is ample literature on upscaling single images using super-resolution convolutional neural networks (CNN), recurrent neural networks (RNN) or generative adversarial networks (GAN), most algorithms are designed for (and validated on) synthetically downsampled 2D images. Here we merge expertise in deep learning, tomographic X-ray imaging, biomineralization, biological anthropology, and dental radiology to design and validate an open-ended upsampling algorithm using 3D images of historical (ancient mummies), clinical (dental cone-beam computed tomography) and basic science (bird eggshell) samples. Unique to our project is the availability of original multi-scale 3D image sets that span multiple resolution/volume scales and which can be accurately superimposed (registered) in 3D, and can be expertly segmented (having meaningful features identified and assigned to the target class on a voxel basis). Images are hierarchical, meaning that the neighborhood of local features is as important as the features themselves; indeed, that is the basis of the CNN operation which identifies and labels features based on their context (and the context of context). GANs include a generator algorithm that constructs artificial features, and a discriminator algorithm that compares artificial and true features: iteration of the two leads to convergence, and to construction of realistic artificial spatial information. By applying GAN-CNN to a low-resolution/large-volume image using a high-resolution/low-volume image as ground truth, we will achieve 3D image upsampling ×n. To circumvent the inevitable increase of the data size (×n3) we will implement a parallel segmentation algorithm that reduces voxel depth, because the ultimate objective is the segmentation of upsampled images. Finally, we will explore the limit of image upsampling in 3D using registered series 3D images of ancient mummies, bird eggshells and human craniofacial complex, acquired at multiple magnifications. This open-ended study will bring biological anthropology, zoology and clinical radiology towards the new tier in bioimaging and 3D image analysis.